QoS
- Quality of Service
- marking, scheduling and dropping of network traffic
- latency = time it takes for packets to travel from source to destination
- propagation delay = time it takes the signal to propagate through the medium; physical constraint; in fiber about 2/3 of speed of light
- serialization delay = time it takes to put the bits onto a link; higher the link speed, lower the serialization delay; = packet size / link speed
- processing delay = time it takes the network device to transfer the packet from the input queue to the output queue; depends on CPU, load, architecture, configuration...
models
- best effort - no QoS
- IntServ = Integrated Services
- connection-oriented model
- end-to-end path of reserved QoS
- RSVP = Resource Reservation Protocol
- does not scale because the network has to maintain too much state
- DiffServ = Differentiated Services
- traffic divided into classes
- PHB = Per-Hop Behavior
- not considered end-to-end
- scalable - the network edge performs classification, marking, policing and shaping, while the core has to perform only lookups
classification
- dividing traffic into classes to be later used in marking
- layer 1 - physical (sub)interface or port
- layer 2 - MAC address; CoS = Class of Service
- layer 2.5 - MPLS experimental (EXP) bits
- layer 3 - IP address, IPP (IP Precedence) or DSCP = Differentiated Services Code Point
- layer 4 - ports
- layer 7 - apps
marking
- changes a traffic descriptor in the header
- trust boundary - to accept or to discard the marks; usually at the access switch or the phone
- layer 2 - only possible on trunks (802.1q or ISL):
- 802.1q tag contains fields: 3-bit PCP (Priority Code Point) and 1-bit DEI (Drop Eligible Indicator)
- CoS defined by 802.1p; 0 lowest, 7 highest
- usually mapped to Layer 3 marking
- layer 3:
- IPv4 header field ToS (Type of Service) - 8 bits: first 3 are IPP(~CS [Class Selector]), the rest unused
- DiffServ: 6-bit DSCP + 2-bit ECN (Explicit Congestion Notification)
- DF = Default Forwarding - DSCP 0
- AF = Assured Forwarding - AFxy xxxyy0; DSCP = 8x + 2y (higher y = higher drop probability)
- EF = Expedited Forwarding - DSCP 46 ~ AF53
- scavenger class - less than best effort (CS1)
tools
admission control
- policing - dropping or (re)marking traffic (usually ingress and usually ISPs)
- shaping - delaying traffic to smooth out bursts (usually egress, usually customers - delay is better than drop)
token bucket algorithm
- CAR = Committed Access Rate - physical port speed; the interface always transmits at AR, but QoS is used to manage the average data rate
- CIR = Committed Information Rate - the policed data rate
- Tc = Time Committed (Committed Time Interval) - time interval in ms to emit traffic bursts (always at AR); Tc = Bc[bits]/(CIR[bps]x1000)
- Bc = Burst Committed (Committed Burst Size) - amount of bytes that could be sent every Tc
- token = 1 B or 8 bits
- types of policers: single-rate two-color marker/policer, single-rate three-color marker/policer (srTCM), dual-rate three-color marker/policer(drTCM)
- the first rate (CIR) is the guaranteed minimum average data rate, the second (PIR) is the absolute maximum allowed
- the first color is conform, the second exceed, the third violate
- Be = Burst Excess - amount of bytes over Bc that could be sent during Tc; accumulated from the excess of CIR (srTCM) PIR (drTCM)
- single-rate two-color: one bucket size Bc filled at the rate CIR; conform/exceed
- single-rate three-color: one bucket size Bc filled at the rate CIR; one bucket size Be filled by the overfill of Bc; conform/exceed/violate
- dual-rate three-color (straight cut both up and down): one bucket size Bc filled at the rate CIR; one bucket size Be filled at the rate PIR; conform/exceed/violate; the logic is reverse (violate -> exceed -> conform)
congestion management
- FIFO = First-In First-Out
- flush = counts SPDs (Selective Packet Discard) - a mechanism that quickly drops low priority packets when the CPU is overloaded in order to save some processing capacity for high priority packets; ensures that important control packets, such as routing updates and keepalives, are not dropped when the IP input queue is full
- FQ = Fair Queuing - transmits equal number of bytes
- WFQ = Weighted FQ - weights the transmitted traffic
- CBWFQ = Class-Based WFQ - user defines the classes instead of IOS
- PQ = Priority Queue - always emptied out first
- LLQ = Low-Latency Queuing - PQ + policer to prevent BW starvation
congestion avoidance
- RED = Random Early Detection/Drop - dropping packets before the queue is full to prevent tail drop
- WRED = Weighted RED
- tail drop - dropping all packets when the output queue is full, results in a saw-tooth traffic pattern due to TCP response to drops
- congestion management offloaded to the end hosts and their upper layer protocols
HQF
- Hierarchical Queing Framework - application of QoS on the CLI, allows multiple QoS methods per interface per direction
- MQC (Kissco) = Modular QoS CLI - configuring the QoS in "modules" (globally, so they can be reused) instead of per interface
- components: class map, policy map, service policy
- it's easier to start backwards when interpreting
- previously CBWFQ
lab
- I'm using EVE-NG and iperf on Linux machines
- the routers used are Cisco, Juniper, Mikrotik, Arista and HP; chosen due to vendor prevalence and usage at my job
- two labs: marking and rate limiting
- the topology is pretty simple, Linux machines with iperf on both sides of the router
- max BW caps around 18 Gbps both way when the server and client are connected directly with no routers in between:
user@ONE:~$ iperf -c 2.2.2.2 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 2.2.2.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 2] local 1.1.1.2 port 5001 connected with 2.2.2.2 port 33636
[ 1] local 1.1.1.2 port 50622 connected with 2.2.2.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 1] 0.0000-10.0139 sec 21.9 GBytes 18.8 Gbits/sec
[ 2] 0.0000-10.0014 sec 21.8 GBytes 18.7 Gbits/sec
user@ONE:~$
Cisco
ip access-list extended ICMP
permit icmp any any
deny ip any any
class-map match-all CM
match access-group name ICMP
policy-map PM
class CM
set dscp cs2
interface GigabitEthernet0/1
service-policy output PM
- single-rate two-color policer:
[ 2] 40.0000-45.0000 sec 1.23 MBytes 2.06 Mbits/sec
[ 1] 40.0000-45.0000 sec 1.12 MBytes 1.89 Mbits/sec
policy-map PM
class class-default
police cir 500000 bc 15625 conform-action transmit exceed-action drop
interface GigabitEthernet0/1
service-policy output PM
[ 1] 730.0000-735.0000 sec 1.25 MBytes 2.10 Mbits/sec
[ 2] 730.0000-735.0000 sec 228 KBytes 373 Kbits/sec
Juniper
firewall {
family inet {
filter ICMP {
term 1 {
from {
protocol icmp;
}
then {
accept;
dscp cs2;
}
}
term 2 {
then {
discard;
}
}
}
}
}
interfaces {
ge-0/0/1 {
unit 0 {
family inet {
filter {
output ICMP;
}
}
}
}
}
- single-rate two-color policer:
[ 1] 1010.0000-1015.0000 sec 36.9 MBytes 61.9 Mbits/sec
[ 2] 1010.0000-1015.0000 sec 19.7 MBytes 33.1 Mbits/sec
firewall {
family inet {
filter POL-FILTER {
term 1 {
then {
policer POL;
accept;
}
}
}
}
policer POL {
if-exceeding {
bandwidth-limit 15m;
burst-size-limit 1875000;
}
then discard;
}
}
interfaces {
ge-0/0/1 {
unit 0 {
family inet {
filter {
input POL-FILTER;
}
}
}
}
}
[ 2] 1535.0000-1540.0000 sec 47.5 MBytes 79.7 Mbits/sec
[ 1] 1535.0000-1540.0000 sec 8.38 MBytes 14.1 Mbits/sec
Mikrotik
/ip firewall mangle
add action=change-dscp chain=forward new-dscp=16 out-interface=ether1 protocol=icmp
- shaping (RouterOS doesn't have proper policing):
[ 2] 20.0000-25.0000 sec 567 KBytes 929 Kbits/sec
[ 1] 20.0000-25.0000 sec 637 KBytes 1.04 Mbits/sec
/queue simple
add max-limit=500k/50M name=SHAPE target=ether1
[ 2] 845.0000-850.0000 sec 255 KBytes 418 Kbits/sec
[ 1] 845.0000-850.0000 sec 542 KBytes 887 Kbits/sec
HPE
acl advanced name ICMP
rule 0 permit icmp
rule 5 deny ip
traffic classifier 1 operator and
if-match acl name ICMP
traffic behavior 1
remark dscp cs2
qos policy 1
classifier 1 behavior 1
interface GigabitEthernet1/0
qos apply policy 1 outbound
- single-rate two-color policer:
[ 2] 190.0000-195.0000 sec 8.16 MBytes 13.7 Mbits/sec
[ 1] 190.0000-195.0000 sec 8.12 MBytes 13.6 Mbits/sec
traffic classifier 1 operator and
if-match any
traffic behavior 1
car cir 1000 cbs 62500 ebs 0 green pass red discard yellow pass
qos policy 1
classifier 1 behavior 1
interface GigabitEthernet1/0
qos apply policy 1 outbound
[ 2] 120.0000-125.0000 sec 276 KBytes 452 Kbits/sec
[ 1] 120.0000-125.0000 sec 16.1 MBytes 27.1 Mbits/sec
Arista
- QoS not supported on virtual images, but syntax is almost identical to Cisco