Date: Fri, 30 Jul 2010 16:07:04 -0700 From: Patrick Mahan <mahan@mahan.org> To: freebsd-net@freebsd.org Subject: AltQ throughput issues (long message) Message-ID: <4C535B18.8020205@mahan.org>
next in thread | raw e-mail | index | archive | help
All,
I am looking for (again) some understanding of AltQ and how it works
w.r.t. packet through put. I posted earlier this month regarding how
to initially configure AltQ (thanks to everyone's help) and now have
it working over the em(4) drive on a FreeBSD 8.0 platform (HP DL350 G5).
I had to bring the em(4) driver from the 8-Stable branch, but it is
working just fine so far (needed to add the drbr_needs_enqueue() to
if_var.h).
I have now gone back to trying to setup up one queue with a bandwith
of 1900 Kbs (1.9 Mbs). I ran a test with 'iperf' using udp and setting
the bandwidth to 25 Mbs. I then ran a test setting the queue bandwith
to 20 Mbs and running 'iperf' again using udp and 25 Mbs bandwith.
In both cases, the throughput only seems to be 89% of the requested
throughput.
Test 1
AltQ queue bandwidth 1.9 Mbs, iperf -b 25M
pfctl -vv -s queue reported:
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 28298 bytes: 42771988 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.8 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 28298 bytes: 42771988 dropped pkts: 397077 bytes: 600380424 ]
[ qlength: 50/ 50 borrows: 0 suspends: 3278 ]
[ measured: 140.8 packets/s, 1.70Mb/s ]
iperf reported
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-200.4 sec 39.7 MBytes 1.66 Mbits/sec 6.998 ms 397190/425533 (93%)
Test 2
AltQ queue bandwidth 20 Mbs, iperf -b 25M
pfctl -vv -s queue reported:
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 356702 bytes: 539329126 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 1500.2 packets/s, 18.15Mb/s ]
queue test7788 on em0 bandwidth 20Mb cbq( default )
[ pkts: 356702 bytes: 539329126 dropped pkts: 149198 bytes: 225587376 ]
[ qlength: 46/ 50 borrows: 0 suspends: 39629 ]
[ measured: 1500.2 packets/s, 18.15Mb/s ]
iperf reported
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-240.0 sec 505 MBytes 17.6 Mbits/sec 0.918 ms 150584/510637 (29%)
Why can AltQ not drive it at full bandwidth? This is just some preliminary testing,
but I want to scale this up to use all available AltQ CBQ queues for various
operations.
As always, my knowledge is always increased with I ask questions on
this list.
Thanks,
Patrick
======================= Test Results ===========================
Network topology:
+--------------+ +--------------+
| | | |
| NPX8 | | NPX3 |
| (em1)+ ===================== +(em3) |
| | | |
| | | (em0) |
+--------------+ +------+-------+
I
I
I
I
I
I
I
I
+------+-------+
| (em0) |
| |
| NPX6 |
| |
| |
+--------------+
NPX8:
em1: 172.16.38.80/24
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:1c:c4:48:93:10
inet 172.16.38.80 netmask 0xffffff00 broadcast 172.16.38.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
NPX3:
em3: 172.16.38.30/24
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:1f:29:5f:c6:aa
inet 172.16.38.30 netmask 0xffffff00 broadcast 172.16.38.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em0: 172.16.13.30/24
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:1f:29:5f:c6:a9
inet 172.16.13.30 netmask 0xffffff00 broadcast 172.16.13.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
NPX6:
em0: 172.16.13.60/24
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:1c:c4:48:95:d1
inet 172.16.13.60 netmask 0xffffff00 broadcast 172.16.13.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
NPX8 IPv4 Routing table
npx8# netstat -nr
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 10.10.1.1 UGS 0 0 bce0
10.10.0.0/16 link#5 U 7 44 bce0
10.10.20.48 link#5 UHS 0 0 lo0
127.0.0.1 link#7 UH 0 0 lo0
172.16.13.0/24 172.16.38.30 UGS 0 0 em1
172.16.38.0/24 link#2 U 0 0 em1
172.16.38.80 link#2 UHS 0 0 lo0
NPX3 IPv4 Routing Table
npx3# netstat -nr
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 10.10.1.1 UGS 0 0 bce0
10.10.0.0/16 link#5 U 8 499 bce0
10.10.20.43 link#5 UHS 0 0 lo0
127.0.0.1 link#7 UH 0 0 lo0
172.16.13.0/24 link#1 U 0 0 em0
172.16.13.30 link#1 UHS 0 0 lo0
172.16.38.0/24 link#4 U 0 0 em3
172.16.38.30 link#4 UHS 0 0 lo0
NPX6 IPv4 Routing Table
npx6# netstat -nr
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 10.10.1.1 UGS 0 0 bce0
10.10.0.0/16 link#5 U 7 243 bce0
10.10.20.46 link#5 UHS 0 0 lo0
127.0.0.1 link#7 UH 0 0 lo0
172.16.13.0/24 link#1 U 0 0 em0
172.16.13.60 link#1 UHS 0 0 lo0
172.16.38.0/24 172.16.13.30 UGS 0 0 em0
NPX3 has forwarding enabled
npx3# sysctl net.inet.ip.forwarding
net.inet.ip.forwarding: 1
NPX8 has a route to NPX6 via em1
172.16.13.0/24 172.16.38.30 UGS 0 1754682 em1
NPX6 has a route to NPX8 via em0
172.16.38.0/24 172.16.13.30 UGS 0 2 em0
On NPX3 the following /etc/pf.conf is enabled
npx3# cat /etc/pf.conf
#######################################################################
# ALT-Q Configuration
#######################################################################
#======================================================================
# Cluster configuration
#======================================================================
altq on em0 cbq bandwidth 1000Mb queue { test7788 }
#======================================================================
# Queueu configuration
#======================================================================
queue test7788 bandwidth 1900Kb priority 1 cbq (default)
#======================================================================
# Filter rules
#======================================================================
pass out quick on em0 inet proto udp from any to any port 7788 no state queue
test7788
On NPX6 I start iperf in server mode
npx6# iperf -s -p 7788 -u -i 30
------------------------------------------------------------
Server listening on UDP port 7788
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.38.80 port 41064 connected with 172.16.13.60 port 7788
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-30.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 30.0-60.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 60.0-90.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 90.0-120.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 120.0-150.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 150.0-180.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-200.0 sec 597 MBytes 25.0 Mbits/sec
[ 3] Sent 425533 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-200.4 sec 39.7 MBytes 1.66 Mbits/sec 6.998 ms 397190/425533 (93%)
On NPX8 I start the iperf client
npx8# iperf -c 172.16.13.10 -p 7788 -u -b 25M -u -i 30 -t 200
------------------------------------------------------------
Client connecting to 172.16.13.60, UDP port 7788
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.13.60 port 7788 connected with 172.16.38.80 port 41064
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-30.0 sec 5.96 MBytes 1.67 Mbits/sec 0.710 ms 59453/63706 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 30.0-60.0 sec 5.95 MBytes 1.66 Mbits/sec 0.736 ms 59616/63859 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 60.0-90.0 sec 5.94 MBytes 1.66 Mbits/sec 0.721 ms 59529/63768 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 90.0-120.0 sec 5.95 MBytes 1.66 Mbits/sec 0.760 ms 59616/63859 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 120.0-150.0 sec 5.95 MBytes 1.66 Mbits/sec 0.732 ms 59618/63862 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 150.0-180.0 sec 5.95 MBytes 1.66 Mbits/sec 0.726 ms 59608/63852 (93%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-200.4 sec 39.7 MBytes 1.66 Mbits/sec 6.998 ms 397190/425533 (93%)
On NPX3 I start monitoring of the Altq
npx3: pfctl -vv -s queue
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 3 bytes: 238 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 3 bytes: 238 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 10 bytes: 532 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.2 packets/s, 58.80 b/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 10 bytes: 532 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.2 packets/s, 58.80 b/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 10 bytes: 532 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.2 packets/s, 51.45 b/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 10 bytes: 532 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 0.2 packets/s, 51.45 b/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 710 bytes: 1058932 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 17.6 packets/s, 211.73Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 710 bytes: 1058932 dropped pkts: 9600 bytes: 14515200 ]
[ qlength: 50/ 50 borrows: 0 suspends: 80 ]
[ measured: 17.6 packets/s, 211.73Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 1418 bytes: 2129428 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 33.1 packets/s, 399.36Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 1418 bytes: 2129428 dropped pkts: 19534 bytes: 29535408 ]
[ qlength: 50/ 50 borrows: 0 suspends: 162 ]
[ measured: 33.1 packets/s, 399.36Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 2125 bytes: 3198412 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 46.7 packets/s, 563.24Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 2125 bytes: 3198412 dropped pkts: 29470 bytes: 44558640 ]
[ qlength: 50/ 50 borrows: 0 suspends: 244 ]
[ measured: 46.7 packets/s, 563.24Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 2832 bytes: 4267396 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 58.5 packets/s, 706.63Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 2832 bytes: 4267396 dropped pkts: 39406 bytes: 59581872 ]
[ qlength: 50/ 50 borrows: 0 suspends: 326 ]
[ measured: 58.5 packets/s, 706.63Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 3539 bytes: 5336380 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 68.9 packets/s, 832.10Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 3539 bytes: 5336380 dropped pkts: 49341 bytes: 74603592 ]
[ qlength: 50/ 50 borrows: 0 suspends: 408 ]
[ measured: 68.9 packets/s, 832.10Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 4246 bytes: 6405364 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 77.9 packets/s, 941.88Kb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 4246 bytes: 6405364 dropped pkts: 59277 bytes: 89626824 ]
[ qlength: 50/ 50 borrows: 0 suspends: 490 ]
[ measured: 77.9 packets/s, 941.88Kb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 4954 bytes: 7475860 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 85.9 packets/s, 1.04Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 4954 bytes: 7475860 dropped pkts: 69211 bytes: 104647032 ]
[ qlength: 50/ 50 borrows: 0 suspends: 572 ]
[ measured: 85.9 packets/s, 1.04Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 5661 bytes: 8544844 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 92.8 packets/s, 1.12Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 5661 bytes: 8544844 dropped pkts: 79147 bytes: 119670264 ]
[ qlength: 50/ 50 borrows: 0 suspends: 654 ]
[ measured: 92.8 packets/s, 1.12Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 6368 bytes: 9613828 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 98.9 packets/s, 1.20Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 6368 bytes: 9613828 dropped pkts: 89082 bytes: 134691984 ]
[ qlength: 50/ 50 borrows: 0 suspends: 736 ]
[ measured: 98.9 packets/s, 1.20Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 7075 bytes: 10682812 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 104.2 packets/s, 1.26Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 7075 bytes: 10682812 dropped pkts: 99018 bytes: 149715216 ]
[ qlength: 50/ 50 borrows: 0 suspends: 818 ]
[ measured: 104.2 packets/s, 1.26Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 7783 bytes: 11753308 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 108.9 packets/s, 1.32Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 7783 bytes: 11753308 dropped pkts: 108953 bytes: 164736936 ]
[ qlength: 50/ 50 borrows: 0 suspends: 900 ]
[ measured: 108.9 packets/s, 1.32Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 8490 bytes: 12822292 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 113.0 packets/s, 1.37Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 8490 bytes: 12822292 dropped pkts: 118888 bytes: 179758656 ]
[ qlength: 50/ 50 borrows: 0 suspends: 982 ]
[ measured: 113.0 packets/s, 1.37Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 9199 bytes: 13894300 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 116.6 packets/s, 1.41Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 9199 bytes: 13894300 dropped pkts: 128823 bytes: 194780376 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1064 ]
[ measured: 116.6 packets/s, 1.41Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 9906 bytes: 14963284 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 119.7 packets/s, 1.45Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 9906 bytes: 14963284 dropped pkts: 138759 bytes: 209803608 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1146 ]
[ measured: 119.7 packets/s, 1.45Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 10613 bytes: 16032268 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 122.4 packets/s, 1.48Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 10613 bytes: 16032268 dropped pkts: 148694 bytes: 224825328 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1228 ]
[ measured: 122.4 packets/s, 1.48Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 11321 bytes: 17102764 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 124.8 packets/s, 1.51Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 11321 bytes: 17102764 dropped pkts: 158629 bytes: 239847048 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1310 ]
[ measured: 124.8 packets/s, 1.51Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 12028 bytes: 18171748 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 126.9 packets/s, 1.53Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 12028 bytes: 18171748 dropped pkts: 168565 bytes: 254870280 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1392 ]
[ measured: 126.9 packets/s, 1.53Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 12737 bytes: 19243756 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 128.7 packets/s, 1.56Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 12737 bytes: 19243756 dropped pkts: 178498 bytes: 269888976 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1474 ]
[ measured: 128.7 packets/s, 1.56Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 13445 bytes: 20314252 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 130.3 packets/s, 1.58Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 13445 bytes: 20314252 dropped pkts: 188433 bytes: 284910696 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1556 ]
[ measured: 130.3 packets/s, 1.58Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 14152 bytes: 21383236 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 131.7 packets/s, 1.59Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 14152 bytes: 21383236 dropped pkts: 198368 bytes: 299932416 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1638 ]
[ measured: 131.7 packets/s, 1.59Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 14859 bytes: 22452220 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 132.9 packets/s, 1.61Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 14859 bytes: 22452220 dropped pkts: 208304 bytes: 314955648 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1720 ]
[ measured: 132.9 packets/s, 1.61Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 15566 bytes: 23521204 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 134.0 packets/s, 1.62Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 15566 bytes: 23521204 dropped pkts: 218240 bytes: 329978880 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1802 ]
[ measured: 134.0 packets/s, 1.62Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 16274 bytes: 24591700 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 134.9 packets/s, 1.63Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 16274 bytes: 24591700 dropped pkts: 228174 bytes: 344999088 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1884 ]
[ measured: 134.9 packets/s, 1.63Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 16981 bytes: 25660684 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 135.7 packets/s, 1.64Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 16981 bytes: 25660684 dropped pkts: 238110 bytes: 360022320 ]
[ qlength: 50/ 50 borrows: 0 suspends: 1966 ]
[ measured: 135.7 packets/s, 1.64Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 17688 bytes: 26729668 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 136.5 packets/s, 1.65Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 17688 bytes: 26729668 dropped pkts: 248047 bytes: 375047064 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2048 ]
[ measured: 136.5 packets/s, 1.65Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 18396 bytes: 27800164 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 137.1 packets/s, 1.66Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 18396 bytes: 27800164 dropped pkts: 257981 bytes: 390067272 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2130 ]
[ measured: 137.1 packets/s, 1.66Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 19103 bytes: 28869148 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 137.6 packets/s, 1.66Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 19103 bytes: 28869148 dropped pkts: 267917 bytes: 405090504 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2212 ]
[ measured: 137.6 packets/s, 1.66Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 19810 bytes: 29938132 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 138.1 packets/s, 1.67Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 19810 bytes: 29938132 dropped pkts: 277853 bytes: 420113736 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2294 ]
[ measured: 138.1 packets/s, 1.67Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 20517 bytes: 31007116 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 138.5 packets/s, 1.68Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 20517 bytes: 31007116 dropped pkts: 287788 bytes: 435135456 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2376 ]
[ measured: 138.5 packets/s, 1.68Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 21225 bytes: 32077612 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 138.9 packets/s, 1.68Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 21225 bytes: 32077612 dropped pkts: 297723 bytes: 450157176 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2458 ]
[ measured: 138.9 packets/s, 1.68Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 21932 bytes: 33146596 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 139.2 packets/s, 1.68Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 21932 bytes: 33146596 dropped pkts: 307659 bytes: 465180408 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2540 ]
[ measured: 139.2 packets/s, 1.68Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 22639 bytes: 34215580 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 139.5 packets/s, 1.69Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 22639 bytes: 34215580 dropped pkts: 317594 bytes: 480202128 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2622 ]
[ measured: 139.5 packets/s, 1.69Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 23346 bytes: 35284564 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 139.7 packets/s, 1.69Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 23346 bytes: 35284564 dropped pkts: 327530 bytes: 495225360 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2704 ]
[ measured: 139.7 packets/s, 1.69Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 24054 bytes: 36355060 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.0 packets/s, 1.69Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 24054 bytes: 36355060 dropped pkts: 337465 bytes: 510247080 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2786 ]
[ measured: 140.0 packets/s, 1.69Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 24761 bytes: 37424044 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.1 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 24761 bytes: 37424044 dropped pkts: 347400 bytes: 525268800 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2868 ]
[ measured: 140.1 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 25468 bytes: 38493028 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.3 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 25468 bytes: 38493028 dropped pkts: 357336 bytes: 540292032 ]
[ qlength: 50/ 50 borrows: 0 suspends: 2950 ]
[ measured: 140.3 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 26176 bytes: 39563524 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.5 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 26176 bytes: 39563524 dropped pkts: 367272 bytes: 555315264 ]
[ qlength: 50/ 50 borrows: 0 suspends: 3032 ]
[ measured: 140.5 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 26883 bytes: 40632508 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.6 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 26883 bytes: 40632508 dropped pkts: 377207 bytes: 570336984 ]
[ qlength: 50/ 50 borrows: 0 suspends: 3114 ]
[ measured: 140.6 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 27590 bytes: 41701492 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.7 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 27590 bytes: 41701492 dropped pkts: 387143 bytes: 585360216 ]
[ qlength: 50/ 50 borrows: 0 suspends: 3196 ]
[ measured: 140.7 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 28298 bytes: 42771988 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 140.8 packets/s, 1.70Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 28298 bytes: 42771988 dropped pkts: 397077 bytes: 600380424 ]
[ qlength: 50/ 50 borrows: 0 suspends: 3278 ]
[ measured: 140.8 packets/s, 1.70Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 28355 bytes: 42858172 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 124.6 packets/s, 1.51Mb/s ]
queue test7788 on em0 bandwidth 1.90Mb cbq( default )
[ pkts: 28355 bytes: 42858172 dropped pkts: 397190 bytes: 600551280 ]
[ qlength: 0/ 50 borrows: 0 suspends: 3284 ]
[ measured: 124.6 packets/s, 1.51Mb/s ]
============================================================================
With the queue set to 20Mb throughput.
NPX3# pfctl -vv -s queue
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 334193 bytes: 505295518 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 1500.0 packets/s, 18.14Mb/s ]
queue test7788 on em0 bandwidth 20Mb cbq( default )
[ pkts: 334193 bytes: 505295518 dropped pkts: 139779 bytes: 211345848 ]
[ qlength: 46/ 50 borrows: 0 suspends: 37128 ]
[ measured: 1500.0 packets/s, 18.14Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 341694 bytes: 516637030 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 1500.1 packets/s, 18.14Mb/s ]
queue test7788 on em0 bandwidth 20Mb cbq( default )
[ pkts: 341697 bytes: 516641566 dropped pkts: 142920 bytes: 216095040 ]
[ qlength: 43/ 50 borrows: 0 suspends: 37961 ]
[ measured: 1500.1 packets/s, 18.15Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 349196 bytes: 527980054 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 1500.1 packets/s, 18.15Mb/s ]
queue test7788 on em0 bandwidth 20Mb cbq( default )
[ pkts: 349196 bytes: 527980054 dropped pkts: 146057 bytes: 220838184 ]
[ qlength: 50/ 50 borrows: 0 suspends: 38795 ]
[ measured: 1500.1 packets/s, 18.14Mb/s ]
queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
[ pkts: 356702 bytes: 539329126 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
[ measured: 1500.2 packets/s, 18.15Mb/s ]
queue test7788 on em0 bandwidth 20Mb cbq( default )
[ pkts: 356702 bytes: 539329126 dropped pkts: 149198 bytes: 225587376 ]
[ qlength: 46/ 50 borrows: 0 suspends: 39629 ]
[ measured: 1500.2 packets/s, 18.15Mb/s ]
NPX8: iperf -c 172.16.13.60 -p 7788 -u -b 25M -u -i 30 -t 240
------------------------------------------------------------
Client connecting to 172.16.13.60, UDP port 7788
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.38.80 port 17106 connected with 172.16.13.60 port 7788
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-30.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 30.0-60.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 60.0-90.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 90.0-120.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 120.0-150.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 150.0-180.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 180.0-210.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 210.0-240.0 sec 89.5 MBytes 25.0 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-240.0 sec 716 MBytes 25.0 Mbits/sec
[ 3] Sent 510637 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-240.0 sec 505 MBytes 17.6 Mbits/sec 0.918 ms 150584/510637 (29%)
NPX6: iperf -s -p 7788 -u -i 30
------------------------------------------------------------
Server listening on UDP port 7788
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 4] local 172.16.13.60 port 7788 connected with 172.16.38.80 port 17106
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.0-30.0 sec 63.1 MBytes 17.6 Mbits/sec 0.735 ms 18774/63782 (29%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 30.0-60.0 sec 63.1 MBytes 17.6 Mbits/sec 0.745 ms 18830/63830 (30%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 60.0-90.0 sec 63.1 MBytes 17.6 Mbits/sec 0.756 ms 18831/63831 (30%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 90.0-120.0 sec 63.1 MBytes 17.6 Mbits/sec 0.741 ms 18831/63831 (30%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 120.0-150.0 sec 63.1 MBytes 17.6 Mbits/sec 0.738 ms 18830/63831 (29%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 150.0-180.0 sec 63.1 MBytes 17.6 Mbits/sec 0.733 ms 18830/63830 (30%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 180.0-210.0 sec 63.1 MBytes 17.6 Mbits/sec 0.718 ms 18828/63829 (29%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 210.0-240.0 sec 63.1 MBytes 17.6 Mbits/sec 0.730 ms 18830/63830 (30%)
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.0-240.0 sec 505 MBytes 17.6 Mbits/sec 0.918 ms 150584/510637 (29%)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C535B18.8020205>