Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 31 Jul 2010 09:10:04 -0700
From:      Patrick Mahan <mahan@mahan.org>
To:        Luigi Rizzo <rizzo@iet.unipi.it>
Cc:        freebsd-net@freebsd.org
Subject:   Re: AltQ throughput issues (long message)
Message-ID:  <4C544ADC.2050109@mahan.org>
In-Reply-To: <20100730233053.GA12554@onelab2.iet.unipi.it>
References:  <4C535B18.8020205@mahan.org> <20100730233053.GA12554@onelab2.iet.unipi.it>

next in thread | previous in thread | raw e-mail | index | archive | help

See my responses inline - PLM

On 07/30/2010 04:30 PM, Luigi Rizzo wrote:
> On Fri, Jul 30, 2010 at 04:07:04PM -0700, Patrick Mahan wrote:
>> All,
>>
>> I am looking for (again) some understanding of AltQ and how it works
>> w.r.t. packet through put.  I posted earlier this month regarding how
>> to initially configure AltQ (thanks to everyone's help) and now have
>> it working over the em(4) drive on a FreeBSD 8.0 platform (HP DL350 G5).
>>
>> I had to bring the em(4) driver from the 8-Stable branch, but it is
>> working just fine so far (needed to add the drbr_needs_enqueue() to
>> if_var.h).
>>
>> I have now gone back to trying to setup up one queue with a bandwith
>> of 1900 Kbs (1.9 Mbs).  I ran a test with 'iperf' using udp and setting
>> the bandwidth to 25 Mbs.  I then ran a test setting the queue bandwith
>> to 20 Mbs and running 'iperf' again using udp and 25 Mbs bandwith.
>>
>> In both cases, the throughput only seems to be 89% of the requested
>> throughput.
>
> part of it can be explained because AltQ counts the whole packet
> (eg. 1514 bytes for a full frame) whereas iperf only considers the
> UDP payload (e.g. 1470 bytes in your case).
>

Okay, but that only accounts for 3% and I am seeing around 11%, any
idea what might be accounting for the remaining 8%?

> The other thing you should check is whether there is any extra
> traffic going through the interface that competes for the bottleneck
> bandwidth. You have such huge drop rates in your tests that i
> would not be surprised if you had ICMP packets going around
> trying to slow down the sender.

No extra traffic.  All machines have a bce0 (not shown in the diagram)
that acts as the management port on a 10.10.0.0 network.  Neither NPX8
nor NPX6 has forwarding enabled.  But that said NPX3 where the AltQ is
enabled does have ip forwarding enabled.  But should not be trying to
route any packets incoming on the bce0, but I will need to confirm.  Will
running pfctl before starting iperf for a few seconds be enough?  Should
I create a second queue to capture these possible 'extra' packets?

As for the em(4) interfaces, they are all connected directly to each other
using cat 6 cross-over cables, no hub/switch involved.

Where do you see the drop?  If you are looking at the end of the pfctl output
that is probably occurring after iperf has finished its run.  I have noticed
that the queue bandwidth numbers sharply decline after iperf has finished?


>
> BTW have you tried dummynet in your config?
>

How would you suggest using dummynet?  Is it workable for a QoS solution?

Thanks as always,

Patrick

> 	cheers
> 	luigi
>
>
>> Test 1
>>    AltQ queue bandwidth 1.9 Mbs, iperf -b 25M
>>    pfctl -vv -s queue reported:
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      28298  bytes:   42771988  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.8 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      28298  bytes:   42771988  dropped pkts: 397077 bytes:
>>    600380424 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   3278 ]
>>    [ measured:   140.8 packets/s, 1.70Mb/s ]
>>
>>    iperf reported
>>
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-200.4 sec  39.7 MBytes  1.66 Mbits/sec  6.998 ms 397190/425533
>> (93%)
>>
>> Test 2
>>    AltQ queue bandwidth 20 Mbs, iperf -b 25M
>>    pfctl -vv -s queue reported:
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:     356702  bytes:  539329126  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:  1500.2 packets/s, 18.15Mb/s ]
>> queue  test7788 on em0 bandwidth 20Mb cbq( default )
>>    [ pkts:     356702  bytes:  539329126  dropped pkts: 149198 bytes:
>>    225587376 ]
>>    [ qlength:  46/ 50  borrows:      0  suspends:  39629 ]
>>    [ measured:  1500.2 packets/s, 18.15Mb/s ]
>>
>>    iperf reported
>>
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-240.0 sec    505 MBytes  17.6 Mbits/sec  0.918 ms 150584/510637
>> (29%)
>>
>> Why can AltQ not drive it at full bandwidth?  This is just some preliminary
>> testing,
>> but I want to scale this up to use all available AltQ CBQ queues for
>> various operations.
>>
>> As always, my knowledge is always increased with I ask questions on
>> this list.
>>
>> Thanks,
>>
>> Patrick
>>
>> ======================= Test Results ===========================
>> Network topology:
>>
>>
>> +--------------+                       +--------------+
>> |              |                       |              |
>> |   NPX8       |                       |    NPX3      |
>> |         (em1)+ ===================== +(em3)         |
>> |              |                       |              |
>> |              |                       |    (em0)     |
>> +--------------+                       +------+-------+
>>                                                I
>>                                                I
>>                                                I
>>                                                I
>>                                                I
>>                                                I
>>                                                I
>>                                                I
>>                                         +------+-------+
>>                                         |    (em0)     |
>>                                         |              |
>>                                         |    NPX6      |
>>                                         |              |
>>                                         |              |
>>                                         +--------------+
>>
>> NPX8:
>>
>>    em1: 172.16.38.80/24
>>
>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>  metric 0 mtu 1500
>>          options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>>          ether 00:1c:c4:48:93:10
>>          inet 172.16.38.80 netmask 0xffffff00 broadcast 172.16.38.255
>>          media: Ethernet autoselect (1000baseT<full-duplex>)
>>          status: active
>>
>>
>> NPX3:
>>
>>    em3: 172.16.38.30/24
>>
>> em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>  metric 0 mtu 1500
>>          options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>>          ether 00:1f:29:5f:c6:aa
>>          inet 172.16.38.30 netmask 0xffffff00 broadcast 172.16.38.255
>>          media: Ethernet autoselect (1000baseT<full-duplex>)
>>          status: active
>>
>>    em0: 172.16.13.30/24
>>
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>  metric 0 mtu 1500
>>          options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>>          ether 00:1f:29:5f:c6:a9
>>          inet 172.16.13.30 netmask 0xffffff00 broadcast 172.16.13.255
>>          media: Ethernet autoselect (1000baseT<full-duplex>)
>>          status: active
>>
>> NPX6:
>>
>>    em0: 172.16.13.60/24
>>
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>  metric 0 mtu 1500
>>          options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>>          ether 00:1c:c4:48:95:d1
>>          inet 172.16.13.60 netmask 0xffffff00 broadcast 172.16.13.255
>>          media: Ethernet autoselect (1000baseT<full-duplex>)
>>          status: active
>>
>> NPX8 IPv4 Routing table
>>
>> npx8# netstat -nr
>> Routing tables
>>
>> Internet:
>> Destination        Gateway            Flags    Refs      Use  Netif Expire
>> default            10.10.1.1          UGS         0        0   bce0
>> 10.10.0.0/16       link#5             U           7       44   bce0
>> 10.10.20.48        link#5             UHS         0        0    lo0
>> 127.0.0.1          link#7             UH          0        0    lo0
>> 172.16.13.0/24     172.16.38.30       UGS         0        0    em1
>> 172.16.38.0/24     link#2             U           0        0    em1
>> 172.16.38.80       link#2             UHS         0        0    lo0
>>
>> NPX3 IPv4 Routing Table
>>
>> npx3# netstat -nr
>> Routing tables
>>
>> Internet:
>> Destination        Gateway            Flags    Refs      Use  Netif Expire
>> default            10.10.1.1          UGS         0        0   bce0
>> 10.10.0.0/16       link#5             U           8      499   bce0
>> 10.10.20.43        link#5             UHS         0        0    lo0
>> 127.0.0.1          link#7             UH          0        0    lo0
>> 172.16.13.0/24     link#1             U           0        0    em0
>> 172.16.13.30       link#1             UHS         0        0    lo0
>> 172.16.38.0/24     link#4             U           0        0    em3
>> 172.16.38.30       link#4             UHS         0        0    lo0
>>
>>
>> NPX6 IPv4 Routing Table
>>
>> npx6# netstat -nr
>> Routing tables
>>
>> Internet:
>> Destination        Gateway            Flags    Refs      Use  Netif Expire
>> default            10.10.1.1          UGS         0        0   bce0
>> 10.10.0.0/16       link#5             U           7      243   bce0
>> 10.10.20.46        link#5             UHS         0        0    lo0
>> 127.0.0.1          link#7             UH          0        0    lo0
>> 172.16.13.0/24     link#1             U           0        0    em0
>> 172.16.13.60       link#1             UHS         0        0    lo0
>> 172.16.38.0/24     172.16.13.30       UGS         0        0    em0
>>
>>
>> NPX3 has forwarding enabled
>>
>> npx3# sysctl net.inet.ip.forwarding
>> net.inet.ip.forwarding: 1
>>
>> NPX8 has a route to NPX6 via em1
>>
>> 172.16.13.0/24     172.16.38.30       UGS         0  1754682    em1
>>
>> NPX6 has a route to NPX8 via em0
>> 172.16.38.0/24     172.16.13.30       UGS         0        2    em0
>>
>>
>> On NPX3 the following /etc/pf.conf is enabled
>>
>> npx3# cat /etc/pf.conf
>> #######################################################################
>> # ALT-Q Configuration
>> #######################################################################
>>
>> #======================================================================
>> # Cluster configuration
>> #======================================================================
>> altq on em0 cbq bandwidth 1000Mb queue { test7788 }
>>
>> #======================================================================
>> # Queueu configuration
>> #======================================================================
>> queue test7788 bandwidth 1900Kb priority 1 cbq (default)
>>
>> #======================================================================
>> # Filter rules
>> #======================================================================
>> pass out quick on em0 inet proto udp from any to any port 7788 no state
>> queue test7788
>>
>>
>> On NPX6 I start iperf in server mode
>>
>> npx6# iperf -s -p 7788 -u -i 30
>> ------------------------------------------------------------
>> Server listening on UDP port 7788
>> Receiving 1470 byte datagrams
>> UDP buffer size: 41.1 KByte (default)
>> ------------------------------------------------------------
>> [  3] local 172.16.38.80 port 41064 connected with 172.16.13.60 port 7788
>> [ ID] Interval       Transfer     Bandwidth
>> [  3]  0.0-30.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 30.0-60.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 60.0-90.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 90.0-120.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 120.0-150.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 150.0-180.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3]  0.0-200.0 sec    597 MBytes  25.0 Mbits/sec
>> [  3] Sent 425533 datagrams
>> [  3] Server Report:
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-200.4 sec  39.7 MBytes  1.66 Mbits/sec  6.998 ms 397190/425533
>> (93%)
>>
>> On NPX8 I start the iperf client
>>
>> npx8# iperf -c 172.16.13.10 -p 7788 -u -b 25M -u -i 30 -t 200
>> ------------------------------------------------------------
>> Client connecting to 172.16.13.60, UDP port 7788
>> Sending 1470 byte datagrams
>> UDP buffer size: 9.00 KByte (default)
>> ------------------------------------------------------------
>> [  3] local 172.16.13.60 port 7788 connected with 172.16.38.80 port 41064
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-30.0 sec  5.96 MBytes  1.67 Mbits/sec  0.710 ms 59453/63706 (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3] 30.0-60.0 sec  5.95 MBytes  1.66 Mbits/sec  0.736 ms 59616/63859 (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3] 60.0-90.0 sec  5.94 MBytes  1.66 Mbits/sec  0.721 ms 59529/63768 (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3] 90.0-120.0 sec  5.95 MBytes  1.66 Mbits/sec  0.760 ms 59616/63859
>> (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3] 120.0-150.0 sec  5.95 MBytes  1.66 Mbits/sec  0.732 ms 59618/63862
>> (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3] 150.0-180.0 sec  5.95 MBytes  1.66 Mbits/sec  0.726 ms 59608/63852
>> (93%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-200.4 sec  39.7 MBytes  1.66 Mbits/sec  6.998 ms 397190/425533
>> (93%)
>>
>>
>> On NPX3 I start monitoring of the Altq
>>
>> npx3: pfctl -vv -s queue
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:          3  bytes:        238  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.0 packets/s, 0 b/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:          3  bytes:        238  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.0 packets/s, 0 b/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:         10  bytes:        532  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.2 packets/s, 58.80 b/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:         10  bytes:        532  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.2 packets/s, 58.80 b/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:         10  bytes:        532  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.2 packets/s, 51.45 b/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:         10  bytes:        532  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:     0.2 packets/s, 51.45 b/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:        710  bytes:    1058932  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    17.6 packets/s, 211.73Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:        710  bytes:    1058932  dropped pkts:   9600 bytes:
>>    14515200 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:     80 ]
>>    [ measured:    17.6 packets/s, 211.73Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       1418  bytes:    2129428  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    33.1 packets/s, 399.36Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       1418  bytes:    2129428  dropped pkts:  19534 bytes:
>>    29535408 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    162 ]
>>    [ measured:    33.1 packets/s, 399.36Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       2125  bytes:    3198412  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    46.7 packets/s, 563.24Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       2125  bytes:    3198412  dropped pkts:  29470 bytes:
>>    44558640 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    244 ]
>>    [ measured:    46.7 packets/s, 563.24Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       2832  bytes:    4267396  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    58.5 packets/s, 706.63Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       2832  bytes:    4267396  dropped pkts:  39406 bytes:
>>    59581872 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    326 ]
>>    [ measured:    58.5 packets/s, 706.63Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       3539  bytes:    5336380  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    68.9 packets/s, 832.10Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       3539  bytes:    5336380  dropped pkts:  49341 bytes:
>>    74603592 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    408 ]
>>    [ measured:    68.9 packets/s, 832.10Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       4246  bytes:    6405364  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    77.9 packets/s, 941.88Kb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       4246  bytes:    6405364  dropped pkts:  59277 bytes:
>>    89626824 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    490 ]
>>    [ measured:    77.9 packets/s, 941.88Kb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       4954  bytes:    7475860  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    85.9 packets/s, 1.04Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       4954  bytes:    7475860  dropped pkts:  69211 bytes:
>>    104647032 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    572 ]
>>    [ measured:    85.9 packets/s, 1.04Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       5661  bytes:    8544844  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    92.8 packets/s, 1.12Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       5661  bytes:    8544844  dropped pkts:  79147 bytes:
>>    119670264 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    654 ]
>>    [ measured:    92.8 packets/s, 1.12Mb/s ]
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       6368  bytes:    9613828  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:    98.9 packets/s, 1.20Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       6368  bytes:    9613828  dropped pkts:  89082 bytes:
>>    134691984 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    736 ]
>>    [ measured:    98.9 packets/s, 1.20Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       7075  bytes:   10682812  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   104.2 packets/s, 1.26Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       7075  bytes:   10682812  dropped pkts:  99018 bytes:
>>    149715216 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    818 ]
>>    [ measured:   104.2 packets/s, 1.26Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       7783  bytes:   11753308  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   108.9 packets/s, 1.32Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       7783  bytes:   11753308  dropped pkts: 108953 bytes:
>>    164736936 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    900 ]
>>    [ measured:   108.9 packets/s, 1.32Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       8490  bytes:   12822292  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   113.0 packets/s, 1.37Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       8490  bytes:   12822292  dropped pkts: 118888 bytes:
>>    179758656 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:    982 ]
>>    [ measured:   113.0 packets/s, 1.37Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       9199  bytes:   13894300  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   116.6 packets/s, 1.41Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       9199  bytes:   13894300  dropped pkts: 128823 bytes:
>>    194780376 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1064 ]
>>    [ measured:   116.6 packets/s, 1.41Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:       9906  bytes:   14963284  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   119.7 packets/s, 1.45Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:       9906  bytes:   14963284  dropped pkts: 138759 bytes:
>>    209803608 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1146 ]
>>    [ measured:   119.7 packets/s, 1.45Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      10613  bytes:   16032268  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   122.4 packets/s, 1.48Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      10613  bytes:   16032268  dropped pkts: 148694 bytes:
>>    224825328 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1228 ]
>>    [ measured:   122.4 packets/s, 1.48Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      11321  bytes:   17102764  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   124.8 packets/s, 1.51Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      11321  bytes:   17102764  dropped pkts: 158629 bytes:
>>    239847048 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1310 ]
>>    [ measured:   124.8 packets/s, 1.51Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      12028  bytes:   18171748  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   126.9 packets/s, 1.53Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      12028  bytes:   18171748  dropped pkts: 168565 bytes:
>>    254870280 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1392 ]
>>    [ measured:   126.9 packets/s, 1.53Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      12737  bytes:   19243756  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   128.7 packets/s, 1.56Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      12737  bytes:   19243756  dropped pkts: 178498 bytes:
>>    269888976 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1474 ]
>>    [ measured:   128.7 packets/s, 1.56Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      13445  bytes:   20314252  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   130.3 packets/s, 1.58Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      13445  bytes:   20314252  dropped pkts: 188433 bytes:
>>    284910696 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1556 ]
>>    [ measured:   130.3 packets/s, 1.58Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      14152  bytes:   21383236  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   131.7 packets/s, 1.59Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      14152  bytes:   21383236  dropped pkts: 198368 bytes:
>>    299932416 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1638 ]
>>    [ measured:   131.7 packets/s, 1.59Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      14859  bytes:   22452220  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   132.9 packets/s, 1.61Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      14859  bytes:   22452220  dropped pkts: 208304 bytes:
>>    314955648 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1720 ]
>>    [ measured:   132.9 packets/s, 1.61Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      15566  bytes:   23521204  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   134.0 packets/s, 1.62Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      15566  bytes:   23521204  dropped pkts: 218240 bytes:
>>    329978880 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1802 ]
>>    [ measured:   134.0 packets/s, 1.62Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      16274  bytes:   24591700  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   134.9 packets/s, 1.63Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      16274  bytes:   24591700  dropped pkts: 228174 bytes:
>>    344999088 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1884 ]
>>    [ measured:   134.9 packets/s, 1.63Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      16981  bytes:   25660684  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   135.7 packets/s, 1.64Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      16981  bytes:   25660684  dropped pkts: 238110 bytes:
>>    360022320 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   1966 ]
>>    [ measured:   135.7 packets/s, 1.64Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      17688  bytes:   26729668  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   136.5 packets/s, 1.65Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      17688  bytes:   26729668  dropped pkts: 248047 bytes:
>>    375047064 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2048 ]
>>    [ measured:   136.5 packets/s, 1.65Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      18396  bytes:   27800164  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   137.1 packets/s, 1.66Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      18396  bytes:   27800164  dropped pkts: 257981 bytes:
>>    390067272 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2130 ]
>>    [ measured:   137.1 packets/s, 1.66Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      19103  bytes:   28869148  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   137.6 packets/s, 1.66Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      19103  bytes:   28869148  dropped pkts: 267917 bytes:
>>    405090504 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2212 ]
>>    [ measured:   137.6 packets/s, 1.66Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      19810  bytes:   29938132  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   138.1 packets/s, 1.67Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      19810  bytes:   29938132  dropped pkts: 277853 bytes:
>>    420113736 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2294 ]
>>    [ measured:   138.1 packets/s, 1.67Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      20517  bytes:   31007116  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   138.5 packets/s, 1.68Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      20517  bytes:   31007116  dropped pkts: 287788 bytes:
>>    435135456 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2376 ]
>>    [ measured:   138.5 packets/s, 1.68Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      21225  bytes:   32077612  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   138.9 packets/s, 1.68Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      21225  bytes:   32077612  dropped pkts: 297723 bytes:
>>    450157176 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2458 ]
>>    [ measured:   138.9 packets/s, 1.68Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      21932  bytes:   33146596  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   139.2 packets/s, 1.68Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      21932  bytes:   33146596  dropped pkts: 307659 bytes:
>>    465180408 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2540 ]
>>    [ measured:   139.2 packets/s, 1.68Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      22639  bytes:   34215580  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   139.5 packets/s, 1.69Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      22639  bytes:   34215580  dropped pkts: 317594 bytes:
>>    480202128 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2622 ]
>>    [ measured:   139.5 packets/s, 1.69Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      23346  bytes:   35284564  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   139.7 packets/s, 1.69Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      23346  bytes:   35284564  dropped pkts: 327530 bytes:
>>    495225360 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2704 ]
>>    [ measured:   139.7 packets/s, 1.69Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      24054  bytes:   36355060  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.0 packets/s, 1.69Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      24054  bytes:   36355060  dropped pkts: 337465 bytes:
>>    510247080 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2786 ]
>>    [ measured:   140.0 packets/s, 1.69Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      24761  bytes:   37424044  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.1 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      24761  bytes:   37424044  dropped pkts: 347400 bytes:
>>    525268800 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2868 ]
>>    [ measured:   140.1 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      25468  bytes:   38493028  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.3 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      25468  bytes:   38493028  dropped pkts: 357336 bytes:
>>    540292032 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   2950 ]
>>    [ measured:   140.3 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      26176  bytes:   39563524  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.5 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      26176  bytes:   39563524  dropped pkts: 367272 bytes:
>>    555315264 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   3032 ]
>>    [ measured:   140.5 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      26883  bytes:   40632508  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.6 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      26883  bytes:   40632508  dropped pkts: 377207 bytes:
>>    570336984 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   3114 ]
>>    [ measured:   140.6 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      27590  bytes:   41701492  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.7 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      27590  bytes:   41701492  dropped pkts: 387143 bytes:
>>    585360216 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   3196 ]
>>    [ measured:   140.7 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      28298  bytes:   42771988  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   140.8 packets/s, 1.70Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      28298  bytes:   42771988  dropped pkts: 397077 bytes:
>>    600380424 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:   3278 ]
>>    [ measured:   140.8 packets/s, 1.70Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:      28355  bytes:   42858172  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:   124.6 packets/s, 1.51Mb/s ]
>> queue  test7788 on em0 bandwidth 1.90Mb cbq( default )
>>    [ pkts:      28355  bytes:   42858172  dropped pkts: 397190 bytes:
>>    600551280 ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:   3284 ]
>>    [ measured:   124.6 packets/s, 1.51Mb/s ]
>>
>> ============================================================================
>>
>> With the queue set to 20Mb throughput.
>>
>> NPX3# pfctl -vv -s queue
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:     334193  bytes:  505295518  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:  1500.0 packets/s, 18.14Mb/s ]
>> queue  test7788 on em0 bandwidth 20Mb cbq( default )
>>    [ pkts:     334193  bytes:  505295518  dropped pkts: 139779 bytes:
>>    211345848 ]
>>    [ qlength:  46/ 50  borrows:      0  suspends:  37128 ]
>>    [ measured:  1500.0 packets/s, 18.14Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:     341694  bytes:  516637030  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:  1500.1 packets/s, 18.14Mb/s ]
>> queue  test7788 on em0 bandwidth 20Mb cbq( default )
>>    [ pkts:     341697  bytes:  516641566  dropped pkts: 142920 bytes:
>>    216095040 ]
>>    [ qlength:  43/ 50  borrows:      0  suspends:  37961 ]
>>    [ measured:  1500.1 packets/s, 18.15Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:     349196  bytes:  527980054  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:  1500.1 packets/s, 18.15Mb/s ]
>> queue  test7788 on em0 bandwidth 20Mb cbq( default )
>>    [ pkts:     349196  bytes:  527980054  dropped pkts: 146057 bytes:
>>    220838184 ]
>>    [ qlength:  50/ 50  borrows:      0  suspends:  38795 ]
>>    [ measured:  1500.1 packets/s, 18.14Mb/s ]
>>
>> queue root_em0 on em0 bandwidth 1Gb priority 0 cbq( wrr root ) {test7788}
>>    [ pkts:     356702  bytes:  539329126  dropped pkts:      0 bytes:      0
>>    ]
>>    [ qlength:   0/ 50  borrows:      0  suspends:      0 ]
>>    [ measured:  1500.2 packets/s, 18.15Mb/s ]
>> queue  test7788 on em0 bandwidth 20Mb cbq( default )
>>    [ pkts:     356702  bytes:  539329126  dropped pkts: 149198 bytes:
>>    225587376 ]
>>    [ qlength:  46/ 50  borrows:      0  suspends:  39629 ]
>>    [ measured:  1500.2 packets/s, 18.15Mb/s ]
>>
>> NPX8: iperf -c 172.16.13.60 -p 7788 -u -b 25M -u -i 30 -t 240
>> ------------------------------------------------------------
>> Client connecting to 172.16.13.60, UDP port 7788
>> Sending 1470 byte datagrams
>> UDP buffer size: 9.00 KByte (default)
>> ------------------------------------------------------------
>> [  3] local 172.16.38.80 port 17106 connected with 172.16.13.60 port 7788
>> [ ID] Interval       Transfer     Bandwidth
>> [  3]  0.0-30.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 30.0-60.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 60.0-90.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 90.0-120.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 120.0-150.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 150.0-180.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 180.0-210.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3] 210.0-240.0 sec  89.5 MBytes  25.0 Mbits/sec
>> [ ID] Interval       Transfer     Bandwidth
>> [  3]  0.0-240.0 sec    716 MBytes  25.0 Mbits/sec
>> [  3] Sent 510637 datagrams
>> [  3] Server Report:
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  3]  0.0-240.0 sec    505 MBytes  17.6 Mbits/sec  0.918 ms 150584/510637
>> (29%)
>>
>> NPX6: iperf -s -p 7788 -u -i 30
>> ------------------------------------------------------------
>> Server listening on UDP port 7788
>> Receiving 1470 byte datagrams
>> UDP buffer size: 41.1 KByte (default)
>> ------------------------------------------------------------
>> [  4] local 172.16.13.60 port 7788 connected with 172.16.38.80 port 17106
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4]  0.0-30.0 sec  63.1 MBytes  17.6 Mbits/sec  0.735 ms 18774/63782 (29%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 30.0-60.0 sec  63.1 MBytes  17.6 Mbits/sec  0.745 ms 18830/63830 (30%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 60.0-90.0 sec  63.1 MBytes  17.6 Mbits/sec  0.756 ms 18831/63831 (30%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 90.0-120.0 sec  63.1 MBytes  17.6 Mbits/sec  0.741 ms 18831/63831
>> (30%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 120.0-150.0 sec  63.1 MBytes  17.6 Mbits/sec  0.738 ms 18830/63831
>> (29%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 150.0-180.0 sec  63.1 MBytes  17.6 Mbits/sec  0.733 ms 18830/63830
>> (30%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 180.0-210.0 sec  63.1 MBytes  17.6 Mbits/sec  0.718 ms 18828/63829
>> (29%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4] 210.0-240.0 sec  63.1 MBytes  17.6 Mbits/sec  0.730 ms 18830/63830
>> (30%)
>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>> Datagrams
>> [  4]  0.0-240.0 sec    505 MBytes  17.6 Mbits/sec  0.918 ms 150584/510637
>> (29%)
>> _______________________________________________
>> freebsd-net@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C544ADC.2050109>