Date: Wed, 25 Feb 2009 15:14:11 +0800 From: Siquijor Philips <siquijorphilips@gmail.com> To: freebsd-net@freebsd.org Subject: Questions on processing smaller frame size Message-ID: <a27b90e40902242314w12c15fddma43e1cd5afec8938@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Can someone explain why FreeBSD network throughput (both incoming and
outgoing traffic) decreases when smaller frame size being processed?
With smaller frame size, corresponding packet rate (packet/sec or pps)
increases and experiencing dropped packets. What causes dropping of
packets with small frame size? My system is running on a 2x Quad-Core
Intel Xeon machine with FreeBSD-7.1 RELEASE using 1-Gigabit 2-port
Chelsio NIC connected on a Spirent traffic generator. Smaller frame
sizes are tested on 64-byte, 128-byte, 256-byte and 512-byte. Default
frame size of 1500-byte which is the default of an Ethernet is working
fine.
Below are my data;
# ifconfig -m
cxgb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=5bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO>
capabilities=7bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO>
ether 00:07:43:05:7b:ac
inet 192.168.0.1 netmask 0xffffe000 broadcast 192.168.255.255
inet6 fe80::207:43ff:fe05:7bac%cxgb0 prefixlen 64 scopeid 0x1
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
supported media:
media autoselect
media 1000baseTX mediaopt full-duplex
media 100baseTX mediaopt full-duplex
media 100baseTX
media 10baseT/UTP mediaopt full-duplex
media 10baseT/UTP
cxgb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=5bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO>
capabilities=7bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO>
ether 00:07:43:05:7b:ad
inet 10.0.0.1 netmask 0xfff00000 broadcast 10.0.255.255
inet6 fe80::207:43ff:fe05:7bad%cxgb1 prefixlen 64 scopeid 0x2
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
supported media:
media autoselect
media 1000baseTX mediaopt full-duplex
media 100baseTX mediaopt full-duplex
media 100baseTX
media 10baseT/UTP mediaopt full-duplex
media 10baseT/UTP
1500-byte frame size -> OK
---------------------
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
\ iface Rx Tx
Total
cxgb0: 938.58 Mb/s 938.58 Mb/s 1.83 Gb/s
cxgb1: 938.28 Mb/s 938.58 Mb/s 1.83 Gb/s
lo0: 0.00 b/s 0.00 b/s
0.00 b/s
total: 1.83 Gb/s 1.83 Gb/s 3.67 Gb/s
# netstat -I cxgb0 -w 1 -d
input (cxgb0) output
packets errs bytes packets errs bytes colls drops
82234 0 123059464 82229 0 123059464 0 0
82219 0 123031040 82240 0 123031040 0 0
82201 0 123017576 82187 0 123019072 0 0
82218 0 123025056 82242 0 123025056 0 0
82214 0 123028048 82247 0 123023560 0 0
82210 0 123023560 82209 0 123022064 0 0
82210 0 123020568 82244 0 123022064 0 0
82210 0 123023560 82226 0 123023560 0 0
82240 0 123057968 82208 0 123051984 0 0
82111 0 122870968 82161 0 122876952 0 0
82299 0 123158200 82312 0 123156704 0 0
82208 0 123013088 82226 0 123013088 0 0
82205 0 123022064 82229 0 123019072 0 0
82224 0 123028048 82213 0 123026552 0 0
82219 0 123025056 82252 0 123026552 0 0
82212 0 123023560 82218 0 123025056 0 0
82223 0 123026552 82231 0 123023560 0 0
82218 0 123023560 82212 0 123022064 0 0
82214 0 123028048 82252 0 123028048 0 0
82214 0 123023560 82226 0 123023560 0 0
# netstat -I cxgb1 -w 1 -d
input (cxgb1) output
packets errs bytes packets errs bytes colls drops
82220 0 122996632 82261 0 123029544 0 0
82211 0 122981672 82204 0 123022064 0 0
82207 0 122971200 82243 0 123022064 0 0
82215 0 122990648 82235 0 123022064 0 0
82220 0 122987656 82222 0 123029544 0 0
82209 0 122974192 82247 0 123025056 0 0
82215 0 122978680 82190 0 123025056 0 0
82226 0 122981672 82250 0 123023560 0 0
82214 0 122984664 82229 0 123022064 0 0
82217 0 122987656 82211 0 123025056 0 0
82214 0 122984664 82226 0 123023560 0 0
82224 0 122992144 82218 0 123023560 0 0
82208 0 122963720 82236 0 123025056 0 0
82218 0 122980176 82233 0 123022064 0 0
82217 0 122986160 82219 0 123028048 0 0
82215 0 122983168 82249 0 123023560 0 0
82208 0 122977184 82192 0 123025056 0 0
82214 0 122969704 82231 0 123019072 0 0
82214 0 122990648 82249 0 123031040 0 0
82210 0 122965216 82207 0 123019072 0 0
512-byte frame size:
---------------------
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
| iface Rx Tx
Total
cxgb0: 911.85 Mb/s 407.88 Mb/s 1.29 Gb/s
cxgb1: 898.39 Mb/s 318.05 Mb/s 1.19 Gb/s
lo0: 0.00 b/s 0.00 b/s
0.00 b/s
total: 1.77 Gb/s 725.93 Mb/s 2.48 Gb/s
# netstat -I cxgb0 -w 1 -d
input (cxgb0) output
packets errs bytes packets errs bytes colls drops
234410 0 120264428 115185 0 58832496 0 120569
230622 0 118386352 106160 0 53905404 0 126996
232698 0 119392700 105872 0 53480716 0 129205
233396 0 119755412 111680 0 57011824 0 123326
232678 0 119324120 110720 0 56231028 0 123968
231795 0 118987824 108736 0 54822344 0 125823
232432 0 119239284 106464 0 54465220 0 127468
233480 0 119887492 112160 0 56965088 0 123223
231789 0 118927372 106816 0 53860700 0 128085
232554 0 119338852 106486 0 54217316 0 127951
232572 0 119318024 103850 0 52624736 0 131009
235524 0 120803924 117382 0 59691524 0 119868
229808 0 117925596 108954 0 55370476 0 123077
234391 0 120326912 105216 0 53702712 0 130974
230581 0 118323360 101312 0 51453796 0 131192
233676 0 119798592 105152 0 53400960 0 130652
232868 0 119469408 100992 0 51286664 0 133883
231428 0 118835932 110791 0 56069484 0 123184
234807 0 120577864 111481 0 56814212 0 125395
231241 0 118634256 107904 0 54803548 0 125396
# netstat -I cxgb1 -w 1 -d
input (cxgb1) output
packets errs bytes packets errs bytes colls drops
231385 0 117446044 79040 0 40151304 0 155399
231482 0 117471952 84192 0 42314876 0 150946
234274 0 118907560 82720 0 42464736 0 153623
229963 0 116715540 78656 0 39487856 0 154837
232107 0 117802152 79146 0 40233092 0 155670
231872 0 117672104 87101 0 44432220 0 147407
232063 0 117799612 80281 0 41005760 0 154019
232221 0 117875812 79574 0 40224456 0 155825
232181 0 117840252 86474 0 44109640 0 148175
233005 0 118255288 79648 0 40453056 0 155805
231327 0 117362224 84791 0 42597324 0 150323
233615 0 118578376 83305 0 42775632 0 152044
232620 0 118073932 88704 0 45054520 0 146961
231816 0 117679724 81312 0 41300908 0 153107
230951 0 117200680 85984 0 43672252 0 147643
232828 0 118150132 81984 0 41639236 0 153747
230936 0 117206776 81896 0 41540176 0 151850
232234 0 117866160 80696 0 40661844 0 154869
234465 0 119013224 90782 0 46159928 0 146240
230022 0 116739924 78224 0 39735760 0 154385
256-byte frame size:
--------------------
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
/ iface Rx Tx
Total
cxgb0: 516.81 Mb/s 89.50 Mb/s 606.31 Mb/s
cxgb1: 510.47 Mb/s 0.00 b/s 510.47 Mb/s
lo0: 0.00 b/s 0.00 b/s 0.00 b/s
total: 1.00 Gb/s 89.50 Mb/s 1.09 Gb/s
# netstat -I cxgb0 -w 1 -d
input (cxgb0) output
packets errs bytes packets errs bytes colls drops
262028 0 66449124 46674 0 11973780 0 215945
265254 0 67428648 45774 0 11543364 0 220970
263969 0 67080384 41728 0 10509156 0 224438
260856 0 66195864 48960 0 12336156 0 213067
280701 0 71610840 73376 0 18483696 0 207469
286981 0 73295964 101984 0 25687368 0 188828
291674 0 74534292 98048 0 24522624 0 197495
275561 0 70195608 64128 0 16142616 0 214413
299603 0 76662180 123520 0 31266900 0 178712
300192 0 76816656 127680 0 32177628 0 176512
301237 0 77091084 134968 0 33967584 0 170384
297626 0 76147344 127505 0 32120424 0 174518
272750 0 69459012 50775 0 12817224 0 223945
269337 0 68588604 48128 0 12125232 0 223996
272580 0 69435576 51840 0 13059396 0 223085
284996 0 72806580 88704 0 22343328 0 198919
269933 0 68693184 50304 0 12672072 0 222521
271766 0 69234984 61376 0 15460956 0 212545
279384 0 71270640 77920 0 19626768 0 204539
279604 0 71364132 80896 0 20377476 0 201143
# netstat -I cxgb1 -w 1 -d
input (cxgb1) output
packets errs bytes packets errs bytes colls drops
262519 0 66155544 1016 0 256032 0 264350
264428 0 66628548 2032 0 512316 0 267643
280279 0 70714980 0 0 0 0 286592
278263 0 70213248 0 0 0 0 282679
270101 0 68067468 0 0 0 0 273504
270874 0 68286708 0 0 0 0 274618
273371 0 68894280 0 0 0 0 277436
268266 0 67634532 0 0 0 0 272268
272676 0 68751144 0 0 0 0 276489
266928 0 67281984 0 0 0 0 269646
261617 0 65890692 0 0 0 0 264105
262947 0 66242484 0 0 0 0 265603
262054 0 66049956 0 0 0 0 264045
263196 0 66303720 0 0 0 0 267500
270744 0 68263020 0 0 0 0 275050
278268 0 70206444 0 0 0 0 283764
266665 0 67211676 0 0 0 0 268860
263425 0 66380076 0 0 0 0 266193
262424 0 66126060 0 0 0 0 265852
266928 0 67285512 0 0 0 0 271427
128-byte frame size:
--------------------
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
| iface Rx Tx
Total
cxgb0: 253.88 Mb/s 39.65 Mb/s 293.54 Mb/s
cxgb1: 251.18 Mb/s 3.79 Mb/s 254.97 Mb/s
lo0: 0.00 b/s 0.00 b/s
0.00 b/s
total: 505.06 Mb/s 43.44 Mb/s 548.51 Mb/s
# netstat -I cxgb0 -w 1 -d
input (cxgb0) output
packets errs bytes packets errs bytes colls drops
272995 0 34151336 50464 0 6256048 0 222873
283224 0 35538152 71520 0 8865256 0 213793
282820 0 35460900 71680 0 8811688 0 214643
270570 0 33850884 48800 0 6041280 0 223482
268077 0 33469088 45440 0 5715780 0 225437
266423 0 33251840 52160 0 6466104 0 214914
267926 0 33494756 47680 0 5910212 0 221368
268975 0 33653476 44160 0 5474476 0 225070
273065 0 34201184 51072 0 6331936 0 223594
281093 0 35278868 70176 0 8698228 0 213052
290242 0 36473856 91424 0 11332732 0 201661
275838 0 34575664 59744 0 7405280 0 219405
271987 0 34020764 57984 0 7187412 0 216945
263805 0 32904640 50048 0 6204960 0 215690
266295 0 33242540 44192 0 5478940 0 223407
266219 0 33201992 45822 0 5629476 0 221918
264204 0 32959448 49090 0 6137256 0 214943
269254 0 33687948 43840 0 5435664 0 226363
270364 0 33849396 46016 0 5704992 0 225344
270689 0 33905196 50016 0 6199256 0 221875
# netstat -I cxgb1 -w 1 -d
input (cxgb1) output
packets errs bytes packets errs bytes colls drops
273869 0 33979720 0 0 0 0 278574
275475 0 34184444 0 0 0 0 280288
279181 0 34661100 0 0 0 0 284236
280953 0 34866816 0 0 0 0 286163
283458 0 35206824 0 0 0 0 289747
273164 0 33895400 0 0 0 0 277772
279365 0 34671392 0 0 0 0 283792
289865 0 35997696 0 0 0 0 296676
268840 0 33355256 0 0 0 0 273146
272989 0 33879528 0 0 0 0 277794
278536 0 34572192 0 0 0 0 283140
278738 0 34598108 0 0 0 0 284031
273043 0 33875436 0 0 0 0 277845
271792 0 33716592 0 0 0 0 275696
278252 0 34531644 0 0 0 0 283087
279277 0 34662216 0 0 0 0 284803
284664 0 35342356 0 0 0 0 291254
282126 0 35033100 0 0 0 0 288165
276860 0 34358044 0 0 0 0 281503
278605 0 34582236 0 0 0 0 284071
64-byte frame size:
--------------------
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
- iface Rx Tx
Total
cxgb0: 126.25 Mb/s 37.31 Mb/s 163.56 Mb/s
cxgb1: 124.84 Mb/s 6.42 Mb/s 131.26 Mb/s
lo0: 0.00 b/s 0.00 b/s
0.00 b/s
total: 251.09 Mb/s 43.73 Mb/s 294.82 Mb/s
# netstat -I cxgb0 -w 1 -d
input (cxgb0) output
packets errs bytes packets errs bytes colls drops
271659 0 16462260 0 0 4754160 0 194931
288256 0 17544540 0 0 6974640 0 175370
284450 0 17308560 0 0 7179120 0 168900
278013 0 16872960 0 0 5713800 0 187902
269222 0 16316160 0 0 4139760 0 200472
277638 0 16865280 0 0 6381600 0 174917
275845 0 16747800 0 0 5573220 0 187276
270689 0 16414200 0 0 4324140 0 201575
264941 0 16023360 0 0 3646020 0 205483
274697 0 16660320 0 0 5223120 0 189366
272307 0 16506180 0 0 5105520 0 190638
272725 0 16535220 0 0 4841400 0 194902
273420 0 16582560 0 0 5320260 0 188407
272601 0 16535640 0 0 4854420 0 194869
273774 0 16602060 0 0 4998180 0 193173
274004 0 16624800 0 0 5591220 0 185281
270268 0 16377120 0 0 4653120 0 194185
270037 0 16367460 0 0 4345560 0 199280
269547 0 16324920 0 0 4917600 0 190163
272384 0 16512600 0 0 5144160 0 188393
# netstat -I cxgb1 -w 1 -d
input (cxgb1) output
packets errs bytes packets errs bytes colls drops
272277 0 16347120 0 0 0 0 276873
276497 0 16610580 0 0 0 0 281353
270445 0 16240320 0 0 0 0 274578
275056 0 16522080 0 0 0 0 279691
277819 0 16690500 0 0 0 0 283827
277585 0 16672740 0 0 0 0 280820
282193 0 16963740 0 0 0 0 287826
270417 0 16233420 0 0 121380 0 271558
268628 0 16127280 0 0 0 0 273754
280227 0 16836120 0 0 0 0 285798
267618 0 16059900 0 0 0 0 271038
274748 0 16502460 0 0 0 0 279588
276467 0 16606200 0 0 0 0 280183
273376 0 16416900 0 0 0 0 277882
269455 0 16176780 0 0 0 0 273417
273813 0 16442760 0 0 0 0 277677
273845 0 16445760 0 0 0 0 276387
276677 0 16621920 0 0 0 0 282040
275206 0 16532700 0 0 0 0 280072
271826 0 16327200 0 0 0 0 276746
Regards,
Siquijor
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a27b90e40902242314w12c15fddma43e1cd5afec8938>