From owner-freebsd-performance@FreeBSD.ORG Fri Oct 16 03:52:10 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15818106566C for ; Fri, 16 Oct 2009 03:52:10 +0000 (UTC) (envelope-from htyin@huawei.com) Received: from usaga02-in.huawei.com (usaga02-in.huawei.com [206.16.17.70]) by mx1.freebsd.org (Postfix) with ESMTP id CFF668FC19 for ; Fri, 16 Oct 2009 03:52:09 +0000 (UTC) Received: from huawei.com (localhost [127.0.0.1]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0KRL00HXT9EVIG@usaga02-in.huawei.com> for freebsd-performance@freebsd.org; Thu, 15 Oct 2009 20:52:08 -0700 (PDT) Received: from Hongtao1SC ([10.193.34.152]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPA id <0KRL00J6R9EQS8@usaga02-in.huawei.com> for freebsd-performance@freebsd.org; Thu, 15 Oct 2009 20:52:07 -0700 (PDT) Date: Thu, 15 Oct 2009 20:52:02 -0700 From: Hongtao Yin In-reply-to: To: =?gb2312?B?J0lzdHaoom4n?= Message-id: <4831593800614E6796A45F20BA4B818E@china.huawei.com> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Office Outlook 11 Content-type: multipart/mixed; boundary="Boundary_(ID_AAjka9hhoFsx6rdqZWl8ZQ)" Thread-index: AcpNkNPyM/T7sWEVTyKaneG8WESK0wAgHXSA References: <78DB4AE8EF5F4A1EBD3992D7404B2725@china.huawei.com> X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: RE: Comparison of FreeBSD/Linux TCP Throughput performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2009 03:52:10 -0000 This is a multi-part message in MIME format. --Boundary_(ID_AAjka9hhoFsx6rdqZWl8ZQ) Content-type: text/plain; charset=gb2312 Content-transfer-encoding: quoted-printable Istvan, =20 I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we = collected test data and the result shows that FreeBSD TCP performance is worse = than Linux. I got troubles to plot using gnuplot, so here I am attaching the = raw data files. The netpipe commands are as follows: PC2: #NPtcp PC1: #NPtcp =A8Ch 192.168.1.20 =20 Regards, Hongtao =20 =20 =20 _____ =20 From: Istv=A8=A2n [mailto:leccine@gmail.com]=20 Sent: 2009=C4=EA10=D4=C215=C8=D5 5:13 To: Hongtao Yin Cc: freebsd-performance@freebsd.org Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance =20 use netpipe TCP pls. =20 And graph it. =20 Thanks in advance, Istvan On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin wrote: Hi, I compared TCP performance between FreeBSD and Linux by running test = tools Netperf and Iperf with Intel NIC. The kernels are full version and default values are used in the testing except TCP Congestion Control algorithm set to Reno. >From the test results we can see Linux TCP performance in throughput is better than FreeBSD. The worst case (send msg size 128) shows that = FreeBSD throughput is only 43% of Linux's. I like to get some feedback if anyone did similar comparison test, or = knows any issues with kernels or drivers. Thanks lot. FreeBSD and Linux Sysctl captures are attached for reference. Regards, Hongtao Test Environments: PC: Dell Precision T3400 (same 4 PCs) CPU: Intel Core 2 Duo CPU E4600@2.4Ghz FreeBSD: V7.1 (full version) (TCP CC: newReno) Linux: V2.6.31.1 (full version) (TCP CC: Reno) Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter = (Gigabit) chip 82546EB (only one port used for each PC) Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108) Iperf: V2.0.4 Netperf: V2.4.4 Setup: ---------- | switch | ---------- ---------------------| | | |-------------------- | | | | | --------| |-------- | | | | | | | | | -------------- -------------- -------------- -------------- | PC1 | | PC2 | | PC3 | | PC4 | | FreeBSD | | FreeBSD | | Linux | | Linux | |192.168.1.10| |192.168.1.20| |192.168.1.30| |192.168.1.40| -------------- -------------- -------------- -------------- =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D Netperf Test Results =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D TCP Throughput Test ------------------- PC2/4: #netserver -p 22113 PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10 Recv Send Send Elapsed Throughput Socket Socket Message Time 10^6 bits/sec Size Size Size Sec. bytes bytes bytes FreeBSD: 65536 32768 32768 10.34 598.11 Linux: 87380 16384 16384 10.04 779.02 PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m 64/128/256/512/1024/2048/4096 Recv Send Send Elapsed Throughput Socket Socket Message Time 10^6 bits/sec Size Size Size Sec. bytes bytes bytes FreeBSD: 65536 32768 64 10.19 417.10 65536 32768 128 10.35 336.63 65536 32768 256 10.36 576.99 65536 32768 512 10.35 569.79 65536 32768 1024 10.35 553.70 65536 32768 2048 10.35 584.20 65536 32768 4096 10.35 602.45 Linux: 87380 16384 64 10.03 778.21 87380 16384 128 10.03 779.72 87380 16384 256 10.04 780.16 87380 16384 512 10.03 776.85 87380 16384 1024 10.04 777.52 87380 16384 2048 10.04 777.83 87380 16384 4096 10.03 780.17 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D Iperf Test Results =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D Bandwidth Test -------------- PC2/4: #iperf -s PC1/3: #iperf -c 192.168.1.20 Interval Transfer Bandwidth sec MBytes Mbits/sec FreeBSD: 0.0-10.3 740 600 Linux: 0.0-10.0 972 815 PC1/3: #iperf -c 192.168.1.20 -d Interval Transfer Bandwidth sec MBytes Mbits/sec FreeBSD: 0.0-10.0 402 337 0.0-10.0 404 338 Linux: 0.0-10.0 926 776 0.0-10.0 44.1 36.9 Parallel Test ------------- PC2/4: #iperf -s PC1/3: #iperf -c 192.168.1.20 -P 2 Interval Transfer Bandwidth sec MBytes Mbits/sec FreeBSD: 0.0-10.3 370 300 0.0-10.3 370 300 SUM: 0.0-10.3 739 600 Linux: 0.0-10.0 479 402 0.0-10.0 473 396 SUM: 0.0-10.0 952 797 _______________________________________________ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" --=20 the sun shines for all --Boundary_(ID_AAjka9hhoFsx6rdqZWl8ZQ) Content-type: application/octet-stream; name=linux_netpipe Content-transfer-encoding: 7bit Content-disposition: attachment; filename=linux_netpipe 0: 1 bytes 1936 times --> 0.15 Mbps in 50.54 usec 1: 2 bytes 1978 times --> 0.30 Mbps in 50.62 usec 2: 3 bytes 1975 times --> 0.45 Mbps in 50.49 usec 3: 4 bytes 1320 times --> 0.60 Mbps in 50.45 usec 4: 6 bytes 1486 times --> 0.91 Mbps in 50.56 usec 5: 8 bytes 988 times --> 1.21 Mbps in 50.35 usec 6: 12 bytes 1241 times --> 1.81 Mbps in 50.49 usec 7: 13 bytes 825 times --> 1.97 Mbps in 50.46 usec 8: 16 bytes 914 times --> 2.42 Mbps in 50.52 usec 9: 19 bytes 1113 times --> 2.85 Mbps in 50.79 usec 10: 21 bytes 1243 times --> 3.17 Mbps in 50.54 usec 11: 24 bytes 1318 times --> 3.62 Mbps in 50.53 usec 12: 27 bytes 1401 times --> 4.09 Mbps in 50.40 usec 13: 29 bytes 881 times --> 4.38 Mbps in 50.46 usec 14: 32 bytes 956 times --> 4.85 Mbps in 50.36 usec 15: 35 bytes 1054 times --> 5.30 Mbps in 50.42 usec 16: 45 bytes 1133 times --> 6.82 Mbps in 50.36 usec 17: 48 bytes 1323 times --> 7.29 Mbps in 50.22 usec 18: 51 bytes 1368 times --> 7.73 Mbps in 50.31 usec 19: 61 bytes 779 times --> 9.29 Mbps in 50.08 usec 20: 64 bytes 982 times --> 9.75 Mbps in 50.07 usec 21: 67 bytes 1029 times --> 10.18 Mbps in 50.24 usec 22: 93 bytes 1069 times --> 14.07 Mbps in 50.42 usec 23: 96 bytes 1322 times --> 14.50 Mbps in 50.51 usec 24: 99 bytes 1340 times --> 15.00 Mbps in 50.35 usec 25: 125 bytes 722 times --> 18.88 Mbps in 50.51 usec 26: 128 bytes 982 times --> 19.33 Mbps in 50.51 usec 27: 131 bytes 1005 times --> 19.81 Mbps in 50.45 usec 28: 189 bytes 1028 times --> 28.17 Mbps in 51.18 usec 29: 192 bytes 1302 times --> 28.84 Mbps in 50.78 usec 30: 195 bytes 1322 times --> 29.20 Mbps in 50.95 usec 31: 253 bytes 684 times --> 35.91 Mbps in 53.75 usec 32: 256 bytes 926 times --> 36.58 Mbps in 53.39 usec 33: 259 bytes 943 times --> 35.04 Mbps in 56.39 usec 34: 381 bytes 903 times --> 36.09 Mbps in 80.53 usec 35: 384 bytes 827 times --> 37.61 Mbps in 77.90 usec 36: 387 bytes 859 times --> 35.44 Mbps in 83.32 usec 37: 509 bytes 409 times --> 38.92 Mbps in 99.78 usec 38: 512 bytes 500 times --> 39.42 Mbps in 99.09 usec 39: 515 bytes 506 times --> 39.37 Mbps in 99.79 usec 40: 765 bytes 505 times --> 58.48 Mbps in 99.81 usec 41: 768 bytes 667 times --> 58.71 Mbps in 99.81 usec 42: 771 bytes 669 times --> 58.92 Mbps in 99.84 usec 43: 1021 bytes 337 times --> 78.04 Mbps in 99.82 usec 44: 1024 bytes 500 times --> 78.25 Mbps in 99.84 usec 45: 1027 bytes 501 times --> 78.48 Mbps in 99.84 usec 46: 1533 bytes 503 times --> 113.72 Mbps in 102.85 usec 47: 1536 bytes 648 times --> 117.04 Mbps in 100.12 usec 48: 1539 bytes 666 times --> 114.54 Mbps in 102.51 usec 49: 2045 bytes 327 times --> 153.81 Mbps in 101.44 usec 50: 2048 bytes 492 times --> 149.29 Mbps in 104.66 usec 51: 2051 bytes 478 times --> 150.08 Mbps in 104.27 usec 52: 3069 bytes 480 times --> 183.63 Mbps in 127.51 usec 53: 3072 bytes 522 times --> 186.45 Mbps in 125.71 usec 54: 3075 bytes 530 times --> 187.94 Mbps in 124.83 usec 55: 4093 bytes 267 times --> 229.30 Mbps in 136.18 usec 56: 4096 bytes 367 times --> 235.25 Mbps in 132.84 usec 57: 4099 bytes 376 times --> 227.97 Mbps in 137.18 usec 58: 6141 bytes 364 times --> 309.08 Mbps in 151.59 usec 59: 6144 bytes 439 times --> 310.29 Mbps in 151.07 usec 60: 6147 bytes 441 times --> 310.13 Mbps in 151.22 usec 61: 8189 bytes 220 times --> 356.91 Mbps in 175.05 usec 62: 8192 bytes 285 times --> 357.32 Mbps in 174.91 usec 63: 8195 bytes 285 times --> 358.05 Mbps in 174.62 usec 64: 12285 bytes 286 times --> 375.11 Mbps in 249.86 usec 65: 12288 bytes 266 times --> 375.21 Mbps in 249.86 usec 66: 12291 bytes 266 times --> 375.30 Mbps in 249.86 usec 67: 16381 bytes 133 times --> 353.40 Mbps in 353.65 usec 68: 16384 bytes 141 times --> 333.80 Mbps in 374.48 usec 69: 16387 bytes 133 times --> 353.42 Mbps in 353.75 usec 70: 24573 bytes 141 times --> 394.92 Mbps in 474.73 usec 71: 24576 bytes 140 times --> 390.87 Mbps in 479.70 usec 72: 24579 bytes 138 times --> 435.83 Mbps in 430.27 usec 73: 32765 bytes 77 times --> 500.27 Mbps in 499.68 usec 74: 32768 bytes 100 times --> 500.29 Mbps in 499.71 usec 75: 32771 bytes 100 times --> 500.33 Mbps in 499.72 usec 76: 49149 bytes 100 times --> 556.87 Mbps in 673.36 usec 77: 49152 bytes 99 times --> 551.50 Mbps in 679.96 usec 78: 49155 bytes 98 times --> 552.23 Mbps in 679.10 usec 79: 65533 bytes 49 times --> 566.89 Mbps in 881.97 usec 80: 65536 bytes 56 times --> 555.27 Mbps in 900.46 usec 81: 65539 bytes 55 times --> 570.94 Mbps in 875.79 usec 82: 98301 bytes 57 times --> 633.33 Mbps in 1184.18 usec 83: 98304 bytes 56 times --> 626.79 Mbps in 1196.57 usec 84: 98307 bytes 55 times --> 630.89 Mbps in 1188.84 usec 85: 131069 bytes 28 times --> 645.60 Mbps in 1548.91 usec 86: 131072 bytes 32 times --> 647.52 Mbps in 1544.35 usec 87: 131075 bytes 32 times --> 652.15 Mbps in 1533.42 usec 88: 196605 bytes 32 times --> 683.08 Mbps in 2195.89 usec 89: 196608 bytes 30 times --> 682.95 Mbps in 2196.37 usec 90: 196611 bytes 30 times --> 685.38 Mbps in 2188.62 usec 91: 262141 bytes 15 times --> 696.10 Mbps in 2873.13 usec 92: 262144 bytes 17 times --> 696.48 Mbps in 2871.59 usec 93: 262147 bytes 17 times --> 696.58 Mbps in 2871.20 usec 94: 393213 bytes 17 times --> 702.85 Mbps in 4268.32 usec 95: 393216 bytes 15 times --> 702.86 Mbps in 4268.30 usec 96: 393219 bytes 15 times --> 704.15 Mbps in 4260.47 usec 97: 524285 bytes 7 times --> 709.22 Mbps in 5640.00 usec 98: 524288 bytes 8 times --> 709.20 Mbps in 5640.12 usec 99: 524291 bytes 8 times --> 708.82 Mbps in 5643.25 usec 100: 786429 bytes 8 times --> 716.97 Mbps in 8368.51 usec 101: 786432 bytes 7 times --> 713.89 Mbps in 8404.63 usec 102: 786435 bytes 7 times --> 714.01 Mbps in 8403.28 usec 103: 1048573 bytes 3 times --> 718.93 Mbps in 11127.67 usec 104: 1048576 bytes 4 times --> 716.32 Mbps in 11168.12 usec 105: 1048579 bytes 4 times --> 716.39 Mbps in 11167.11 usec 106: 1572861 bytes 4 times --> 726.29 Mbps in 16522.26 usec 107: 1572864 bytes 4 times --> 719.14 Mbps in 16686.50 usec 108: 1572867 bytes 3 times --> 718.72 Mbps in 16696.33 usec 109: 2097149 bytes 3 times --> 731.29 Mbps in 21879.00 usec 110: 2097152 bytes 3 times --> 722.07 Mbps in 22158.50 usec 111: 2097155 bytes 3 times --> 724.44 Mbps in 22086.18 usec 112: 3145725 bytes 3 times --> 733.23 Mbps in 32731.65 usec 113: 3145728 bytes 3 times --> 735.02 Mbps in 32652.18 usec 114: 3145731 bytes 3 times --> 735.73 Mbps in 32620.83 usec 115: 4194301 bytes 3 times --> 734.49 Mbps in 43567.34 usec 116: 4194304 bytes 3 times --> 736.95 Mbps in 43422.50 usec 117: 4194307 bytes 3 times --> 736.49 Mbps in 43449.64 usec 118: 6291453 bytes 3 times --> 740.40 Mbps in 64830.03 usec 119: 6291456 bytes 3 times --> 740.67 Mbps in 64806.34 usec 120: 6291459 bytes 3 times --> 739.82 Mbps in 64880.49 usec 121: 8388605 bytes 3 times --> 737.90 Mbps in 86732.19 usec 122: 8388608 bytes 3 times --> 734.54 Mbps in 87129.87 usec 123: 8388611 bytes 3 times --> 738.20 Mbps in 86697.14 usec --Boundary_(ID_AAjka9hhoFsx6rdqZWl8ZQ) Content-type: application/octet-stream; name=freebsd_netpipe Content-transfer-encoding: 7bit Content-disposition: attachment; filename=freebsd_netpipe 0: 1 bytes 1593 times --> 0.12 Mbps in 62.50 usec 1: 2 bytes 1599 times --> 0.24 Mbps in 62.46 usec 2: 3 bytes 1601 times --> 0.37 Mbps in 62.46 usec 3: 4 bytes 1067 times --> 0.49 Mbps in 62.51 usec 4: 6 bytes 1199 times --> 0.73 Mbps in 62.51 usec 5: 8 bytes 799 times --> 0.98 Mbps in 62.54 usec 6: 12 bytes 999 times --> 1.47 Mbps in 62.46 usec 7: 13 bytes 667 times --> 1.59 Mbps in 62.45 usec 8: 16 bytes 739 times --> 1.95 Mbps in 62.46 usec 9: 19 bytes 900 times --> 2.32 Mbps in 62.46 usec 10: 21 bytes 1011 times --> 2.57 Mbps in 62.46 usec 11: 24 bytes 1067 times --> 2.93 Mbps in 62.46 usec 12: 27 bytes 1134 times --> 3.30 Mbps in 62.51 usec 13: 29 bytes 710 times --> 3.54 Mbps in 62.46 usec 14: 32 bytes 772 times --> 3.91 Mbps in 62.45 usec 15: 35 bytes 850 times --> 4.27 Mbps in 62.53 usec 16: 45 bytes 913 times --> 5.50 Mbps in 62.46 usec 17: 48 bytes 1067 times --> 5.86 Mbps in 62.46 usec 18: 51 bytes 1100 times --> 6.23 Mbps in 62.46 usec 19: 61 bytes 627 times --> 7.45 Mbps in 62.45 usec 20: 64 bytes 787 times --> 7.82 Mbps in 62.47 usec 21: 67 bytes 825 times --> 8.18 Mbps in 62.46 usec 22: 93 bytes 860 times --> 11.36 Mbps in 62.46 usec 23: 96 bytes 1067 times --> 11.72 Mbps in 62.52 usec 24: 99 bytes 1083 times --> 12.09 Mbps in 62.46 usec 25: 125 bytes 582 times --> 15.27 Mbps in 62.46 usec 26: 128 bytes 794 times --> 15.64 Mbps in 62.45 usec 27: 131 bytes 813 times --> 16.00 Mbps in 62.46 usec 28: 189 bytes 831 times --> 23.09 Mbps in 62.46 usec 29: 192 bytes 1067 times --> 23.46 Mbps in 62.45 usec 30: 195 bytes 1075 times --> 23.82 Mbps in 62.46 usec 31: 253 bytes 558 times --> 30.91 Mbps in 62.46 usec 32: 256 bytes 797 times --> 31.27 Mbps in 62.47 usec 33: 259 bytes 806 times --> 31.60 Mbps in 62.53 usec 34: 381 bytes 815 times --> 46.48 Mbps in 62.53 usec 35: 384 bytes 1066 times --> 46.86 Mbps in 62.52 usec 36: 387 bytes 1070 times --> 47.22 Mbps in 62.52 usec 37: 509 bytes 545 times --> 61.95 Mbps in 62.68 usec 38: 512 bytes 796 times --> 62.31 Mbps in 62.69 usec 39: 515 bytes 800 times --> 62.44 Mbps in 62.93 usec 40: 765 bytes 802 times --> 91.83 Mbps in 63.55 usec 41: 768 bytes 1048 times --> 92.06 Mbps in 63.65 usec 42: 771 bytes 1049 times --> 92.58 Mbps in 63.54 usec 43: 1021 bytes 530 times --> 73.94 Mbps in 105.35 usec 44: 1024 bytes 474 times --> 74.21 Mbps in 105.27 usec 45: 1027 bytes 475 times --> 74.57 Mbps in 105.07 usec 46: 1533 bytes 478 times --> 82.34 Mbps in 142.04 usec 47: 1536 bytes 469 times --> 83.80 Mbps in 139.84 usec 48: 1539 bytes 477 times --> 83.66 Mbps in 140.36 usec 49: 2045 bytes 238 times --> 101.61 Mbps in 153.55 usec 50: 2048 bytes 325 times --> 100.01 Mbps in 156.23 usec 51: 2051 bytes 320 times --> 101.09 Mbps in 154.79 usec 52: 3069 bytes 323 times --> 125.49 Mbps in 186.59 usec 53: 3072 bytes 357 times --> 125.44 Mbps in 186.85 usec 54: 3075 bytes 356 times --> 125.43 Mbps in 187.05 usec 55: 4093 bytes 178 times --> 166.69 Mbps in 187.34 usec 56: 4096 bytes 266 times --> 166.78 Mbps in 187.37 usec 57: 4099 bytes 266 times --> 167.11 Mbps in 187.14 usec 58: 6141 bytes 267 times --> 250.04 Mbps in 187.38 usec 59: 6144 bytes 355 times --> 250.19 Mbps in 187.36 usec 60: 6147 bytes 355 times --> 250.31 Mbps in 187.36 usec 61: 8189 bytes 178 times --> 329.14 Mbps in 189.82 usec 62: 8192 bytes 263 times --> 329.81 Mbps in 189.50 usec 63: 8195 bytes 263 times --> 329.40 Mbps in 189.81 usec 64: 12285 bytes 263 times --> 375.15 Mbps in 249.84 usec 65: 12288 bytes 266 times --> 375.00 Mbps in 250.00 usec 66: 12291 bytes 266 times --> 374.86 Mbps in 250.15 usec 67: 16381 bytes 133 times --> 401.20 Mbps in 311.51 usec 68: 16384 bytes 160 times --> 400.31 Mbps in 312.26 usec 69: 16387 bytes 160 times --> 400.41 Mbps in 312.23 usec 70: 24573 bytes 160 times --> 430.30 Mbps in 435.68 usec 71: 24576 bytes 153 times --> 430.81 Mbps in 435.23 usec 72: 24579 bytes 153 times --> 430.09 Mbps in 436.01 usec 73: 32765 bytes 76 times --> 479.72 Mbps in 521.09 usec 74: 32768 bytes 95 times --> 478.95 Mbps in 521.97 usec 75: 32771 bytes 95 times --> 479.98 Mbps in 520.90 usec 76: 49149 bytes 96 times --> 512.29 Mbps in 731.96 usec 77: 49152 bytes 91 times --> 509.08 Mbps in 736.62 usec 78: 49155 bytes 90 times --> 508.77 Mbps in 737.11 usec 79: 65533 bytes 45 times --> 512.45 Mbps in 975.67 usec 80: 65536 bytes 51 times --> 510.17 Mbps in 980.07 usec 81: 65539 bytes 51 times --> 513.75 Mbps in 973.28 usec 82: 98301 bytes 51 times --> 536.30 Mbps in 1398.42 usec 83: 98304 bytes 47 times --> 535.35 Mbps in 1400.96 usec 84: 98307 bytes 47 times --> 535.64 Mbps in 1400.23 usec 85: 131069 bytes 23 times --> 540.77 Mbps in 1849.17 usec 86: 131072 bytes 27 times --> 539.68 Mbps in 1852.95 usec 87: 131075 bytes 26 times --> 542.03 Mbps in 1844.96 usec 88: 196605 bytes 27 times --> 554.36 Mbps in 2705.76 usec 89: 196608 bytes 24 times --> 550.97 Mbps in 2722.48 usec 90: 196611 bytes 24 times --> 553.04 Mbps in 2712.31 usec 91: 262141 bytes 12 times --> 553.75 Mbps in 3611.70 usec 92: 262144 bytes 13 times --> 552.19 Mbps in 3621.96 usec 93: 262147 bytes 13 times --> 552.69 Mbps in 3618.70 usec 94: 393213 bytes 13 times --> 319.51 Mbps in 9389.38 usec 95: 393216 bytes 7 times --> 558.48 Mbps in 5371.72 usec 96: 393219 bytes 12 times --> 310.32 Mbps in 9667.63 usec 97: 524285 bytes 3 times --> 562.05 Mbps in 7116.79 usec 98: 524288 bytes 7 times --> 561.72 Mbps in 7120.93 usec 99: 524291 bytes 7 times --> 561.72 Mbps in 7121.07 usec 100: 786429 bytes 7 times --> 564.83 Mbps in 10622.71 usec 101: 786432 bytes 6 times --> 564.31 Mbps in 10632.50 usec 102: 786435 bytes 6 times --> 564.50 Mbps in 10628.92 usec 103: 1048573 bytes 3 times --> 566.81 Mbps in 14114.14 usec 104: 1048576 bytes 3 times --> 566.79 Mbps in 14114.50 usec 105: 1048579 bytes 3 times --> 565.11 Mbps in 14156.50 usec 106: 1572861 bytes 3 times --> 568.89 Mbps in 21093.53 usec 107: 1572864 bytes 3 times --> 569.04 Mbps in 21088.16 usec 108: 1572867 bytes 3 times --> 569.20 Mbps in 21082.16 usec 109: 2097149 bytes 3 times --> 353.93 Mbps in 45206.03 usec 110: 2097152 bytes 3 times --> 567.81 Mbps in 28178.33 usec 111: 2097155 bytes 3 times --> 565.47 Mbps in 28295.00 usec 112: 3145725 bytes 3 times --> 313.65 Mbps in 76518.81 usec 113: 3145728 bytes 3 times --> 564.73 Mbps in 42498.35 usec 114: 3145731 bytes 3 times --> 403.88 Mbps in 59423.49 usec 115: 4194301 bytes 3 times --> 438.02 Mbps in 73055.35 usec 116: 4194304 bytes 3 times --> 437.75 Mbps in 73100.49 usec 117: 4194307 bytes 3 times --> 298.08 Mbps in 107355.16 usec 118: 6291453 bytes 3 times --> 404.09 Mbps in 118785.50 usec 119: 6291456 bytes 3 times --> 354.53 Mbps in 135389.84 usec 120: 6291459 bytes 3 times --> 315.14 Mbps in 152314.66 usec 121: 8388605 bytes 3 times --> 390.67 Mbps in 163821.34 usec 122: 8388608 bytes 3 times --> 297.76 Mbps in 214937.33 usec 123: 8388611 bytes 3 times --> 324.16 Mbps in 197430.69 usec --Boundary_(ID_AAjka9hhoFsx6rdqZWl8ZQ)--