From owner-freebsd-hackers Sun Dec 2 11:18:49 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 3429E37B416 for ; Sun, 2 Dec 2001 11:18:44 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id fB2JIg592414; Sun, 2 Dec 2001 11:18:42 -0800 (PST) (envelope-from dillon) Date: Sun, 2 Dec 2001 11:18:42 -0800 (PST) From: Matthew Dillon Message-Id: <200112021918.fB2JIg592414@apollo.backplane.com> To: Richard Sharpe Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Patch #3 (TCP / Linux / Performance) References: <20011128153817.T61580@monorchid.lemis.com> <15364.38174.938500.946169@caddis.yogotech.com> <20011128104629.A43642@walton.maths.tcd.ie> <5.1.0.14.1.20011130181236.00a80160@postamt1.charite.de> <200111302047.fAUKlT811090@apollo.backplane.com> <200111302130.fAULUU324648@apollo.backplane.com> <3C08CF9D.2030109@ns.aus.com> <200112012138.fB1LcG837063@apollo.backplane.com> <200112020810.fB28Arr77757@apollo.backplane.com> <3C0A3204.3010009@ns.aus.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :OK Matt, that last patch did the trick. : :I am now getting 68 and 69Mb/s between my Linux system and the FreeBSD :system. Excellent! :I have also tried the loopback interface, and I am getting 371Mb/s for 1 process, dropping to about 320Mb/s for 5. Excellent! :This seems like it is close to the limit for the machine I am using, as :CPU hits 100% when I ran the above tbench runs. : :I will have to try it with Gigabit Ethernet, but won't be able to do so :until next week or the week after (after I get to the US). Here's what I get over a gigabit ethernet link between two DELL2550's: ./tbench 1 10.1.0.1 Throughput 7.72693 MB/sec (NB=9.65866 MB/sec 77.2693 MBit/sec) 1 procs ./tbench 2 10.1.0.1 Throughput 17.1535 MB/sec (NB=21.4418 MB/sec 171.535 MBit/sec) 2 procs ./tbench 3 10.1.0.1 Throughput 23.185 MB/sec (NB=28.9813 MB/sec 231.85 MBit/sec) 3 procs ./tbench 4 10.1.0.1 Throughput 24.041 MB/sec (NB=30.0512 MB/sec 240.41 MBit/sec) 4 procs ./tbench 6 10.1.0.1 Throughput 33.8823 MB/sec (NB=42.3528 MB/sec 338.823 MBit/sec) 6 procs ./tbench 8 10.1.0.1 Throughput 40.9986 MB/sec (NB=51.2482 MB/sec 409.986 MBit/sec) 8 procs ./tbench 10 10.1.0.1 Throughput 43.8519 MB/sec (NB=54.8149 MB/sec 438.519 MBit/sec) 10 procs ./tbench 15 10.1.0.1 Throughput 46.8878 MB/sec (NB=58.6098 MB/sec 468.878 MBit/sec) 15 procs ./tbench 20 10.1.0.1 Throughput 47.2446 MB/sec (NB=59.0558 MB/sec 472.446 MBit/sec) 20 procs It seems to max-out at around 75,000 packets per second (input + output). I doubt these results could be duplicated on anything but a DELL2550. It dedicates an entire internal 64 bit 66MHz PCI bus just to the on-board gigabit ethernet. test1:/home/dillon/dbench> netstat -in 1 input (Total) output packets errs bytes packets errs bytes colls 38680 0 37237815 36340 0 20079011 0 41770 0 45968686 33317 0 13029493 0 33282 0 17257495 38002 0 29369102 0 ... :Does the FreeBSD tcp stack do zero copy (page flip the data to :userspace)? In the localhost case, it seems like there are two copies :to/from userspace there. : :-- :Richard Sharpe, rsharpe@ns.aus.com, LPIC-1 There are zero-copy patches floating around but I haven't looked at them to determine how messy they might be. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message