From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 19:44:43 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C02581065677 for ; Mon, 30 Jun 2008 19:44:43 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 9A04E8FC1E for ; Mon, 30 Jun 2008 19:44:43 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m5UJifQE005284; Mon, 30 Jun 2008 15:44:41 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m5UJifJD081781 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 30 Jun 2008 15:44:41 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200806301944.m5UJifJD081781@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 30 Jun 2008 15:44:48 -0400 To: Paul , FreeBSD Net From: Mike Tancsa In-Reply-To: <4867420D.7090406@gtcomm.net> References: <4867420D.7090406@gtcomm.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2008 19:44:43 -0000 At 04:04 AM 6/29/2008, Paul wrote: >This is just a question but who can get more than 400k pps >forwarding performance ? OK, I setup 2 boxes on either end of a RELENG_7 box from about May 7th just now, to see with 2 boxes blasting across it how it would work. *However*, this is with no firewall loaded and, I must enable ip fast forwarding. Without that enabled, the box just falls over. even at 20Kpps, I start seeing all sorts of messages spewing to route -n monitor got message of size 96 on Mon Jun 30 15:39:10 2008 RTM_MISS: Lookup failed on this address: len 96, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: default I am starting to wonder if those messages are the results of corrupted packets the machine just cant keep up with ? CPU is CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz (2660.01-MHz 686-class CPU) input (Total) output packets errs bytes packets errs bytes colls 611945 0 77892098 611955 0 77013002 0 616727 0 78215508 616742 0 77303454 0 617066 0 78162130 617082 0 77238434 0 618238 0 78302314 618225 0 77377582 0 617035 0 78141000 617038 0 77215672 0 617625 0 78225600 617588 0 77301734 0 616190 0 78017320 616165 0 77091774 0 615583 0 78064130 615628 0 77152800 0 617662 0 78254388 617658 0 77332340 0 618000 0 78269912 617950 0 77344554 0 617248 0 78183136 617315 0 77259588 0 617325 0 78204566 617289 0 77282094 0 618391 0 78337734 618357 0 77413756 0 616025 0 78116070 616082 0 77203116 0 To generate the packets, I am just using /usr/src/tools/tools/netblast on 2 endpoints starting at about the same time # ./netblast 10.10.1.2 500 100 40 start: 1214854131.083679919 finish: 1214854171.084668592 send calls: 20139141 send errors: 0 approx send rate: 503478 approx error rate: 0 # ./netblast 10.10.1.3 500 10 40 start: 1214854273.882202815 finish: 1214854313.882319031 send calls: 23354971 send errors: 18757223 approx send rate: 114943 approx error rate: 0 The box in the middle doing the forwarding 1[spare-r7]# ifconfig -u em0: flags=8843 metric 0 mtu 1500 options=19b ether 00:1b:21:08:32:a8 inet 10.20.1.1 netmask 0xffffff00 broadcast 10.20.1.255 media: Ethernet autoselect (1000baseTX ) status: active em1: flags=8843 metric 0 mtu 1500 options=9b ether 00:1b:21:08:32:a9 inet 192.168.43.193 netmask 0xffffff00 broadcast 192.168.43.255 media: Ethernet autoselect (100baseTX ) status: active em3: flags=8843 metric 0 mtu 1500 options=19b ether 00:30:48:90:4c:ff inet 10.10.1.1 netmask 0xffffff00 broadcast 10.10.1.255 media: Ethernet autoselect (1000baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 inet 127.0.0.1 netmask 0xff000000 I am going to try a few more tests with and without, firewall rules etc as well as an updated kernel to RELENG_7 as of today and see how that goes. ---Mike