From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 00:58:20 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 915831065670 for ; Sun, 6 Jul 2008 00:58:19 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 692D68FC0C for ; Sun, 6 Jul 2008 00:58:19 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-179-28.hsd1.fl.comcast.net ([76.108.179.28] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KFIWD-0003uQ-J2; Sat, 05 Jul 2008 20:54:21 -0400 Message-ID: <48701921.7090107@gtcomm.net> Date: Sat, 05 Jul 2008 21:00:17 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Bart Van Kerckhove References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> In-Reply-To: <486FFF70.3090402@gtcomm.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 00:58:20 -0000 UP 32 bit test vs 64 bit: negligible difference in forwarding performance without polling slightly better polling performance but still errors at lower packet rates same massive hit with ipfw loaded Installing dragonfly in a bit.. If anyone has a really fast PPC type system or SUN or something i'd love to try it :) Something with a really big L1 cache :P Paul wrote: > ULE + PREEMPTION for non SMP > no major differences with SMP with ULE/4BSD and preemption ON/OFF > > 32 bit UP test coming up with new cpu > and I'm installing dragonfly sometime this weekend :] > UP: 1mpps in one direction with no firewall/no routing table is not > too bad, but 1mpps both directions is the goal here > 700kpps with full bgp table in one direction is not too bad > Ipfw needs a lot of work, barely gets 500kpps with no routing table > with a few ipfw rules loaded.. that's horrible > Linux barely takes a hit when you start loading iptables rules , but > then again linux has a HUGE problem with routing > random packet sources/ports .. grr > My problem Is I need some box to do fast routing and some to do > firewall.. :/ > I'll have 32 bit 7-stable UP test with ipfw/routing table and then > move on to dragonfly. > I'll post the dragonfly results here as well as sign up for their > mailing list. > > > Bart Van Kerckhove wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Paul / Ingo, >> >>>> I tried all of this :/ still, 256/512 descriptors seem to work the >>>> best. Happy to let you log into the machine and fiddle around if you >>>> want :) >> I've been watching this thread closely, since I'm in a very similair >> situation. >> A few questions/remarks: >> >> Does ULE provide better performance than 4BSD for forwarding? >> Did you try freebsd4 as well? This thread had a report about that quite >> opposite to my own experiences, -4 seemed to be a lot faster at >> forwarding >> than anything else I 've tried so far. >> Obviously the thing I'm interested in is IMIX - and 64byte packets. >> Does anyone have any benchmarks for DragonFly? I asked around on IRC, >> but >> that nor google turned up any useful results. >> >> >>> I don't think you will be able to route 64byte packets at 1gbit >>> wirespeed (2Mpps) with a current x86 platform. >>> >> Are there actual hardware related reasons this should not be >> possible, or >> is this purely lack of dedicated work towards this goal? >> >> >> >>> Theres a "sun" used at quagga dev as bgp-route-server. >>> http://quagga.net/route-server.php >>> (but they don't answered my question regarding fw-performance). >>> >> >> >> the Quagga guys are running a sun T1000 (niagara 1) route server - I >> happen >> to have the machine in my racks, >> please let me know if you want to run some tests on it, I'm sure they >> won't >> mind ;-) >> It should also make a great testbed for SMP performance testing imho >> (and >> they're pretty cheap these days) >> Also, feel free to use me as a relay for your questions, they're not >> always >> very reachable. >> >> >> >>> Perhaps you have some better luck at some different hardware systems >>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and >>> special network-cards (netfpga) for real routing. >>> >> The netfpga site seems more or less dead - is this project still alive? >> It does look like a very interesting idea, even though it's currently >> quite >> linux-centric (and according to docs doesn't have VLAN nor ip6 >> support, the >> former being quite a dealbreaker) >> >> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a >> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of >> information you are providing us :) >> >> Met vriendelijke groet / With kind regards, >> >> Bart Van Kerckhove >> http://friet.net/pgp.txt >> >> -----BEGIN PGP SIGNATURE----- >> >> iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi >> eca31f7WQ/oXq9tJ8TEDN3CA >> =YGYq >> -----END PGP SIGNATURE----- >> >> >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 01:06:16 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D8E0106567A for ; Sun, 6 Jul 2008 01:06:16 +0000 (UTC) (envelope-from bart@it-ss.be) Received: from mx20.it-ss.be (mx20.it-ss.be [195.28.164.230]) by mx1.freebsd.org (Postfix) with ESMTP id A40888FC19 for ; Sun, 6 Jul 2008 01:06:15 +0000 (UTC) (envelope-from bart@it-ss.be) Received: from authrelay.it-ss.be (authrelay.it-ss.be [195.28.164.225]) by mx20.it-ss.be (8.13.8/8.13.8) with ESMTP id m65Mrcnr013545 for ; Sun, 6 Jul 2008 00:53:38 +0200 Received: from bartwrkstxp (220.108-242-81.adsl-dyn.isp.belgacom.be [81.242.108.220]) (authenticated bits=0) by mx10.it-ss.be (8.13.6/8.13.6) with ESMTP id m65MrVdZ007311; Sun, 6 Jul 2008 00:53:33 +0200 Message-ID: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> From: "Bart Van Kerckhove" To: "Ingo Flaschberger" , "Paul" References: <4867420D.7090406@gtcomm.net> <486986D9.3000607@monkeybrains.net><48699960.9070100@gtcomm.net><20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> Date: Sun, 6 Jul 2008 00:52:53 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Scanned-By: MIMEDefang 2.63 on 195.28.164.230 X-Scanned-By: MIMEDefang 2.63 on 195.28.164.225 Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 01:06:16 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Paul / Ingo, > >> I tried all of this :/ still, 256/512 descriptors seem to work the >> best. Happy to let you log into the machine and fiddle around if you >> want :) I've been watching this thread closely, since I'm in a very similair situation. A few questions/remarks: Does ULE provide better performance than 4BSD for forwarding? Did you try freebsd4 as well? This thread had a report about that quite opposite to my own experiences, -4 seemed to be a lot faster at forwarding than anything else I 've tried so far. Obviously the thing I'm interested in is IMIX - and 64byte packets. Does anyone have any benchmarks for DragonFly? I asked around on IRC, but that nor google turned up any useful results. > I don't think you will be able to route 64byte packets at 1gbit > wirespeed (2Mpps) with a current x86 platform. Are there actual hardware related reasons this should not be possible, or is this purely lack of dedicated work towards this goal? >Theres a "sun" used at quagga dev as bgp-route-server. >http://quagga.net/route-server.php >(but they don't answered my question regarding fw-performance). the Quagga guys are running a sun T1000 (niagara 1) route server - I happen to have the machine in my racks, please let me know if you want to run some tests on it, I'm sure they won't mind ;-) It should also make a great testbed for SMP performance testing imho (and they're pretty cheap these days) Also, feel free to use me as a relay for your questions, they're not always very reachable. > Perhaps you have some better luck at some different hardware systems > (ppc, mips, ..?) or use freebsd only for routing-table-updates and > special network-cards (netfpga) for real routing. The netfpga site seems more or less dead - is this project still alive? It does look like a very interesting idea, even though it's currently quite linux-centric (and according to docs doesn't have VLAN nor ip6 support, the former being quite a dealbreaker) Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of information you are providing us :) Met vriendelijke groet / With kind regards, Bart Van Kerckhove http://friet.net/pgp.txt -----BEGIN PGP SIGNATURE----- iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi eca31f7WQ/oXq9tJ8TEDN3CA =YGYq -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 03:40:33 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D82F106568E for ; Sun, 6 Jul 2008 03:40:33 +0000 (UTC) (envelope-from cmarlatt@rxsec.com) Received: from core.rxsec.com (core.rxsec.com [64.132.46.102]) by mx1.freebsd.org (Postfix) with SMTP id AE5718FC0A for ; Sun, 6 Jul 2008 03:40:32 +0000 (UTC) (envelope-from cmarlatt@rxsec.com) Received: (qmail 93269 invoked by uid 2009); 6 Jul 2008 03:02:37 -0000 Received: from 10.1.0.239 by core.rxsec.com (envelope-from , uid 2008) with qmail-scanner-1.25-st-qms (clamdscan: 0.86.2/1102. spamassassin: 3.0.4. perlscan: 1.25-st-qms. Clear:RC:0(10.1.0.239):SA:0(-4.4/5.0):. Processed in 6.064556 secs); 06 Jul 2008 03:02:38 -0000 X-Spam-Status: No, hits=-4.4 required=5.0 X-Antivirus-RXSEC-Mail-From: cmarlatt@rxsec.com via core.rxsec.com X-Antivirus-RXSEC: 1.25-st-qms (Clear:RC:0(10.1.0.239):SA:0(-4.4/5.0):. Processed in 6.064556 secs Process 93146) Received: from unknown (HELO ?10.1.0.239?) (cmarlatt@rxsec.com@10.1.0.239) by core.rxsec.com with SMTP; 6 Jul 2008 03:02:30 -0000 Message-ID: <48703855.2040503@rxsec.com> Date: Sat, 05 Jul 2008 23:13:25 -0400 From: Chris Marlatt Organization: Receive Security User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Bart Van Kerckhove , FreeBSD Net References: <4867420D.7090406@gtcomm.net> <48699960.9070100@gtcomm.net><20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> In-Reply-To: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 03:40:33 -0000 Bart Van Kerckhove wrote: > The netfpga site seems more or less dead - is this project still alive? > It does look like a very interesting idea, even though it's currently quite > linux-centric (and according to docs doesn't have VLAN nor ip6 support, the > former being quite a dealbreaker) > Just last Thursday they made another release so it certainly doesn't look dead. I've been following the project for awhile now to see where it's going to go. The lack of FreeBSD support isn't great but I doubt it's going to happen until someone steps up and makes it so. The same is likely true for VLAN support. So far it's primarily been a proof of concept from what I can tell and could be molded into any number of different applications with the appropriate support. Considering all high performance routing platforms separate the management and routing/switching into two (or more) different hardware sections it wouldn't surprise me at all to see this as the only real option to get some serious routing and firewalling performance out of i386/amd64 type servers. Throwing faster and faster cpus at it is only going to get you so far (re: opteron 2212 vs 2222). Even so, 1.1Mpps is a considerable rate. Regards, Chris From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 04:46:14 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ABC82106567B for ; Sun, 6 Jul 2008 04:46:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outY.internet-mail-service.net (outy.internet-mail-service.net [216.240.47.248]) by mx1.freebsd.org (Postfix) with ESMTP id 81EFE8FC0A for ; Sun, 6 Jul 2008 04:46:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 128CE2435; Sat, 5 Jul 2008 21:46:14 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id A86BB2D600E; Sat, 5 Jul 2008 21:46:13 -0700 (PDT) Message-ID: <48704E15.1070803@elischer.org> Date: Sat, 05 Jul 2008 21:46:13 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Bart Van Kerckhove References: <4867420D.7090406@gtcomm.net> <48699960.9070100@gtcomm.net><20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> In-Reply-To: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 04:46:14 -0000 Bart Van Kerckhove wrote: > > >> Perhaps you have some better luck at some different hardware systems >> (ppc, mips, ..?) or use freebsd only for routing-table-updates and >> special network-cards (netfpga) for real routing. > The netfpga site seems more or less dead - is this project still alive? > It does look like a very interesting idea, even though it's currently quite > linux-centric (and according to docs doesn't have VLAN nor ip6 support, the > former being quite a dealbreaker) netfpga is very much alive. I'm on the mailing lists.. but it is summer break and it's an academically driven project. From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 08:13:49 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4811D1065674 for ; Sun, 6 Jul 2008 08:13:49 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (host-122-100-2-232.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id EAAB88FC1D for ; Sun, 6 Jul 2008 08:13:48 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 212AA17336; Sun, 6 Jul 2008 18:13:46 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.20.30.101] (60.218.233.220.exetel.com.au [220.233.218.60]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 0F1C717259 for ; Sun, 6 Jul 2008 18:13:42 +1000 (EST) Message-ID: <48707EA6.2020506@modulus.org> Date: Sun, 06 Jul 2008 18:13:26 +1000 From: Andrew Snow User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <48704E15.1070803@elischer.org> In-Reply-To: <48704E15.1070803@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 08:13:49 -0000 I'm no expert, but I imagine the problem is because the net processing of FreeBSD is not pipelined enough. We are now able to affordably throw many gigabytes of RAM into a machine, as well 2 to 8 CPUs. So why not allow for big buffers and multiple processing steps? I be happy to give up a bit of latency in order to increase the parallel processing ability of packets travelling through the system. I could be wrong but I imagine it would be better to treat the processing of pockets as a series of stages with queues (that can grow quite large if necessary). From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 09:10:01 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C4F38106567A for ; Sun, 6 Jul 2008 09:10:01 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.245]) by mx1.freebsd.org (Postfix) with ESMTP id 708F18FC0C for ; Sun, 6 Jul 2008 09:10:01 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by an-out-0708.google.com with SMTP id b33so413201ana.13 for ; Sun, 06 Jul 2008 02:10:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=J7avj1ZK4cpOD2W0/oLKhwEnHVvxZOmGKf0np42CvNA=; b=U7YAdqFyqSb1ivD5dUyb0WvPMHcIgrUhwvppvsgpilzl6pP8Zg5OFaVaF3SIZcUCum 2JZDVbvtBpiyhZ74iaiY83zcF8Iy1kyVCDXpbwS0awg+hzKeS5MAnctRvsK7rQQDWSvS hUzu1YzrXHqee7RXom0E0sUVmSZuUMPTKqrMo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=uuXgK3AadMVwkgERMkQLqTYcSv7n94yfpZWnwPs3yoLM7hwAB8+heOcgaAhg82E0HM b+REw1O7hRiANqrzveBOnTWm42K5cV4YmpDufbqJ2X2w7y3/6jqBxywt1wiXT0iz9Dik s7cAnl/YttbjpNvjuYHSwd772jjgNm+ljcACA= Received: by 10.100.139.20 with SMTP id m20mr1364121and.77.1215335400305; Sun, 06 Jul 2008 02:10:00 -0700 (PDT) Received: by 10.100.107.6 with HTTP; Sun, 6 Jul 2008 02:10:00 -0700 (PDT) Message-ID: Date: Sun, 6 Jul 2008 17:10:00 +0800 From: "Sepherosa Ziehau" To: Paul In-Reply-To: <486E65E6.3060301@gtcomm.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> Cc: FreeBSD Net , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 09:10:01 -0000 On Sat, Jul 5, 2008 at 2:03 AM, Paul wrote: > I tried all of this :/ still, 256/512 descriptors seem to work the best. > Happy to let you log into the machine and fiddle around if you want :) I think you need to ktr the packet processing time. Standard gigabit max at ~1488095pps, which means you could be @max rate only if each packet's processing time is <= ~672ns. Since you are using fastforwarding, the calculation should be quite straightforward for you; I don't think you should expect anything beyond the calculated result. Best Regards, sephe -- Live Free or Die From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 09:14:37 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FFC31065677; Sun, 6 Jul 2008 09:14:37 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 109FA8FC16; Sun, 6 Jul 2008 09:14:37 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m669EaoR068119; Sun, 6 Jul 2008 09:14:36 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m669EadM068115; Sun, 6 Jul 2008 09:14:36 GMT (envelope-from linimon) Date: Sun, 6 Jul 2008 09:14:36 GMT Message-Id: <200807060914.m669EadM068115@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/123200: [netgraph] Server failure due to netgraph mpd and dhcpclient X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 09:14:37 -0000 Old Synopsis: Server failure due to netgraph mpd and dhcpclient New Synopsis: [netgraph] Server failure due to netgraph mpd and dhcpclient Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jul 6 09:09:28 UTC 2008 Responsible-Changed-Why: Over to maintainer(s). Note that some feedback has been received. http://www.freebsd.org/cgi/query-pr.cgi?pr=123200 From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 09:16:47 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D6671065674; Sun, 6 Jul 2008 09:16:47 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D2A838FC0C; Sun, 6 Jul 2008 09:16:46 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m669Gksr068327; Sun, 6 Jul 2008 09:16:46 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m669GkdS068323; Sun, 6 Jul 2008 09:16:46 GMT (envelope-from linimon) Date: Sun, 6 Jul 2008 09:16:46 GMT Message-Id: <200807060916.m669GkdS068323@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/122685: It is not visible passing packets in tcpdump X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 09:16:47 -0000 Synopsis: It is not visible passing packets in tcpdump Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jul 6 09:16:38 UTC 2008 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=122685 From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 09:18:28 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 106951065679; Sun, 6 Jul 2008 09:18:28 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D58738FC23; Sun, 6 Jul 2008 09:18:27 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m669IRC1068467; Sun, 6 Jul 2008 09:18:27 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m669IRom068463; Sun, 6 Jul 2008 09:18:27 GMT (envelope-from linimon) Date: Sun, 6 Jul 2008 09:18:27 GMT Message-Id: <200807060918.m669IRom068463@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-net@FreeBSD.org, sam@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/122145: [build] error while compiling with device ath_rate_amrr X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 09:18:28 -0000 Old Synopsis: error while compiling with device ath_rate_amrr New Synopsis: [build] error while compiling with device ath_rate_amrr Responsible-Changed-From-To: freebsd-net->sam Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jul 6 09:17:46 UTC 2008 Responsible-Changed-Why: Over to committer for MFC reminder to 6, if desired. (Otherwise, just set to 'suspended') http://www.freebsd.org/cgi/query-pr.cgi?pr=122145 From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 12:45:53 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9FDE1065672 for ; Sun, 6 Jul 2008 12:45:53 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 6F8238FC18 for ; Sun, 6 Jul 2008 12:45:53 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id E497646BAC; Sun, 6 Jul 2008 08:26:54 -0400 (EDT) Date: Sun, 6 Jul 2008 13:26:54 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Paul In-Reply-To: <486FFF70.3090402@gtcomm.net> Message-ID: <20080706132148.E44832@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2008 12:45:53 -0000 On Sat, 5 Jul 2008, Paul wrote: > ULE + PREEMPTION for non SMP no major differences with SMP with ULE/4BSD and > preemption ON/OFF > > 32 bit UP test coming up with new cpu and I'm installing dragonfly sometime > this weekend :] UP: 1mpps in one direction with no firewall/no routing table > is not too bad, but 1mpps both directions is the goal here 700kpps with full > bgp table in one direction is not too bad Ipfw needs a lot of work, barely > gets 500kpps with no routing table with a few ipfw rules loaded.. that's > horrible Linux barely takes a hit when you start loading iptables rules , > but then again linux has a HUGE problem with routing random packet > sources/ports .. grr My problem Is I need some box to do fast routing and > some to do firewall.. :/ I'll have 32 bit 7-stable UP test with ipfw/routing > table and then move on to dragonfly. I'll post the dragonfly results here as > well as sign up for their mailing list. First off, I would recommend using an 8-CURRENT kernel where possible (obviously, with all debugging features disabled), because that's where most of the work is going on right now. MFCs are scheduled for quite a bit of it, but over the course of several months, so using the 8-CURRENT kernel would allow you to help us test and exercise the new code, as well as improve our confidence in it so that it can be MFC'd in a timely manner :-). Experience suggests that forwarding workloads see significant lock contention in the routing and transmit queue code. The former needs some kernel hacking to address in order to improve parallelism for routing lookups. The latter is harder to address given the hardware you're using: modern 10gbps cards frequently offer multiple transmit queues that can be used independently (which our cxgb driver supports), but 1gbps cards generally don't. LOCK_PROFILING is an excellent tool for diagnosing locking hot spots -- it has a significant performance hit, but the results are generally accurate despite this. If your hardware supports hwpmc, that is also an excellent tool for monitoring what's going on. Seeing snapshots of, say, 10-20 seconds of profiling in the steady state, would help us understand better what is going on in your environment. There's some quite interesting work going on to improve network memory allocator efficiency, but that's a bit aways from commit to 8.x as I understand it, and probably not on the 7.x merge path due to the potential disruption it could cause. There's also a patch going around to offload the em_start function to the task queue that processes its input, which significantly reduces lock contention if you have bursty transmit. I'll see if I can't dig it up. Robert N M Watson Computer Laboratory University of Cambridge > > > Bart Van Kerckhove wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Paul / Ingo, >> >>>> I tried all of this :/ still, 256/512 descriptors seem to work the >>>> best. Happy to let you log into the machine and fiddle around if you >>>> want :) >> I've been watching this thread closely, since I'm in a very similair >> situation. >> A few questions/remarks: >> >> Does ULE provide better performance than 4BSD for forwarding? >> Did you try freebsd4 as well? This thread had a report about that quite >> opposite to my own experiences, -4 seemed to be a lot faster at forwarding >> than anything else I 've tried so far. >> Obviously the thing I'm interested in is IMIX - and 64byte packets. >> Does anyone have any benchmarks for DragonFly? I asked around on IRC, but >> that nor google turned up any useful results. >> >> >>> I don't think you will be able to route 64byte packets at 1gbit >>> wirespeed (2Mpps) with a current x86 platform. >>> >> Are there actual hardware related reasons this should not be possible, or >> is this purely lack of dedicated work towards this goal? >> >> >> >>> Theres a "sun" used at quagga dev as bgp-route-server. >>> http://quagga.net/route-server.php >>> (but they don't answered my question regarding fw-performance). >>> >> >> >> the Quagga guys are running a sun T1000 (niagara 1) route server - I happen >> to have the machine in my racks, >> please let me know if you want to run some tests on it, I'm sure they won't >> mind ;-) >> It should also make a great testbed for SMP performance testing imho (and >> they're pretty cheap these days) >> Also, feel free to use me as a relay for your questions, they're not always >> very reachable. >> >> >> >>> Perhaps you have some better luck at some different hardware systems >>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and >>> special network-cards (netfpga) for real routing. >>> >> The netfpga site seems more or less dead - is this project still alive? >> It does look like a very interesting idea, even though it's currently quite >> linux-centric (and according to docs doesn't have VLAN nor ip6 support, the >> former being quite a dealbreaker) >> >> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a >> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of >> information you are providing us :) >> >> Met vriendelijke groet / With kind regards, >> >> Bart Van Kerckhove >> http://friet.net/pgp.txt >> >> -----BEGIN PGP SIGNATURE----- >> >> iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi >> eca31f7WQ/oXq9tJ8TEDN3CA >> =YGYq >> -----END PGP SIGNATURE----- >> >> >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 08:47:25 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B14521065687 for ; Mon, 7 Jul 2008 08:47:25 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id C97188FC15 for ; Mon, 7 Jul 2008 08:47:24 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 29125 invoked from network); 7 Jul 2008 07:37:53 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 07:37:53 -0000 Message-ID: <4871D81B.8070507@freebsd.org> Date: Mon, 07 Jul 2008 10:47:23 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Robert Watson References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <20080706132148.E44832@fledge.watson.org> In-Reply-To: <20080706132148.E44832@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 08:47:25 -0000 Robert Watson wrote: > Experience suggests that forwarding workloads see significant lock > contention in the routing and transmit queue code. The former needs > some kernel hacking to address in order to improve parallelism for > routing lookups. The latter is harder to address given the hardware > you're using: modern 10gbps cards frequently offer multiple transmit > queues that can be used independently (which our cxgb driver supports), > but 1gbps cards generally don't. Actually the routing code is not contended. The workload in router is mostly serialized without much opportunity for contention. With many interfaces and any-to-any traffic patterns it may get some contention. The locking overhead per packet is always there and has some impact though. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 08:54:09 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 763EA106564A; Mon, 7 Jul 2008 08:54:09 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3D4578FC26; Mon, 7 Jul 2008 08:54:09 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id C27B046C6C; Mon, 7 Jul 2008 04:54:08 -0400 (EDT) Date: Mon, 7 Jul 2008 09:54:08 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Andre Oppermann In-Reply-To: <4871D81B.8070507@freebsd.org> Message-ID: <20080707095013.N63144@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <20080706132148.E44832@fledge.watson.org> <4871D81B.8070507@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 08:54:09 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Robert Watson wrote: >> Experience suggests that forwarding workloads see significant lock >> contention in the routing and transmit queue code. The former needs some >> kernel hacking to address in order to improve parallelism for routing >> lookups. The latter is harder to address given the hardware you're using: >> modern 10gbps cards frequently offer multiple transmit queues that can be >> used independently (which our cxgb driver supports), but 1gbps cards >> generally don't. > > Actually the routing code is not contended. The workload in router is > mostly serialized without much opportunity for contention. With many > interfaces and any-to-any traffic patterns it may get some contention. The > locking overhead per packet is always there and has some impact though. Yes, I don't see any real sources of contention until we reach the output code, which will run in the input if_em taskqueue threads, as the input path generates little or no contention of the packets are not destined for local delivery. I was a little concerned about mention of degrading performance as firewall complexity grows -- I suspect there's a nice project for someone to do looking at why this is the case. I was under the impression that, in 7.x and later, we use rwlocks to protect firewall state, and that unless stateful firewall rules are used, these are locked read-only rather than writable... Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 09:02:08 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E8B91065674 for ; Mon, 7 Jul 2008 09:02:08 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 6535B8FC26 for ; Mon, 7 Jul 2008 09:02:07 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 29270 invoked from network); 7 Jul 2008 07:52:36 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 07:52:36 -0000 Message-ID: <4871DB8E.5070903@freebsd.org> Date: Mon, 07 Jul 2008 11:02:06 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Ingo Flaschberger References: <4867420D.7090406@gtcomm.net> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 09:02:08 -0000 Ingo Flaschberger wrote: > Dear Paul, > >> I tried all of this :/ still, 256/512 descriptors seem to work the best. >> Happy to let you log into the machine and fiddle around if you want :) > > yes, but I'm shure I will also not be able to achieve much more pps. > As it seems that you hit hardware-software-level-barriers, my only idea > is to test dragonfly bsd, which seems to have less software overhead. I tested DragonFly some time ago with an Agilent N2X tester and it was by far the slowest of the pack. > I don't think you will be able to route 64byte packets at 1gbit > wirespeed (2Mpps) with a current x86 platform. You have to take inter-frame gap and other overheads too. That gives about 1.244Mpps max on a 1GigE interface. In general the chipsets and buses are able to transfer quite a bit of data. On a dual-opteron 848 I was able to sink 2.5Mpps into the machine with "ifconfig em[01] monitor" without hitting the cpu ceiling. This means that the bus and interrupt handling is not where most of the time is spent. When I did my profiling the saturation point was the cache miss penalty for accessing the packet headers. At saturation point about 50% of the time was spent waiting for the memory to make its way into the CPU. > I hoped to reach 1Mpps with the hardware I mentioned some mails before, > but 2Mpps is far far away. > Currently I get 160kpps via pci-32mbit-33mhz-1,2ghz mobile pentium. This is more or less expected. PCI32 is not able to sustain high packet rates. The bus setup times kill the speed. For larger packets the ratio gets much better and some reasonable throughput can be achieved. > Perhaps you have some better luck at some different hardware systems > (ppc, mips, ..?) or use freebsd only for routing-table-updates and > special network-cards (netfpga) for real routing. NetFPGA doesn't have enough TCAM space to be useful for real routing (as in Internet sized routing table). The trick many embedded networking CPUs use is cache prefetching that is integrated with the network controller. The first 64-128bytes of every packet are transferred automatically into the L2 cache by the hardware. This allows relatively slow CPUs (700 MHz Broadcom BCM1250 in Cisco NPE-G1 or 1.67-GHz Freescale 7448 in NPE-G2) to get more than 1Mpps. Until something like this is possible on Intel or AMD x86 CPUs we have a ceiling limited by RAM speed. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 09:11:39 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 874D81065684 for ; Mon, 7 Jul 2008 09:11:39 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id E0DB78FC1B for ; Mon, 7 Jul 2008 09:11:38 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 29551 invoked from network); 7 Jul 2008 08:02:07 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 08:02:07 -0000 Message-ID: <4871DDC9.6060706@freebsd.org> Date: Mon, 07 Jul 2008 11:11:37 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Robert Watson References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <20080706132148.E44832@fledge.watson.org> <4871D81B.8070507@freebsd.org> <20080707095013.N63144@fledge.watson.org> In-Reply-To: <20080707095013.N63144@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 09:11:39 -0000 Robert Watson wrote: > > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Robert Watson wrote: >>> Experience suggests that forwarding workloads see significant lock >>> contention in the routing and transmit queue code. The former needs >>> some kernel hacking to address in order to improve parallelism for >>> routing lookups. The latter is harder to address given the hardware >>> you're using: modern 10gbps cards frequently offer multiple transmit >>> queues that can be used independently (which our cxgb driver >>> supports), but 1gbps cards generally don't. >> >> Actually the routing code is not contended. The workload in router is >> mostly serialized without much opportunity for contention. With many >> interfaces and any-to-any traffic patterns it may get some >> contention. The locking overhead per packet is always there and has >> some impact though. > > Yes, I don't see any real sources of contention until we reach the > output code, which will run in the input if_em taskqueue threads, as the > input path generates little or no contention of the packets are not > destined for local delivery. I was a little concerned about mention of The interface output was the second largest block after the cache misses IIRC. The output part seems to have received only moderate attention and detailed performance analysis compared to the interface input path. Most network drivers do a write to the hardware for every packet sent in addition to other overhead that may be necessary for their transmit DMA rings. That adds significant overhead compared to the RX path where those costs are amortized over a larger number packets. > degrading performance as firewall complexity grows -- I suspect there's > a nice project for someone to do looking at why this is the case. I was > under the impression that, in 7.x and later, we use rwlocks to protect > firewall state, and that unless stateful firewall rules are used, these > are locked read-only rather than writable... The overhead of just looking at the packet (twice) in ipfw or other firewall packets is a huge overhead. The main loop of ipfw is a very large block of code. Unless one implements compilation of firewall to native machine code there is not much that can be done. With LLVM we will see some very interesting opportunity in that area. Other than that the ipfw instruction over per rule seems to be quite close to the optimum. I'm not saying one shouldn't take a close look with a profiler to verify this is actually the case. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 09:47:06 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B54401065679 for ; Mon, 7 Jul 2008 09:47:06 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0CE598FC15 for ; Mon, 7 Jul 2008 09:47:05 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 30592 invoked from network); 7 Jul 2008 08:37:34 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 08:37:34 -0000 Message-ID: <4871E618.1080500@freebsd.org> Date: Mon, 07 Jul 2008 11:47:04 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Paul References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> In-Reply-To: <48701921.7090107@gtcomm.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 09:47:06 -0000 Paul, to get a systematic analysis of the performance please do the following tests and put them into a table for easy comparison: 1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 monitor) 2. inbound pps w/ fastforward into a single blackhole route 3. inbound pps /w fastforward into a single blackhole route w/ ipfw and just one allow all rule 4. inbound pps /w fastforward into a single blackhole route w/ ipfw and just one deny all rule 5. inbound pps /w fastforward into the disc(4) discard network interface 6. inbound pps /w fastforward into the disc(4) discard network interface w/ ipfw and just one allow all rule All surrounding parameters like RX/TX interface queue length, scheduler and so may me varied but should be noted. -- Andre Paul wrote: > UP 32 bit test vs 64 bit: > negligible difference in forwarding performance without polling > slightly better polling performance but still errors at lower packet rates > same massive hit with ipfw loaded > > Installing dragonfly in a bit.. > If anyone has a really fast PPC type system or SUN or something i'd love > to try it :) > Something with a really big L1 cache :P > > > Paul wrote: >> ULE + PREEMPTION for non SMP >> no major differences with SMP with ULE/4BSD and preemption ON/OFF >> >> 32 bit UP test coming up with new cpu >> and I'm installing dragonfly sometime this weekend :] >> UP: 1mpps in one direction with no firewall/no routing table is not >> too bad, but 1mpps both directions is the goal here >> 700kpps with full bgp table in one direction is not too bad >> Ipfw needs a lot of work, barely gets 500kpps with no routing table >> with a few ipfw rules loaded.. that's horrible >> Linux barely takes a hit when you start loading iptables rules , but >> then again linux has a HUGE problem with routing >> random packet sources/ports .. grr >> My problem Is I need some box to do fast routing and some to do >> firewall.. :/ >> I'll have 32 bit 7-stable UP test with ipfw/routing table and then >> move on to dragonfly. >> I'll post the dragonfly results here as well as sign up for their >> mailing list. >> >> >> Bart Van Kerckhove wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Paul / Ingo, >>> >>>>> I tried all of this :/ still, 256/512 descriptors seem to work the >>>>> best. Happy to let you log into the machine and fiddle around if you >>>>> want :) >>> I've been watching this thread closely, since I'm in a very similair >>> situation. >>> A few questions/remarks: >>> >>> Does ULE provide better performance than 4BSD for forwarding? >>> Did you try freebsd4 as well? This thread had a report about that quite >>> opposite to my own experiences, -4 seemed to be a lot faster at >>> forwarding >>> than anything else I 've tried so far. >>> Obviously the thing I'm interested in is IMIX - and 64byte packets. >>> Does anyone have any benchmarks for DragonFly? I asked around on IRC, >>> but >>> that nor google turned up any useful results. >>> >>> >>>> I don't think you will be able to route 64byte packets at 1gbit >>>> wirespeed (2Mpps) with a current x86 platform. >>>> >>> Are there actual hardware related reasons this should not be >>> possible, or >>> is this purely lack of dedicated work towards this goal? >>> >>> >>> >>>> Theres a "sun" used at quagga dev as bgp-route-server. >>>> http://quagga.net/route-server.php >>>> (but they don't answered my question regarding fw-performance). >>>> >>> >>> >>> the Quagga guys are running a sun T1000 (niagara 1) route server - I >>> happen >>> to have the machine in my racks, >>> please let me know if you want to run some tests on it, I'm sure they >>> won't >>> mind ;-) >>> It should also make a great testbed for SMP performance testing imho >>> (and >>> they're pretty cheap these days) >>> Also, feel free to use me as a relay for your questions, they're not >>> always >>> very reachable. >>> >>> >>> >>>> Perhaps you have some better luck at some different hardware systems >>>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and >>>> special network-cards (netfpga) for real routing. >>>> >>> The netfpga site seems more or less dead - is this project still alive? >>> It does look like a very interesting idea, even though it's currently >>> quite >>> linux-centric (and according to docs doesn't have VLAN nor ip6 >>> support, the >>> former being quite a dealbreaker) >>> >>> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a >>> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of >>> information you are providing us :) >>> >>> Met vriendelijke groet / With kind regards, >>> >>> Bart Van Kerckhove >>> http://friet.net/pgp.txt >>> >>> -----BEGIN PGP SIGNATURE----- >>> >>> iQA/AwUBSG/tMgoIFchBM0BKEQKUSQCcCJqsw2wtUX7HQi050HEDYX3WPuMAnjmi >>> eca31f7WQ/oXq9tJ8TEDN3CA >>> =YGYq >>> -----END PGP SIGNATURE----- >>> >>> >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 09:56:46 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1D971065676 for ; Mon, 7 Jul 2008 09:56:46 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 41CF58FC22 for ; Mon, 7 Jul 2008 09:56:46 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 30719 invoked from network); 7 Jul 2008 08:47:14 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 08:47:14 -0000 Message-ID: <4871E85C.8090907@freebsd.org> Date: Mon, 07 Jul 2008 11:56:44 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Paul References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> In-Reply-To: <486B41D5.3060609@gtcomm.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 09:56:46 -0000 Paul wrote: > SMP DISABLED on my Opteron 2212 (ULE, Preemption on) > Yields ~750kpps in em0 and out em1 (one direction) > I am miffed why this yields more pps than > a) with all 4 cpus running and b) 4 cpus with lagg load balanced over 3 > incoming connections so 3 taskq threads SMP adds quite some overhead in the generic case is currently not well suited for high performance packet forwarding. On SMP interrupts are delivered to one CPU but not necessarily the one that will later on handle the taskqueue to process the packets. That adds overhead. Ideally the interrupt for each network interface is bound to exactly one pre-determined CPU and the taskqueue is bound to the same CPU. That way the overhead for interrupt and taskqueue scheduling can be kept at a minimum. Most of the infrastructure to do this binding already exists in the kernel but is not yet exposed to the outside for us to make use of it. I'm also not sure if the ULE scheduler skips the more global locks when interrupt and the thread are on the same CPU. Distributing the interrupts and taskqueues among the available CPUs gives concurrent forwarding with bi- or multi-directional traffic. All incoming traffic from any particular interface is still serialized though. -- Andre > I would be willing to set up test equipment (several servers plugged > into a switch) with ipkvm and power port access > if someone or a group of people want to figure out ways to improve the > routing process, ipfw, and lagg. > > Maximum PPS with one ipfw rule on UP: > tops out about 570Kpps.. almost 200kpps lower ? (frown) > > I'm going to drop in a 3ghz opteron instead of the 2ghz 2212 that's in > here and see how that scales, using UP same kernel etc I have now. > > > > > > Julian Elischer wrote: >> Paul wrote: >>> ULE without PREEMPTION is now yeilding better results. >>> input (em0) output >>> packets errs bytes packets errs bytes colls >>> 571595 40639 34564108 1 0 226 0 >>> 577892 48865 34941908 1 0 178 0 >>> 545240 84744 32966404 1 0 178 0 >>> 587661 44691 35534512 1 0 178 0 >>> 587839 38073 35544904 1 0 178 0 >>> 587787 43556 35540360 1 0 178 0 >>> 540786 39492 32712746 1 0 178 0 >>> 572071 55797 34595650 1 0 178 0 >>> >>> *OUCH, IPFW HURTS.. >>> loading ipfw, and adding one ipfw rule allow ip from any to any drops >>> 100Kpps off :/ what's up with THAT? >>> unloaded ipfw module and back 100kpps more again, that's not right >>> with ONE rule.. :/ >> >> ipfw need sto gain a lock on hte firewall before running, >> and is quite complex.. I can believe it.. >> >> in FreeBSD 4.8 I was able to use ipfw and filter 1Gb between two >> interfaces (bridged) but I think it has slowed down since then due to >> the SMP locking. >> >> >>> >>> em0 taskq is still jumping cpus.. is there any way to lock it to one >>> cpu or is this just a function of ULE >>> >>> running a tar czpvf all.tgz * and seeing if pps changes.. >>> negligible.. guess scheduler is doing it's job at least.. >>> >>> Hmm. even when it's getting 50-60k errors per second on the interface >>> I can still SCP a file through that interface although it's not >>> fast.. 3-4MB/s.. >>> >>> You know, I wouldn't care if it added 5ms latency to the packets when >>> it was doing 1mpps as long as it didn't drop any.. Why can't it do >>> that? Queue them up and do them in bigggg chunks so none are >>> dropped........hmm? >>> >>> 32 bit system is compiling now.. won't do > 400kpps with GENERIC >>> kernel, as with 64 bit did 450k with GENERIC, although that could be >>> the difference between opteron 270 and opteron 2212.. >>> >>> Paul >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 10:00:22 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84F2D106567E for ; Mon, 7 Jul 2008 10:00:22 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id 141748FC13 for ; Mon, 7 Jul 2008 10:00:21 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67A0GaM025391 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2008 20:00:18 +1000 Date: Mon, 7 Jul 2008 20:00:16 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Andre Oppermann In-Reply-To: <4871DB8E.5070903@freebsd.org> Message-ID: <20080707191918.B4703@besplex.bde.org> References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 10:00:22 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Ingo Flaschberger wrote: >> I don't think you will be able to route 64byte packets at 1gbit wirespeed >> (2Mpps) with a current x86 platform. > > You have to take inter-frame gap and other overheads too. That gives > about 1.244Mpps max on a 1GigE interface. What are the other overheads? I calculate 1.644Mpps counting the inter-frame gap, with 64-byte packets and 64-header_size payloads. If the 64 bytes is for the payload, then the max is much lower. >> I hoped to reach 1Mpps with the hardware I mentioned some mails before, but >> 2Mpps is far far away. >> Currently I get 160kpps via pci-32mbit-33mhz-1,2ghz mobile pentium. > > This is more or less expected. PCI32 is not able to sustain high > packet rates. The bus setup times kill the speed. For larger packets > the ratio gets much better and some reasonable throughput can be achieved. I get about 640 kpps without forwarding (sendto: slightly faster; recvfrom: slightly slower) on a 2.2GHz A64. Underclocking the memory from 200MHz to 100MHz only reduces the speed by about 10%, while not overclocking the CPU by 10% reduces the speed by the same 10%, so the system is apparently still mainly CPU-bound. > NetFPGA doesn't have enough TCAM space to be useful for real routing > (as in Internet sized routing table). The trick many embedded networking > CPUs use is cache prefetching that is integrated with the network > controller. The first 64-128bytes of every packet are transferred > automatically into the L2 cache by the hardware. This allows relatively > slow CPUs (700 MHz Broadcom BCM1250 in Cisco NPE-G1 or 1.67-GHz Freescale > 7448 in NPE-G2) to get more than 1Mpps. Until something like this is > possible on Intel or AMD x86 CPUs we have a ceiling limited by RAM speed. Does using fa$ter memory (speed and/or latency) help here? 64 bytes is so small that latency may be more of a problem, especially without a prefetch. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 10:48:39 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F40FB1065679; Mon, 7 Jul 2008 10:48:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9328D8FC1D; Mon, 7 Jul 2008 10:48:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id AC6E446B06; Mon, 7 Jul 2008 06:48:37 -0400 (EDT) Date: Mon, 7 Jul 2008 11:48:37 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Andre Oppermann In-Reply-To: <4871E85C.8090907@freebsd.org> Message-ID: <20080707114538.K63144@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 10:48:39 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Distributing the interrupts and taskqueues among the available CPUs gives > concurrent forwarding with bi- or multi-directional traffic. All incoming > traffic from any particular interface is still serialized though. ... although not on multiple input queue-enabled hardware and drivers. While I've really only focused on local traffic performance with my 10gbps Chelsio setup, it should be possible to do packet forwarding from multiple input queues using that hardware and driver today. I'll update the netisr2 patches, which allow work to be pushed to multiple CPUs from a single input queue. However, these necessarily take a cache miss or two on packet header data in order to break out the packets from the input queue into flows that can be processed independently without ordering constraints, so if those cache misses on header data are a big part of the performance of a configuration, load balancing in this manner may not help. What would be neat is if the cards without multiple input queues could still tag receive descriptors with a flow identifier generated from the IP/TCP/etc layers that could be used for work placement. Robert N M Watson Computer Laboratory University of Cambridge > > -- > Andre > >> I would be willing to set up test equipment (several servers plugged into a >> switch) with ipkvm and power port access >> if someone or a group of people want to figure out ways to improve the >> routing process, ipfw, and lagg. >> >> Maximum PPS with one ipfw rule on UP: >> tops out about 570Kpps.. almost 200kpps lower ? (frown) >> >> I'm going to drop in a 3ghz opteron instead of the 2ghz 2212 that's in here >> and see how that scales, using UP same kernel etc I have now. >> >> >> >> >> >> Julian Elischer wrote: >>> Paul wrote: >>>> ULE without PREEMPTION is now yeilding better results. >>>> input (em0) output >>>> packets errs bytes packets errs bytes colls >>>> 571595 40639 34564108 1 0 226 0 >>>> 577892 48865 34941908 1 0 178 0 >>>> 545240 84744 32966404 1 0 178 0 >>>> 587661 44691 35534512 1 0 178 0 >>>> 587839 38073 35544904 1 0 178 0 >>>> 587787 43556 35540360 1 0 178 0 >>>> 540786 39492 32712746 1 0 178 0 >>>> 572071 55797 34595650 1 0 178 0 >>>> *OUCH, IPFW HURTS.. >>>> loading ipfw, and adding one ipfw rule allow ip from any to any drops >>>> 100Kpps off :/ what's up with THAT? >>>> unloaded ipfw module and back 100kpps more again, that's not right with >>>> ONE rule.. :/ >>> >>> ipfw need sto gain a lock on hte firewall before running, >>> and is quite complex.. I can believe it.. >>> >>> in FreeBSD 4.8 I was able to use ipfw and filter 1Gb between two >>> interfaces (bridged) but I think it has slowed down since then due to the >>> SMP locking. >>> >>> >>>> >>>> em0 taskq is still jumping cpus.. is there any way to lock it to one cpu >>>> or is this just a function of ULE >>>> >>>> running a tar czpvf all.tgz * and seeing if pps changes.. >>>> negligible.. guess scheduler is doing it's job at least.. >>>> >>>> Hmm. even when it's getting 50-60k errors per second on the interface I >>>> can still SCP a file through that interface although it's not fast.. >>>> 3-4MB/s.. >>>> >>>> You know, I wouldn't care if it added 5ms latency to the packets when it >>>> was doing 1mpps as long as it didn't drop any.. Why can't it do that? >>>> Queue them up and do them in bigggg chunks so none are >>>> dropped........hmm? >>>> >>>> 32 bit system is compiling now.. won't do > 400kpps with GENERIC kernel, >>>> as with 64 bit did 450k with GENERIC, although that could be >>>> the difference between opteron 270 and opteron 2212.. >>>> >>>> Paul >>>> >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:07:03 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E2BD10656D2 for ; Mon, 7 Jul 2008 11:07:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3EDD68FC2F for ; Mon, 7 Jul 2008 11:07:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m67B73hN062116 for ; Mon, 7 Jul 2008 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m67B72MO062112 for freebsd-net@FreeBSD.org; Mon, 7 Jul 2008 11:07:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 7 Jul 2008 11:07:02 GMT Message-Id: <200807071107.m67B72MO062112@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 11:07:03 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c o kern/35442 net [sis] [patch] Problem transmitting runts in if_sis dri a kern/38554 net changing interface ipaddress doesn't seem to work s kern/39937 net ipstealth issue s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/79895 net [ipf] 5.4-RC2 breaks ipfilter NAT when using netgraph s kern/81147 net [net] [patch] em0 reinitialization while adding aliase o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/92090 net [bge] bge0: watchdog timeout -- resetting f kern/92552 net A serious bug in most network drivers from 5.X to 6.X o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau f kern/102344 net [ipf] Some packets do not pass through network interfa o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] s kern/105943 net Network stack may modify read-only mbuf chain copies o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/108542 net [bce]: Huge network latencies with 6.2-RELEASE / STABL o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/109308 net [pppd] [panic] Multiple panics kernel ppp suspected [r o kern/109733 net [bge] bge link state issues [regression] o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o kern/112722 net [udp] IP v4 udp fragmented packet reject o kern/113842 net [ip6] PF_INET6 proto domain state can't be cleared wit o kern/114714 net [gre][patch] gre(4) is not MPSAFE and does not support o kern/114839 net [fxp] fxp looses ability to speak with traffic o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/116077 net [ip] [patch] 6.2-STABLE panic during use of multi-cast o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/116328 net [bge]: Solid hang with bge interface o kern/116747 net [ndis] FreeBSD 7.0-CURRENT crash with Dell TrueMobile o kern/116837 net [tun] [panic] [patch] ifconfig tunX destroy: panic o kern/117043 net [em] Intel PWLA8492MT Dual-Port Network adapter EEPROM o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o kern/117423 net [vlan] Duplicate IP on different interfaces o kern/117448 net [carp] 6.2 kernel crash [regression] o kern/118880 net [ip6] IP_RECVDSTADDR & IP_SENDSRCADDR not implemented o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/119345 net [ath] Unsuported Atheros 5424/2424 and CPU speedstep n o kern/119361 net [bge] bge(4) transmit performance problem o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/120130 net [carp] [panic] carp causes kernel panics in any conste o kern/120266 net [panic] gnugk causes kernel panic when closing UDP soc o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/121080 net [bge] IPv6 NUD problem on multi address config on bge0 o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/121298 net [em] [panic] Fatal trap 12: page fault while in kernel o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121624 net [em] [regression] Intel em WOL fails after upgrade to o kern/121872 net [wpi] driver fails to attach on a fujitsu-siemens s711 o kern/121983 net [fxp] fxp0 MBUF and PAE o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup [reg o kern/122058 net [em] [panic] Panic on em1: taskq o kern/122082 net [in_pcb] NULL pointer dereference in in_pcbdrop o kern/122195 net [ed] Alignment problems in if_ed f kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122427 net [apm] [panic] apm and mDNSResponder cause panic during o kern/122551 net [bge] Broadcom 5715S no carrier on HP BL460c blade usi o kern/122685 net It is not visible passing packets in tcpdump o kern/122743 net [panic] vm_page_unwire: invalid wire count: 0 o kern/122772 net [em] em0 taskq panic, tcp reassembly bug causes radix f kern/122794 net [lagg] Kernel panic after brings lagg(8) up if NICs ar f conf/122858 net [nsswitch.conf] nsswitch in 7.0 is f*cked up o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/123066 net [ipsec] [panic] kernel trap with ipsec o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 f kern/123172 net [bce] Watchdog timeout problems with if_bce f kern/123200 net [netgraph] Server failure due to netgraph mpd and dhcp o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123347 net [bge] bge1: watchdog timeout -- linkstate changed to D o kern/123429 net [nfe] [hang] "ifconfig nfe up" causes a hard system lo o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o kern/123603 net [tcp] tcp_do_segment and Received duplicate SYN o kern/123617 net [tcp] breaking connection when client downloading file o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123881 net [tcp] Turning on TCP blackholing causes slow localhost o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/124127 net [msk] watchdog timeout (missed Tx interrupts) -- recov o kern/124753 net [ieee80211] net80211 discards power-save queue packets o kern/124904 net [fxp] EEPROM corruption with Compaq NC3163 NIC o kern/125079 net [ppp] host routes added by ppp with gateway flag (regr o kern/125195 net [fxp] fxp(4) driver failed to initialize device Intel 94 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o conf/23063 net [PATCH] for static ARP tables in rc.network o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/54383 net [nfs] [patch] NFS root configurations without dynamic s kern/60293 net FreeBSD arp poison patch o kern/64556 net [sis] if_sis short cable fix problems with NetGear FA3 o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 o kern/77913 net [wi] [patch] Add the APDL-325 WLAN pccard to wi(4) o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o bin/79228 net [patch] extend arp(8) to be able to create blackhole r o kern/91594 net [em] FreeBSD > 5.4 w/ACPI fails to detect Intel Pro/10 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/95267 net packet drops periodically appear o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/102035 net [plip] plip networking disables parallel port printing o conf/102502 net [patch] ifconfig name does't rename netgraph node in n o conf/107035 net [patch] bridge interface given in rc.conf not taking a o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o kern/112179 net [sis] [patch] sis driver for natsemi DP83815D autonego o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o bin/117339 net [patch] route(8): loading routing management commands o kern/118727 net [netgraph] [patch] [request] add new ng_pf module a kern/118879 net [bge] [patch] bge has checksum problems on the 5703 ch o bin/118987 net ifconfig(8): ifconfig -l (address_family) does not wor o kern/119432 net [arp] route add -host -iface causes arp e f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/120232 net [nfe] [patch] Bring in nfe(4) to RELENG_6 o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121443 net [gif] LOR icmp6_input/nd6_lookup o kern/121706 net [netinet] [patch] "rtfree: 0xc4383870 has 1 refs" emit s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122068 net [ppp] ppp can not set the correct interface with pptpd o kern/122295 net [bge] bge Ierr rate increase (since 6.0R) [regression] o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122697 net [ath] Atheros card is not well supported o kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge f kern/122839 net [multicast] FreeBSD 7 multicast routing problem o kern/122928 net [em] interface watchdog timeouts and stops receiving p o kern/123892 net [tap] [patch] No buffer space available p kern/123961 net [vr] [patch] Allow vr interface to handle vlans o bin/124004 net ifconfig(8): Cannot assign both an IP and a MAC addres o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124609 net [ipsec] [panic] ipsec 'remainder too big' panic with p o kern/124767 net [iwi] Wireless connection using iwi0 driver (Intel 220 o kern/125003 net [gif] incorrect EtherIP header format. o kern/125239 net [gre] kernel crash when using gre o kern/125258 net [socket] socket's SO_REUSEADDR option does not work 56 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:13:22 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 884101065672 for ; Mon, 7 Jul 2008 11:13:22 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id B170C8FC1F for ; Mon, 7 Jul 2008 11:13:20 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 31286 invoked from network); 7 Jul 2008 10:03:48 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 10:03:48 -0000 Message-ID: <4871FA4F.40206@freebsd.org> Date: Mon, 07 Jul 2008 13:13:19 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Robert Watson References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <20080707114538.K63144@fledge.watson.org> In-Reply-To: <20080707114538.K63144@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 11:13:22 -0000 Robert Watson wrote: > > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Distributing the interrupts and taskqueues among the available CPUs >> gives concurrent forwarding with bi- or multi-directional traffic. All >> incoming traffic from any particular interface is still serialized >> though. > > ... although not on multiple input queue-enabled hardware and drivers. > While I've really only focused on local traffic performance with my > 10gbps Chelsio setup, it should be possible to do packet forwarding from > multiple input queues using that hardware and driver today. > > I'll update the netisr2 patches, which allow work to be pushed to > multiple CPUs from a single input queue. However, these necessarily > take a cache miss or two on packet header data in order to break out the > packets from the input queue into flows that can be processed > independently without ordering constraints, so if those cache misses on > header data are a big part of the performance of a configuration, load > balancing in this manner may not help. What would be neat is if the > cards without multiple input queues could still tag receive descriptors > with a flow identifier generated from the IP/TCP/etc layers that could > be used for work placement. The cache miss is really the elephant in the room. If the network card supports multiple RX rings with separate interrupts and a stable hash based (that includes IP+Port src+dst) distribution they can be bound to different CPUs. It is very important to maintain the packet order for flows that go through the router. Otherwise TCP and VoIP will suffer. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:18:01 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 044C2106567E for ; Mon, 7 Jul 2008 11:18:01 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5A0B58FC0C for ; Mon, 7 Jul 2008 11:18:00 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 31356 invoked from network); 7 Jul 2008 10:08:27 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 10:08:27 -0000 Message-ID: <4871FB66.1060406@freebsd.org> Date: Mon, 07 Jul 2008 13:17:58 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Bruce Evans References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.or g> In-Reply-To: <20080707191918.B4703@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 11:18:01 -0000 Bruce Evans wrote: > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Ingo Flaschberger wrote: >>> I don't think you will be able to route 64byte packets at 1gbit >>> wirespeed (2Mpps) with a current x86 platform. >> >> You have to take inter-frame gap and other overheads too. That gives >> about 1.244Mpps max on a 1GigE interface. > > What are the other overheads? I calculate 1.644Mpps counting the > inter-frame > gap, with 64-byte packets and 64-header_size payloads. If the 64 bytes > is for the payload, then the max is much lower. The theoretical maximum at 64byte frames is 1,488,100. I've looked up my notes the 1.244Mpps number can be ajusted to 1.488Mpps. >>> I hoped to reach 1Mpps with the hardware I mentioned some mails >>> before, but 2Mpps is far far away. >>> Currently I get 160kpps via pci-32mbit-33mhz-1,2ghz mobile pentium. >> >> This is more or less expected. PCI32 is not able to sustain high >> packet rates. The bus setup times kill the speed. For larger packets >> the ratio gets much better and some reasonable throughput can be >> achieved. > > I get about 640 kpps without forwarding (sendto: slightly faster; > recvfrom: slightly slower) on a 2.2GHz A64. Underclocking the memory > from 200MHz to 100MHz only reduces the speed by about 10%, while not > overclocking the CPU by 10% reduces the speed by the same 10%, so the > system is apparently still mainly CPU-bound. On PCI32@33MHz? He's using a 1.2GHz Mobile Pentium on top of that. >> NetFPGA doesn't have enough TCAM space to be useful for real routing >> (as in Internet sized routing table). The trick many embedded networking >> CPUs use is cache prefetching that is integrated with the network >> controller. The first 64-128bytes of every packet are transferred >> automatically into the L2 cache by the hardware. This allows relatively >> slow CPUs (700 MHz Broadcom BCM1250 in Cisco NPE-G1 or 1.67-GHz Freescale >> 7448 in NPE-G2) to get more than 1Mpps. Until something like this is >> possible on Intel or AMD x86 CPUs we have a ceiling limited by RAM speed. > > Does using fa$ter memory (speed and/or latency) help here? 64 bytes > is so small that latency may be more of a problem, especially without > a prefetch. Latency. For IPv4 packet forwarding only one cache line per packet is fetched. More memory speed only helps with the DMA from/to the network card. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:32:01 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73155106567E; Mon, 7 Jul 2008 11:32:01 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8E5198FC18; Mon, 7 Jul 2008 11:31:59 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4871FEAF.1060501@FreeBSD.org> Date: Mon, 07 Jul 2008 13:31:59 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Andre Oppermann References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <20080706132148.E44832@fledge.watson.org> <4871D81B.8070507@freebsd.org> In-Reply-To: <4871D81B.8070507@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Robert Watson , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 11:32:01 -0000 Andre Oppermann wrote: > Robert Watson wrote: >> Experience suggests that forwarding workloads see significant lock >> contention in the routing and transmit queue code. The former needs >> some kernel hacking to address in order to improve parallelism for >> routing lookups. The latter is harder to address given the hardware >> you're using: modern 10gbps cards frequently offer multiple transmit >> queues that can be used independently (which our cxgb driver >> supports), but 1gbps cards generally don't. > > Actually the routing code is not contended. The workload in router > is mostly serialized without much opportunity for contention. With > many interfaces and any-to-any traffic patterns it may get some > contention. The locking overhead per packet is always there and has > some impact though. > Actually contention from route locking is a major bottleneck even on packet generation from multiple CPUs on a single host. It is becoming increasingly necessary that someone look into fixing this. Kris From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 12:31:05 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E98501065678; Mon, 7 Jul 2008 12:31:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail01.syd.optusnet.com.au (mail01.syd.optusnet.com.au [211.29.132.182]) by mx1.freebsd.org (Postfix) with ESMTP id 50CAD8FC16; Mon, 7 Jul 2008 12:31:03 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail01.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67CUt7a010675 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2008 22:30:57 +1000 Date: Mon, 7 Jul 2008 22:30:53 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Andre Oppermann In-Reply-To: <4871FB66.1060406@freebsd.org> Message-ID: <20080707213356.G7572@besplex.bde.org> References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 12:31:05 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Bruce Evans wrote: >> What are the other overheads? I calculate 1.644Mpps counting the >> inter-frame >> gap, with 64-byte packets and 64-header_size payloads. If the 64 bytes >> is for the payload, then the max is much lower. > > The theoretical maximum at 64byte frames is 1,488,100. I've looked > up my notes the 1.244Mpps number can be ajusted to 1.488Mpps. Where is the extra? I still get 1.644736 Mpps (10^9/(8*64+96)). 1.488095 is for 64 bits extra (10^9/(8*64+96+64)). >>>> I hoped to reach 1Mpps with the hardware I mentioned some mails before, >>>> but 2Mpps is far far away. >>>> Currently I get 160kpps via pci-32mbit-33mhz-1,2ghz mobile pentium. >>> >>> This is more or less expected. PCI32 is not able to sustain high >>> packet rates. The bus setup times kill the speed. For larger packets >>> the ratio gets much better and some reasonable throughput can be achieved. >> >> I get about 640 kpps without forwarding (sendto: slightly faster; >> recvfrom: slightly slower) on a 2.2GHz A64. Underclocking the memory >> from 200MHz to 100MHz only reduces the speed by about 10%, while not >> overclocking the CPU by 10% reduces the speed by the same 10%, so the >> system is apparently still mainly CPU-bound. > > On PCI32@33MHz? He's using a 1.2GHz Mobile Pentium on top of that. Yes. My example shows that FreeBSD is more CPU-bound than I/O bound up to CPUs considerably faster than a 1.2GHz Pentium (though PentiumM is fast relative to its clock speed). The memory interface may matter more than the CPU clock. >>> NetFPGA doesn't have enough TCAM space to be useful for real routing >>> (as in Internet sized routing table). The trick many embedded networking >>> CPUs use is cache prefetching that is integrated with the network >>> controller. The first 64-128bytes of every packet are transferred >>> automatically into the L2 cache by the hardware. This allows relatively >>> slow CPUs (700 MHz Broadcom BCM1250 in Cisco NPE-G1 or 1.67-GHz Freescale >>> 7448 in NPE-G2) to get more than 1Mpps. Until something like this is >>> possible on Intel or AMD x86 CPUs we have a ceiling limited by RAM speed. >> >> Does using fa$ter memory (speed and/or latency) help here? 64 bytes >> is so small that latency may be more of a problem, especially without >> a prefetch. > > Latency. For IPv4 packet forwarding only one cache line per packet > is fetched. More memory speed only helps with the DMA from/to the > network card. I use low-end memory, but on the machine that does 640 kpps it somehow has latency almost 4 times as low as on new FreeBSD cluster machines (~42 nsec instead of ~150). perfmon (fixed for AXP and A64) and hwpmc report an average of 11 k8-dc-misses per sendto() while sending via bge at 640 kpps. 11 * 42 accounts for 442 nsec out of the 1562 per packet at this rate. 11 * 150 = 1650 would probably make this rate unachievable despite the system having 20 times as much CPU and bus. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 12:44:41 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A33911065672; Mon, 7 Jul 2008 12:44:41 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3DEB48FC47; Mon, 7 Jul 2008 12:44:40 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 583C546C65; Mon, 7 Jul 2008 08:44:40 -0400 (EDT) Date: Mon, 7 Jul 2008 13:44:40 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20080707213356.G7572@besplex.bde.org> Message-ID: <20080707134036.S63144@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 12:44:41 -0000 On Mon, 7 Jul 2008, Bruce Evans wrote: > I use low-end memory, but on the machine that does 640 kpps it somehow has > latency almost 4 times as low as on new FreeBSD cluster machines (~42 nsec > instead of ~150). perfmon (fixed for AXP and A64) and hwpmc report an > average of 11 k8-dc-misses per sendto() while sending via bge at 640 kpps. > 11 * 42 accounts for 442 nsec out of the 1562 per packet at this rate. 11 * > 150 = 1650 would probably make this rate unachievable despite the system > having 20 times as much CPU and bus. Since you're doing fine-grained performance measurements of a code path that interests me a lot, could you compare the cost per-send on UDP for the following four cases: (1) sendto() to a specific address and port on a socket that has been bound to INADDR_ANY and a specific port. (2) sendto() on a specific address and port on a socket that has been bound to a specific IP address (not INADDR_ANY) and a specific port. (3) send() on a socket that has been connect()'d to a specific IP address and a specific port, and bound to INADDR_ANY and a specific port. (4) send() on a socket that has been connect()'d to a specific IP address and a specific port, and bound to a specific IP address (not INADDR_ANY) and a specific port. The last of these should really be quite a bit faster than the first of these, but I'd be interested in seeing specific measurements for each if that's possible! Thanks, Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 12:46:08 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4C7E106564A; Mon, 7 Jul 2008 12:46:08 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7BD5B8FC16; Mon, 7 Jul 2008 12:46:08 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 1AF8546C61; Mon, 7 Jul 2008 08:46:08 -0400 (EDT) Date: Mon, 7 Jul 2008 13:46:08 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20080707134036.S63144@fledge.watson.org> Message-ID: <20080707134514.P63144@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> <20080707134036.S63144@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 12:46:08 -0000 On Mon, 7 Jul 2008, Robert Watson wrote: > The last of these should really be quite a bit faster than the first of > these, but I'd be interested in seeing specific measurements for each if > that's possible! And, if you're feeling particualrly subject to suggestion, you might consider comparing 7.0 recent 8.x along the same dimensions :-). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 12:56:24 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85EE31065684; Mon, 7 Jul 2008 12:56:24 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id 115978FC21; Mon, 7 Jul 2008 12:56:23 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67CuJrT026816 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2008 22:56:21 +1000 Date: Mon, 7 Jul 2008 22:56:19 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Robert Watson In-Reply-To: <20080707134036.S63144@fledge.watson.org> Message-ID: <20080707224659.B7844@besplex.bde.org> References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> <20080707134036.S63144@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 12:56:24 -0000 On Mon, 7 Jul 2008, Robert Watson wrote: > Since you're doing fine-grained performance measurements of a code path that > interests me a lot, could you compare the cost per-send on UDP for the > following four cases: > > (1) sendto() to a specific address and port on a socket that has been bound > to > INADDR_ANY and a specific port. > > (2) sendto() on a specific address and port on a socket that has been bound > to > a specific IP address (not INADDR_ANY) and a specific port. > > (3) send() on a socket that has been connect()'d to a specific IP address and > a specific port, and bound to INADDR_ANY and a specific port. > > (4) send() on a socket that has been connect()'d to a specific IP address > and a specific port, and bound to a specific IP address (not INADDR_ANY) > and a specific port. > > The last of these should really be quite a bit faster than the first of > these, but I'd be interested in seeing specific measurements for each if > that's possible! Not sure if I understand networking well enough to set these up quickly. Does netrate use one of (3) or (4) now? I can tell you vaguely about old results for netrate (send()) vs ttcp (sendto()). send() is lighter weight of course, and this made a difference of 10-20%, but after further tuning the difference became smaller, which suggests that everything ends up waiting for something in common. Now I can measure cache misses better and hope that a simple count of cache misses will be a more reproducible indicator of significant bottlenecks than pps. I got nowhere trying to reduce instruction counts, possibly because it would take avoiding 100's of instructions to get the same benefit as avoiding a single cache miss. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 13:28:38 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D22A1065696; Mon, 7 Jul 2008 13:28:38 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0A9888FC19; Mon, 7 Jul 2008 13:28:38 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (gavin@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m67DSb4Z079992; Mon, 7 Jul 2008 13:28:37 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m67DSbh4079988; Mon, 7 Jul 2008 13:28:37 GMT (envelope-from gavin) Date: Mon, 7 Jul 2008 13:28:37 GMT Message-Id: <200807071328.m67DSbh4079988@freefall.freebsd.org> To: francisgendreau@videotron.ca, gavin@FreeBSD.org, freebsd-net@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/125195: [fxp] fxp(4) driver failed to initialize device Intel 82801DB X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 13:28:38 -0000 Synopsis: [fxp] fxp(4) driver failed to initialize device Intel 82801DB State-Changed-From-To: open->feedback State-Changed-By: gavin State-Changed-When: Mon Jul 7 13:27:12 UTC 2008 State-Changed-Why: To submitter: Could you give the putput of "pciconf -l |grep fxp" please? http://www.freebsd.org/cgi/query-pr.cgi?pr=125195 From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 13:31:00 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 974E91065678 for ; Mon, 7 Jul 2008 13:31:00 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp02.sth.basefarm.net (ch-smtp02.sth.basefarm.net [80.76.149.213]) by mx1.freebsd.org (Postfix) with ESMTP id 18B968FC19 for ; Mon, 7 Jul 2008 13:30:59 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from c83-253-25-183.bredband.comhem.se ([83.253.25.183]:64382 helo=falcon.midgard.homeip.net) by ch-smtp02.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1KFqZM-000246-9G for freebsd-net@FreeBSD.org; Mon, 07 Jul 2008 15:15:53 +0200 Received: (qmail 32464 invoked from network); 7 Jul 2008 15:15:50 +0200 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with ESMTP; 7 Jul 2008 15:15:50 +0200 Received: (qmail 69245 invoked by uid 1001); 7 Jul 2008 15:15:50 +0200 Date: Mon, 7 Jul 2008 15:15:50 +0200 From: Erik Trulsson To: Bruce Evans Message-ID: <20080707131550.GA69202@owl.midgard.homeip.net> References: <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080707213356.G7572@besplex.bde.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Originating-IP: 83.253.25.183 X-Scan-Result: No virus found in message 1KFqZM-000246-9G. X-Scan-Signature: ch-smtp02.sth.basefarm.net 1KFqZM-000246-9G a6ef90968226b3c1c836da5aec42df3b Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 13:31:00 -0000 On Mon, Jul 07, 2008 at 10:30:53PM +1000, Bruce Evans wrote: > On Mon, 7 Jul 2008, Andre Oppermann wrote: > > > Bruce Evans wrote: > >> What are the other overheads? I calculate 1.644Mpps counting the > >> inter-frame > >> gap, with 64-byte packets and 64-header_size payloads. If the 64 bytes > >> is for the payload, then the max is much lower. > > > > The theoretical maximum at 64byte frames is 1,488,100. I've looked > > up my notes the 1.244Mpps number can be ajusted to 1.488Mpps. > > Where is the extra? I still get 1.644736 Mpps (10^9/(8*64+96)). > 1.488095 is for 64 bits extra (10^9/(8*64+96+64)). A standard ethernet frame (on the wire) consists of: 7 octets preamble 1 octet Start Frame Delimiter 6 octets destination address 6 octets source address 2 octets length/type 46-1500 octets data (+padding if needed) 4 octets Frame Check Sequence Followed by (at least) 96 bits interFrameGap, before the next frame starts. For minimal packet size this gives a maximum packet rate at 1Gbit/s of 1e9/((7+1+6+6+2+46+4)*8+96)/ = 1488095 packets/second You probably missed the preamble and start frame delimiter in your calculation. -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 13:37:30 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1C1A1065684 for ; Mon, 7 Jul 2008 13:37:30 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 07E6C8FC1A for ; Mon, 7 Jul 2008 13:37:29 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 32780 invoked from network); 7 Jul 2008 12:27:56 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 12:27:56 -0000 Message-ID: <48721C18.4060109@freebsd.org> Date: Mon, 07 Jul 2008 15:37:28 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Bruce Evans References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> In-Reply-To: <20080707213356.G7572@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 13:37:30 -0000 Bruce Evans wrote: > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Bruce Evans wrote: >>> What are the other overheads? I calculate 1.644Mpps counting the >>> inter-frame >>> gap, with 64-byte packets and 64-header_size payloads. If the 64 bytes >>> is for the payload, then the max is much lower. >> >> The theoretical maximum at 64byte frames is 1,488,100. I've looked >> up my notes the 1.244Mpps number can be ajusted to 1.488Mpps. > > Where is the extra? I still get 1.644736 Mpps (10^9/(8*64+96)). > 1.488095 is for 64 bits extra (10^9/(8*64+96+64)). The preamble has 64 bits and is in addition to the inter-frame gap. >>>>> I hoped to reach 1Mpps with the hardware I mentioned some mails >>>>> before, but 2Mpps is far far away. >>>>> Currently I get 160kpps via pci-32mbit-33mhz-1,2ghz mobile pentium. >>>> >>>> This is more or less expected. PCI32 is not able to sustain high >>>> packet rates. The bus setup times kill the speed. For larger packets >>>> the ratio gets much better and some reasonable throughput can be >>>> achieved. >>> >>> I get about 640 kpps without forwarding (sendto: slightly faster; >>> recvfrom: slightly slower) on a 2.2GHz A64. Underclocking the memory >>> from 200MHz to 100MHz only reduces the speed by about 10%, while not >>> overclocking the CPU by 10% reduces the speed by the same 10%, so the >>> system is apparently still mainly CPU-bound. >> >> On PCI32@33MHz? He's using a 1.2GHz Mobile Pentium on top of that. > > Yes. My example shows that FreeBSD is more CPU-bound than I/O bound up > to CPUs considerably faster than a 1.2GHz Pentium (though PentiumM is > fast relative to its clock speed). The memory interface may matter more > than the CPU clock. > >>>> NetFPGA doesn't have enough TCAM space to be useful for real routing >>>> (as in Internet sized routing table). The trick many embedded >>>> networking >>>> CPUs use is cache prefetching that is integrated with the network >>>> controller. The first 64-128bytes of every packet are transferred >>>> automatically into the L2 cache by the hardware. This allows >>>> relatively >>>> slow CPUs (700 MHz Broadcom BCM1250 in Cisco NPE-G1 or 1.67-GHz >>>> Freescale >>>> 7448 in NPE-G2) to get more than 1Mpps. Until something like this is >>>> possible on Intel or AMD x86 CPUs we have a ceiling limited by RAM >>>> speed. >>> >>> Does using fa$ter memory (speed and/or latency) help here? 64 bytes >>> is so small that latency may be more of a problem, especially without >>> a prefetch. >> >> Latency. For IPv4 packet forwarding only one cache line per packet >> is fetched. More memory speed only helps with the DMA from/to the >> network card. > > I use low-end memory, but on the machine that does 640 kpps it somehow > has latency almost 4 times as low as on new FreeBSD cluster machines > (~42 nsec instead of ~150). perfmon (fixed for AXP and A64) and hwpmc > report an average of 11 k8-dc-misses per sendto() while sending via > bge at 640 kpps. 11 * 42 accounts for 442 nsec out of the 1562 per > packet at this rate. 11 * 150 = 1650 would probably make this rate > unachievable despite the system having 20 times as much CPU and bus. We were talking routing here. That is a packet received via network interface and sent out on another. Crosses the PCI bus twice. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 13:39:00 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F8E8106568C; Mon, 7 Jul 2008 13:39:00 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id ABD368FC14; Mon, 7 Jul 2008 13:38:59 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 6EFC646C66; Mon, 7 Jul 2008 09:38:58 -0400 (EDT) Date: Mon, 7 Jul 2008 14:38:58 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20080707224659.B7844@besplex.bde.org> Message-ID: <20080707142018.U63144@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> <20080707134036.S63144@fledge.watson.org> <20080707224659.B7844@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 13:39:00 -0000 On Mon, 7 Jul 2008, Bruce Evans wrote: >> (1) sendto() to a specific address and port on a socket that has been bound >> to >> INADDR_ANY and a specific port. >> >> (2) sendto() on a specific address and port on a socket that has been bound >> to >> a specific IP address (not INADDR_ANY) and a specific port. >> >> (3) send() on a socket that has been connect()'d to a specific IP address >> and >> a specific port, and bound to INADDR_ANY and a specific port. >> >> (4) send() on a socket that has been connect()'d to a specific IP address >> and a specific port, and bound to a specific IP address (not INADDR_ANY) >> and a specific port. >> >> The last of these should really be quite a bit faster than the first of >> these, but I'd be interested in seeing specific measurements for each if >> that's possible! > > Not sure if I understand networking well enough to set these up quickly. > Does netrate use one of (3) or (4) now? (3) and (4) are effectively the same thing, I think, since connect(2) should force the selection of a source IP address, but I think it's not a bad idea to confirm that. :-) The structure of the desired micro-benchmark here is basically: int main(int argc, char *argv) { struct sockaddr_in sin; /* Parse command line arguments such as addresss and ports. */ if (bind_desired) { /* Set up sockaddr_in. */ if (bind(s, (struct sockaddr *)&sin, sizeof(sin)) < 0) err(-1, "bind"); } /* Set up destination sockaddr_in. */ if (connect_desired) { if (connect(s, (struct sockaddr *)&sin, sizeof(sin)) < 0) err(-1, "connect"); } while (appropriate_condition) { if (connect_desired) { if (send(s, ...) < 0) errors++; } else { if (sendto(s, (struct sockaddr *)&sin, sizeof(sin)) < 0) errors++; } } } > I can tell you vaguely about old results for netrate (send()) vs ttcp > (sendto()). send() is lighter weight of course, and this made a difference > of 10-20%, but after further tuning the difference became smaller, which > suggests that everything ends up waiting for something in common. > > Now I can measure cache misses better and hope that a simple count of cache > misses will be a more reproducible indicator of significant bottlenecks than > pps. I got nowhere trying to reduce instruction counts, possibly because it > would take avoiding 100's of instructions to get the same benefit as > avoiding a single cache miss. If you look at the design of the higher performance UDP applications, they will generally bind a specific IP (perhaps every IP on the host with its own socket), and if they do sustained communication to a specific endpoint they will use connect(2) rather than providing an address for each send(2) system call to the kernel. udp_output(2) makes the trade-offs there fairly clear: with the most recent rev, the optimal case is one connect(2) has been called, allowing a single inpcb read lock and no global data structure access, vs. an application calling sendto(2) for each system call and the local binding remaining INADDR_ANY. Middle ground applications, such as named(8) will force a local binding using bind(2), but then still have to pass an address to each sendto(2). In the future, this case will be further optimized in our code by using a global read lock rather than a global write lock: we have to check for collisions, but we don't actually have to reserve the new 4-tuple for the UDP socket as it's an ephemeral association rather than a connect(2). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 15:35:26 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E7F31065673; Mon, 7 Jul 2008 15:35:26 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail04.syd.optusnet.com.au (mail04.syd.optusnet.com.au [211.29.132.185]) by mx1.freebsd.org (Postfix) with ESMTP id C03C48FC26; Mon, 7 Jul 2008 15:35:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail04.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67FZJfk011715 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 01:35:21 +1000 Date: Tue, 8 Jul 2008 01:35:19 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Andre Oppermann In-Reply-To: <4871E618.1080500@freebsd.org> Message-ID: <20080708002228.G680@besplex.bde.org> References: <4867420D.7090406@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr><4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 15:35:26 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Paul, > > to get a systematic analysis of the performance please do the following > tests and put them into a table for easy comparison: > > 1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 > monitor) >... I won't be running many of these tests, but found this one interesting -- I didn't know about monitor mode. It gives the following behaviour: -monitor ttcp receiving on bge0 at 397 kpps: 35% idle (8.0-CURRENT) 13.6 cm/p monitor ttcp receiving on bge0 at 397 kpps: 83% idle (8.0-CURRENT) 5.8 cm/p -monitor ttcp receiving on em0 at 580 kpps: 5% idle (~5.2) 12.5 cm/p monitor ttcp receiving on em0 at 580 kpps: 65% idle (~5.2) 4.8 cm/p cm/p = k8-dc-misses (bge0 system) cm/p = k7-dc-misses (em0 system) So it seems that the major overheads are not near the driver (as I already knew), and upper layers are responsible for most of the cache misses. The packet header is accessed even in monitor mode, so I think most of the cache misses in upper layers are not related to the packet header. Maybe they are due mainly to perfect non-locality for mbufs. Other cm/p numbers: ttcp sending on bge0 at 640 kpps: (~5.2) 11 cm/p ttcp sending on bge0 at 580 kpps: (8.0-CURRENT) 9 cm/p (-current is 10% slower despite having lower cm/p. This seems to be due to extra instructions executed) ping -fq -c1000000 localhost at 171 kpps: (8.0-CURRENT) 12-33 cm/p (This is certainly CPU-bound. lo0 is much slower than bge0. Latency (rtt) is 2 us. It is 3 us in ~5.2 and was 4 in -current until very recently.) ping -fq -c1000000 etherhost at 40 kpps: (8.0-CURRENT) 55 cm/p (The rate is quite low because flood ping doesn't actually flood. It tries to limit the rate to max(100, 1/latency), but it tends to go at a rate of ql(t)/latency where ql(t) is the average hardware queue length at the current time t. ql(t) starts at 1 and builds up after a minute or 2 to a maximum of about 10 on my hardware. Latency is always ~100 us, so the average ql(t) must have been ~4.) ping -fq -c1000000 etherhost at 20 kpps: (8.0-CURRENT) 45 cm/p (Another run to record the average latency (it was 121) showed high variance.) netblast sending on bge0 at 582 kpps: (8.0-CURRENT) 9.8 cm/p (Packet blasting benchmarks actually flood, unlike flood ping. This is hard to implement, since select() for output-ready doesn't work. netblast has to busy wait, while ttcp guesses how long to sleep but cannot sleep for a short enough interval unless queues are too large or hz is too small. My systems are configured with HZ = 100 and snd.ifq too large so that sleeping for 1/Hz works for ttcp. netblast still busy-waits. This gives an interesting difference for netblast. It tries to send 800 k packets in 1 second by only successfully sends 582 k. 9.8 cm/p is for #misses / 582k. The 300k unsuccessful sends apparently don't cause many cache misses. But variance is high...) ttcp sending on bge0 at 577 kpps: (8.0-CURRENT) 15.5 cm/p (Another run shows high variance.) ttcp rates have low variance for a given kernel but high variance for different kernels (an extra unrelated byte in the text section can cause a 30% change). High variance would also be explained by non-locality of mbufs. Cycling through lots of mbufs would maximize cache misses but random reuse of mbufs would give variance. Or the cycling and variance might be more in general allocation. There is sillyness in getsockaddr(): sendit() calls getsockaddr() and getsockaddr() always uses malloc(), but allocation on the stack works for at the call from sendit(). This malloc() seemed to be responsible for a cache miss or two, but when I changed it to use the stack the results were inconclusive. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 16:20:10 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6FB81065678 for ; Mon, 7 Jul 2008 16:20:10 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 08E468FC1E for ; Mon, 7 Jul 2008 16:20:09 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 34019 invoked from network); 7 Jul 2008 15:10:34 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Jul 2008 15:10:34 -0000 Message-ID: <48724238.2020103@freebsd.org> Date: Mon, 07 Jul 2008 18:20:08 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: Bruce Evans References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> In-Reply-To: <20080708002228.G680@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 16:20:10 -0000 Bruce Evans wrote: > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Paul, >> >> to get a systematic analysis of the performance please do the following >> tests and put them into a table for easy comparison: >> >> 1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 >> monitor) >> ... > > I won't be running many of these tests, but found this one interesting -- > I didn't know about monitor mode. It gives the following behaviour: > > -monitor ttcp receiving on bge0 at 397 kpps: 35% idle (8.0-CURRENT) 13.6 > cm/p > monitor ttcp receiving on bge0 at 397 kpps: 83% idle (8.0-CURRENT) 5.8 > cm/p > -monitor ttcp receiving on em0 at 580 kpps: 5% idle (~5.2) 12.5 > cm/p > monitor ttcp receiving on em0 at 580 kpps: 65% idle (~5.2) 4.8 > cm/p > > cm/p = k8-dc-misses (bge0 system) > cm/p = k7-dc-misses (em0 system) > > So it seems that the major overheads are not near the driver (as I already > knew), and upper layers are responsible for most of the cache misses. > The packet header is accessed even in monitor mode, so I think most of > the cache misses in upper layers are not related to the packet header. > Maybe they are due mainly to perfect non-locality for mbufs. Monitor mode doesn't access the payload packet header. It only looks at the mbuf (which has a structure called mbuf packet header). The mbuf header it hot in the cache because the driver just touched it and filled in the information. The packet content (the payload) is cold and just arrived via DMA in DRAM. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 16:41:52 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55A6A1065674 for ; Mon, 7 Jul 2008 16:41:52 +0000 (UTC) (envelope-from nettwork@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 971278FC13 for ; Mon, 7 Jul 2008 16:41:51 +0000 (UTC) (envelope-from nettwork@gmx.de) Received: (qmail invoked by alias); 07 Jul 2008 16:15:09 -0000 Received: from e178233171.adsl.alicedsl.de (HELO [10.0.0.100]) [85.178.233.171] by mail.gmx.net (mp007) with SMTP; 07 Jul 2008 18:15:10 +0200 X-Authenticated: #46460896 X-Provags-ID: V01U2FsdGVkX19o38J0xW5oi/OZ6hvbzePwDoOmMriTkqf6I+gD1z 1X5cdcHwOyDMGF From: Achim To: freebsd-net@freebsd.org Date: Mon, 7 Jul 2008 16:15:40 +0000 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200807071615.40987.nettwork@gmx.de> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.64 Subject: smbmount / smbclient : strangely varying transfer speeds X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 16:41:52 -0000 Hello List, I've experienced the following with both a kubuntu and a FBSD7 client and FBSD7 as server: When i try to copy a file off a *mounted* CIFS/SMB-share I get transfer rates below 1 MByte/sec. If i start a second, concurrent transfer i am getting transfer rates around 8MB/s on *each* transfer (Gigabit link). As soon as one transfer stops, the other is dropping to the old rate below 1MB/s. Copying TO the server works fine, atleast from the kubuntu client, the fbsd client isnt here atm. It speeds up the initial transfer to about 3.5MB/s, about half of what a concurrent download does. A single transfer via smbclient yields ~8MB/s. In other Words: Performance with a single client is degraded when the client is smbmount and downloading. With a second transfer in any direction, performance becomes better, to about 3.5 resp. 8 MB/s depending on the second connection up- or downloading. Unlike smbmount, single smbclient transfers yield acceptable results. Anyone with an idea as what to try? My wireshark skills aren't too advanced and i could not find any notable difference between the captures of each transfer type (single mounted, multiple mounted and single smblient), anything i should watch out for? The machines are connected over a simple soho gigabit switch, no fancy network between them. thanks in advance, Achim From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 18:16:49 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 004691065682; Mon, 7 Jul 2008 18:16:48 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.freebsd.org (Postfix) with ESMTP id 82DF58FC2F; Mon, 7 Jul 2008 18:16:48 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67IGgGo021885 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 04:16:44 +1000 Date: Tue, 8 Jul 2008 04:16:42 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Andre Oppermann In-Reply-To: <48724238.2020103@freebsd.org> Message-ID: <20080708034304.R21502@delplex.bde.org> References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Ingo Flaschberger , FreeBSD Net , Bart Van Kerckhove , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 18:16:49 -0000 On Mon, 7 Jul 2008, Andre Oppermann wrote: > Bruce Evans wrote: >> So it seems that the major overheads are not near the driver (as I already >> knew), and upper layers are responsible for most of the cache misses. >> The packet header is accessed even in monitor mode, so I think most of >> the cache misses in upper layers are not related to the packet header. >> Maybe they are due mainly to perfect non-locality for mbufs. > > Monitor mode doesn't access the payload packet header. It only looks > at the mbuf (which has a structure called mbuf packet header). The mbuf > header it hot in the cache because the driver just touched it and filled > in the information. The packet content (the payload) is cold and just > arrived via DMA in DRAM. Why does it use ntohs() then? :-). From if_ethersubr.c: % static void % ether_input(struct ifnet *ifp, struct mbuf *m) % { % struct ether_header *eh; % u_short etype; % % if ((ifp->if_flags & IFF_UP) == 0) { % m_freem(m); % return; % } % #ifdef DIAGNOSTIC % if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) { % if_printf(ifp, "discard frame at !IFF_DRV_RUNNING\n"); % m_freem(m); % return; % } % #endif % /* % * Do consistency checks to verify assumptions % * made by code past this point. % */ % if ((m->m_flags & M_PKTHDR) == 0) { % if_printf(ifp, "discard frame w/o packet header\n"); % ifp->if_ierrors++; % m_freem(m); % return; % } % if (m->m_len < ETHER_HDR_LEN) { % /* XXX maybe should pullup? */ % if_printf(ifp, "discard frame w/o leading ethernet " % "header (len %u pkt len %u)\n", % m->m_len, m->m_pkthdr.len); % ifp->if_ierrors++; % m_freem(m); % return; % } % eh = mtod(m, struct ether_header *); Point outside of mbuf header. % etype = ntohs(eh->ether_type); First access outside of mbuf header. But this seems to be bogus and might be fixed by compiler optimization, since etype is not used until after the monitor mode returns. This may have been broken by debugging cruft -- in 5.2, etype is used immediately after here in a printf about discarding oversize frames. The compiler might also pessimize things by reordering code. % if (m->m_pkthdr.rcvif == NULL) { % if_printf(ifp, "discard frame w/o interface pointer\n"); % ifp->if_ierrors++; % m_freem(m); % return; % } % #ifdef DIAGNOSTIC % if (m->m_pkthdr.rcvif != ifp) { % if_printf(ifp, "Warning, frame marked as received on %s\n", % m->m_pkthdr.rcvif->if_xname); % } % #endif % % if (ETHER_IS_MULTICAST(eh->ether_dhost)) { % if (ETHER_IS_BROADCAST(eh->ether_dhost)) % m->m_flags |= M_BCAST; % else % m->m_flags |= M_MCAST; % ifp->if_imcasts++; % } Another dereference of eh (2 unless optimizable and optimized). Here the result is actually used early, but I think you don't care enough about maintaing if_imcasts to do this. % % #ifdef MAC % /* % * Tag the mbuf with an appropriate MAC label before any other % * consumers can get to it. % */ % mac_ifnet_create_mbuf(ifp, m); % #endif % % /* % * Give bpf a chance at the packet. % */ % ETHER_BPF_MTAP(ifp, m); I think this can access the whole packet, but usually doesn't. % % /* % * If the CRC is still on the packet, trim it off. We do this once % * and once only in case we are re-entered. Nothing else on the % * Ethernet receive path expects to see the FCS. % */ % if (m->m_flags & M_HASFCS) { % m_adj(m, -ETHER_CRC_LEN); % m->m_flags &= ~M_HASFCS; % } % % ifp->if_ibytes += m->m_pkthdr.len; % % /* Allow monitor mode to claim this frame, after stats are updated. */ % if (ifp->if_flags & IFF_MONITOR) { % m_freem(m); % return; % } Finally return in monitor mode. I don't see any stats update before here except for the stray if_imcasts one. BTW, stats behave strangely in monitor mode: - netstat -I 1 works except: - the byte counts are 0 every second second (the next second counts the previous 2), while the packet counts are update every second - one system started showing bge0 stats for all interfaces. Perhaps unrelated. - systat -ip shows all counts 0. I think this is due to stats maintained by the driver working but other stats not. The mixture seems strange at user level. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 18:30:09 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C4E08106567D for ; Mon, 7 Jul 2008 18:30:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B2B568FC23 for ; Mon, 7 Jul 2008 18:30:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m67IU9WE008240 for ; Mon, 7 Jul 2008 18:30:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m67IU9ZG008237; Mon, 7 Jul 2008 18:30:09 GMT (envelope-from gnats) Date: Mon, 7 Jul 2008 18:30:09 GMT Message-Id: <200807071830.m67IU9ZG008237@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Alexander Motin Cc: Subject: Re: kern/123200: [netgraph] Server failure due to netgraph mpd and dhcpclient X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alexander Motin List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 18:30:09 -0000 The following reply was made to PR kern/123200; it has been noted by GNATS. From: Alexander Motin To: bug-followup@FreeBSD.org, zaulychny@yahoo.com Cc: Subject: Re: kern/123200: [netgraph] Server failure due to netgraph mpd and dhcpclient Date: Mon, 07 Jul 2008 21:27:58 +0300 If I understand right, you are receiving route to you VPN server using DHCP. I think you could get in trouble when DHCP lease time ended and you loose that route making VPN connection route default. In it's place it could cause routing loop by wrapping tunnel inside itself, causing in-kernel recursion loop. I have some feedbacks that stack protection mechanisms added to stable allow system better handle such case. Could you upgrade you system to the 6-STABLE and try again? -- Alexander Motin From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 18:43:00 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FB95106568A; Mon, 7 Jul 2008 18:42:58 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 2A3678FC1B; Mon, 7 Jul 2008 18:42:58 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KFvbw-0008H5-6L; Mon, 07 Jul 2008 14:38:52 -0400 Message-ID: <48726422.7050703@gtcomm.net> Date: Mon, 07 Jul 2008 14:44:50 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Andre Oppermann References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> In-Reply-To: <4871E85C.8090907@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 18:43:00 -0000 > one that will later on handle the taskqueue to process the packets. > That adds overhead. Ideally the interrupt for each network interface > is bound to exactly one pre-determined CPU and the taskqueue is bound > to the same CPU. That way the overhead for interrupt and taskqueue > scheduling can be kept at a minimum. Most of the infrastructure to > do this binding already exists in the kernel but is not yet exposed > to the outside for us to make use of it. I'm also not sure if the > ULE scheduler skips the more global locks when interrupt and the > thread are on the same CPU. > > Distributing the interrupts and taskqueues among the available CPUs > gives concurrent forwarding with bi- or multi-directional traffic. > All incoming traffic from any particular interface is still serialized > though. > I used etherchannel to distribute incoming packets over 3 separate cpus evenly but the output was on one interface.. What I got was less performance than with one cpu and all three cpus were close to 100% utilizied. em0,em1,em2 were all receiving packets and sending them out em3. The machine had 4 cpus in it. em3 taskq was low cpu usage and em0,1,2 were using cpu0,1,2(for example) almost fully used. With all that cpu power being used and I got less performance than with 1 cpu :/ Obviously in SMP there is a big issue somewhere. Also my 82571 NIC supports multiple received queues and multiple transmit queues so why hasn't anyone written the driver to support this? It's not a 10gb card and it still supports it and it's widely available and not too expensive either. The new 82575/6 chips support even more queues and the two port version will be out this month and the 4 port in october (PCI-E cards). Motherboards are already shipping with the 82576.. (82571 supports 2x/2x 575/6 support 4x/4x) Paul From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 18:55:15 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D555106566C; Mon, 7 Jul 2008 18:55:15 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id E77308FC14; Mon, 7 Jul 2008 18:55:14 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KFvnp-0005Dl-74; Mon, 07 Jul 2008 14:51:09 -0400 Message-ID: <4872670A.9050801@gtcomm.net> Date: Mon, 07 Jul 2008 14:57:14 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Bruce Evans References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> In-Reply-To: <20080707213356.G7572@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 18:55:15 -0000 > I use low-end memory, but on the machine that does 640 kpps it somehow > has latency almost 4 times as low as on new FreeBSD cluster machines > (~42 nsec instead of ~150). perfmon (fixed for AXP and A64) and hwpmc > report an average of 11 k8-dc-misses per sendto() while sending via > bge at 640 kpps. 11 * 42 accounts for 442 nsec out of the 1562 per > packet at this rate. 11 * 150 = 1650 would probably make this rate > unachievable despite the system having 20 times as much CPU and bus. > Any of the buffered dimms or ddr3 or high cas ddr2 are going to have a lot more latency than older ones because the frequency is so high or the buffering. The best is to use ddr2 with the lowest timings that it supports at the highest frequency but not the highest frequency it supports at higher timings.. for instance i have some 1100mhz ddr2 ram but it's 5-5-5-15 but it will do 5-4-4-12 at 1000 or 900 Mhz so I think the latency may have more impact on the speed than the actual MHz of the ram itself. This works for several benchmarks which I have tested before running the ram at 1:1 with the FSB (400 FSB(1600fsb actual) with ram at 800 and the latency is a lot lower than ram at 1:1.20 FSB even though the bandwidth is higher) With higher latency in the 'server' machines we probably need to do things in bigger chunks.. Anyone using a FBSD router isn't going to care about a 1ms delay in the packet but they will care if packets are dropped or reordered. Paul From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 19:15:46 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3799106564A; Mon, 7 Jul 2008 19:15:46 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id DB7288FC22; Mon, 7 Jul 2008 19:15:45 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67JFcFV001599 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 05:15:41 +1000 Date: Tue, 8 Jul 2008 05:15:38 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Bruce Evans In-Reply-To: <20080708034304.R21502@delplex.bde.org> Message-ID: <20080708045135.V1022@besplex.bde.org> References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net><486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net><486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net><486B4F11.6040906@gtcomm.net><486BC7F5.5070604@gtcomm.net><20080703160540.W6369@delplex.bde.org><486C7F93.7010308@gtcomm.net><20080703195521.O6973@delplex.bde.org><486D35A0.4000302@gtcomm.net><486DF1A3.9000409@gtcomm.net><486E65E6.3060301@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Bart Van Kerckhove , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 19:15:46 -0000 On Tue, 8 Jul 2008, Bruce Evans wrote: > On Mon, 7 Jul 2008, Andre Oppermann wrote: > >> Bruce Evans wrote: >>> So it seems that the major overheads are not near the driver (as I already >>> knew), and upper layers are responsible for most of the cache misses. >>> The packet header is accessed even in monitor mode, so I think most of >>> the cache misses in upper layers are not related to the packet header. >>> Maybe they are due mainly to perfect non-locality for mbufs. >> >> Monitor mode doesn't access the payload packet header. It only looks >> at the mbuf (which has a structure called mbuf packet header). The mbuf >> header it hot in the cache because the driver just touched it and filled >> in the information. The packet content (the payload) is cold and just >> arrived via DMA in DRAM. > > Why does it use ntohs() then? :-). From if_ethersubr.c: > ... > % eh = mtod(m, struct ether_header *); > > Point outside of mbuf header. > > % etype = ntohs(eh->ether_type); > > First access outside of mbuf header. > ... > % % /* Allow monitor mode to claim this frame, after stats are updated. > */ > % if (ifp->if_flags & IFF_MONITOR) { > % m_freem(m); > % return; > % } > > Finally return in monitor mode. > > I don't see any stats update before here except for the stray if_imcasts > one. There are some error stats with printfs, but I've never seen these do anything except with a buggy sk driver. Testing verifies that accessing eh above gives a cache miss. Under ~5.2 receiving on bge0 at 397 kpps: -monitor: 17% idle 19 cm/p (18% less idle than under -current) monitor: 66% idle 8 cm/p (17% less idle than under -current) +monitor: 71% idle 7 cm/p (idle time under -current not measured) +monitor is monitor mode with the exit moved to the top of ether_input(). If the cache miss takes the time measured by lmbench2 (42 ns), then 397 k of these per second gives 17 ms or 1.7% CPU, which is vaguely consistent with the improvement of 5% by not taking this cache miss. Avoiding most of the 19 cache misses should give much more than a 5% improvement. Maybe -current gets its 17% improvement by avoiding some. More userland stats weirdness in userland: - in monitor mode, em0 gives byte counts delayed while bge0 gives byte counts always 0. - netstat -I 1 seems to be broken in ~5.2 in all modes -- it gives output for interfaces with drivers but no hardware. All this is for UP. An SMP kernel on the same UP system loses < 5% for at least tx. Bruce From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 19:53:45 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 700B31065680 for ; Mon, 7 Jul 2008 19:53:45 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.189]) by mx1.freebsd.org (Postfix) with ESMTP id EA6C78FC65 for ; Mon, 7 Jul 2008 19:53:44 +0000 (UTC) (envelope-from artemb@gmail.com) Received: by fk-out-0910.google.com with SMTP id k31so1267360fkk.11 for ; Mon, 07 Jul 2008 12:53:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=IZz6YA8lcBB0+cwM3lKHpFz41Tggpbc/toqW9XtBvt8=; b=oRQPdQQU9ZMDIqrcDyXt+inB34qI8XY+oR1scoOzIZ/g+Izshj+cJCzT7zWDnEHv3M IHz71VuM7HvjyQ9nD3Aj3NEVD9Nu5N5Tcp5b0nFaUPOlPPHHLkCUe4Id4qoLjSurmtsR NXyA3rHXRy5Gqogj544VkbfOzSPrka2fBuMjY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=HDy8Kq0/gULQGtivKUAcONbcdnaBElu7cO67nNJpCk+iq1Bd8K/+u2jwnNzEnipVVL Jlc41pVNG3sKm5AE5BNEWM+LUoQ/Xw1dDcfZAN+cOyS1m8eoRu39PpYeqm3A4BqhmugN OPRMbdAwY7Rcie4fxqh6QP//Hj3Swk36jQPjM= Received: by 10.86.25.17 with SMTP id 17mr4557527fgy.50.1215458872941; Mon, 07 Jul 2008 12:27:52 -0700 (PDT) Received: by 10.86.83.19 with HTTP; Mon, 7 Jul 2008 12:27:52 -0700 (PDT) Message-ID: Date: Mon, 7 Jul 2008 12:27:52 -0700 From: "Artem Belevich" Sender: artemb@gmail.com To: "FreeBSD Net" In-Reply-To: <20080708045135.V1022@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> X-Google-Sender-Auth: 1701e430fa4259c1 Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 19:53:45 -0000 Hi, As was already mentioned, we can't avoid all cache misses as there's data that's recently been updated in memory via DMA and therefor kicked out of cache. However, we may hide some of the latency penalty by prefetching 'interesting' data early. I.e. we know that we want to access some ethernet headers, so we may start pulling relevant data into cache early. Ideally, by the time we need to access the field, it will already be in the cache. When we're counting nanoseconds per packet this may bring some performance gain. Just my $0.02. --Artem From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 20:06:20 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F3B1106567A for ; Mon, 7 Jul 2008 20:06:20 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id D17BC8FC12 for ; Mon, 7 Jul 2008 20:06:19 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m67K6HRL006133; Mon, 7 Jul 2008 16:06:17 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m67K6GTE020938 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2008 16:06:16 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807072006.m67K6GTE020938@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 07 Jul 2008 16:06:16 -0400 To: Paul From: Mike Tancsa In-Reply-To: <48726422.7050703@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 20:06:20 -0000 At 02:44 PM 7/7/2008, Paul wrote: >Also my 82571 NIC supports multiple received queues and multiple >transmit queues so why hasn't >anyone written the driver to support this? It's not a 10gb card and >it still supports it and it's widely Intel actually maintains the driver. Not sure if there are plans or not, but perhaps they can comment ? ---Mike >available and not too expensive either. The new 82575/6 chips >support even more queues and the >two port version will be out this month and the 4 port in october >(PCI-E cards). Motherboards are >already shipping with the 82576.. (82571 supports 2x/2x 575/6 >support 4x/4x) > >Paul > > > > > > > > >_______________________________________________ >freebsd-net@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 20:21:11 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B94201065684 for ; Mon, 7 Jul 2008 20:21:11 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 880408FC2B for ; Mon, 7 Jul 2008 20:21:11 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KFx93-0003lj-DW; Mon, 07 Jul 2008 16:17:09 -0400 Message-ID: <48727B31.3070003@gtcomm.net> Date: Mon, 07 Jul 2008 16:23:13 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Mike Tancsa References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807072006.m67K6GTE020938@lava.sentex.ca> In-Reply-To: <200807072006.m67K6GTE020938@lava.sentex.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 20:21:11 -0000 I hope so, if they maintain the driver then why wouldn't they make it take advantage of their own hardware? I hope they are stuck focusing on windows users :/ Mike Tancsa wrote: > At 02:44 PM 7/7/2008, Paul wrote: >> Also my 82571 NIC supports multiple received queues and multiple >> transmit queues so why hasn't >> anyone written the driver to support this? It's not a 10gb card and >> it still supports it and it's widely > > > Intel actually maintains the driver. Not sure if there are plans or > not, but perhaps they can comment ? > > ---Mike From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 20:25:14 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F04991065683 for ; Mon, 7 Jul 2008 20:25:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outT.internet-mail-service.net (outt.internet-mail-service.net [216.240.47.243]) by mx1.freebsd.org (Postfix) with ESMTP id D36108FC17 for ; Mon, 7 Jul 2008 20:25:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id BFF0A2408; Mon, 7 Jul 2008 13:25:17 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 9304B2D6006; Mon, 7 Jul 2008 13:25:13 -0700 (PDT) Message-ID: <48727BA9.6020702@elischer.org> Date: Mon, 07 Jul 2008 13:25:13 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Artem Belevich References: <4867420D.7090406@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 20:25:15 -0000 Artem Belevich wrote: > Hi, > > As was already mentioned, we can't avoid all cache misses as there's > data that's recently been updated in memory via DMA and therefor > kicked out of cache. > > However, we may hide some of the latency penalty by prefetching > 'interesting' data early. I.e. we know that we want to access some > ethernet headers, so we may start pulling relevant data into cache > early. Ideally, by the time we need to access the field, it will > already be in the cache. When we're counting nanoseconds per packet > this may bring some performance gain. Prefetching when you are waiting for the data isn't a help. what you need is a speculative prefetch where you an tell teh processor "We will probably need the following address so start getting it while we go do other stuff". As far as I know we have no capacity to do that.. > > Just my $0.02. > --Artem > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 22:13:03 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B59CA106567D for ; Mon, 7 Jul 2008 22:13:03 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail13.syd.optusnet.com.au (mail13.syd.optusnet.com.au [211.29.132.194]) by mx1.freebsd.org (Postfix) with ESMTP id 452B98FC13 for ; Mon, 7 Jul 2008 22:13:03 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-215-175.belrs3.nsw.optusnet.com.au [122.106.215.175]) by mail13.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m67MCwVJ016846 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 08:12:59 +1000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.2/8.14.2) with ESMTP id m67MCwnH068137; Tue, 8 Jul 2008 08:12:58 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.2/8.14.2/Submit) id m67MCvGe068136; Tue, 8 Jul 2008 08:12:57 +1000 (EST) (envelope-from peter) Date: Tue, 8 Jul 2008 08:12:57 +1000 From: Peter Jeremy To: Julian Elischer Message-ID: <20080707221257.GH62764@server.vk2pj.dyndns.org> References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hQiwHBbRI9kgIhsi" Content-Disposition: inline In-Reply-To: <48727BA9.6020702@elischer.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.18 (2008-05-17) Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 22:13:03 -0000 --hQiwHBbRI9kgIhsi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2008-Jul-07 13:25:13 -0700, Julian Elischer wrote: >what you need is a speculative prefetch where you an tell teh=20 >processor "We will probably need the following address so start=20 >getting it while we go do other stuff". This looks like the PREFETCH instructions that exist in at least amd64 and SPARC. Unfortunately, their optimal use is very implementation- dependent and the AMD documentation suggests that incorrect use can degrade performance. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --hQiwHBbRI9kgIhsi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkhylOkACgkQ/opHv/APuIf8cACgtZVatoLIA+W0RtB+atFbbhiy WBgAoMHGNn/i3Rs4y4wD7c9Odcfswne+ =MR/n -----END PGP SIGNATURE----- --hQiwHBbRI9kgIhsi-- From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 22:17:23 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AB6C1065675 for ; Mon, 7 Jul 2008 22:17:23 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outD.internet-mail-service.net (outd.internet-mail-service.net [216.240.47.227]) by mx1.freebsd.org (Postfix) with ESMTP id 2F3A08FC1E for ; Mon, 7 Jul 2008 22:17:23 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id BF0682364; Mon, 7 Jul 2008 15:17:50 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id A0C282D601A; Mon, 7 Jul 2008 15:17:22 -0700 (PDT) Message-ID: <487295F2.7070802@elischer.org> Date: Mon, 07 Jul 2008 15:17:22 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Peter Jeremy References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> <20080707221257.GH62764@server.vk2pj.dyndns.org> In-Reply-To: <20080707221257.GH62764@server.vk2pj.dyndns.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 22:17:23 -0000 Peter Jeremy wrote: > On 2008-Jul-07 13:25:13 -0700, Julian Elischer wrote: >> what you need is a speculative prefetch where you an tell teh >> processor "We will probably need the following address so start >> getting it while we go do other stuff". > > This looks like the PREFETCH instructions that exist in at least amd64 > and SPARC. Unfortunately, their optimal use is very implementation- > dependent and the AMD documentation suggests that incorrect use can > degrade performance. > It might be worth looking to see if the network processing threads might be able to prefetch the IP header at least :-) From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 22:33:05 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EF74106567F for ; Mon, 7 Jul 2008 22:33:05 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id A94988FC17 for ; Mon, 7 Jul 2008 22:33:04 +0000 (UTC) (envelope-from artemb@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1234370fgb.35 for ; Mon, 07 Jul 2008 15:33:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=Z+FFaDpSreNnaUoiBpVDDpeQ3+lvtwYBkPFkCaCTpJk=; b=c2XYo5p+nh8TV6ajVA7uV1zIBGdiLdunVnmzq3wZO12qZegQubuz10uOCnybVrzq7A Cb9BjarMatRbLUoZUHjJIGBdfm7nuk2Z1DB9PBqAb7FxF9l8URciXg5b3zbXbm7T0vOK zWc3LYhqVchtqGF/2gFH3P8Y32vcfZQsZLt5w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=F8eqmGIFK9IF9ZNk3I/2kvt6VlE/yO+V1BYjEyJERvn4g89+KAg5wTmIyJqGF4V66j 4kmqBxLDXc0KU5Qm32XbC20dM2gDS7NSZHNTjW0spjLGSQPxL++8sSDo5xRxMK6akJon xevowR0ukKB51fgcgQ5I8JyjM01rebck9fp5A= Received: by 10.86.52.6 with SMTP id z6mr4726861fgz.48.1215469982840; Mon, 07 Jul 2008 15:33:02 -0700 (PDT) Received: by 10.86.83.19 with HTTP; Mon, 7 Jul 2008 15:33:02 -0700 (PDT) Message-ID: Date: Mon, 7 Jul 2008 15:33:02 -0700 From: "Artem Belevich" Sender: artemb@gmail.com To: "Julian Elischer" In-Reply-To: <48727BA9.6020702@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> X-Google-Sender-Auth: 45f63404a9c52e94 Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 22:33:05 -0000 > Prefetching when you are waiting for the data isn't a help. Agreed. Got to start prefetch around ns before you actually need the data and move on doing other things that do not depend on the data you've just started prefetching. > what you need is a speculative prefetch where you an tell teh processor "We > will probably need the following address so start getting it while we go do > other stuff". It does not have to be 'speculative' either. In this particular case we have very good idea that we *will* need some data from ethernet header and, probably, IP and TCP headers as well. We might as well tel the hardware to start pulling data in without stalling the CPU. Intel has instructions specifically for this purpose. I assume AMD has them too. --Artem From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 23:03:56 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F679106564A for ; Mon, 7 Jul 2008 23:03:56 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 040B48FC1C for ; Mon, 7 Jul 2008 23:03:55 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KFzgU-0006sS-M3; Mon, 07 Jul 2008 18:59:50 -0400 Message-ID: <4872A155.70606@gtcomm.net> Date: Mon, 07 Jul 2008 19:05:57 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Artem Belevich References: <4867420D.7090406@gtcomm.net> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Julian Elischer Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 23:03:56 -0000 We could add this as a part of the fastforwarding code and for a router turn it on and for a server leave it off. When I use a FBSD box for a router, it doesn't do anything else, so there could be two optimized paths that is one for routing/forwarding/firewalling only and one for use as a server. Artem Belevich wrote: >> Prefetching when you are waiting for the data isn't a help. >> > > Agreed. Got to start prefetch around ns > before you actually need the data and move on doing other things that > do not depend on the data you've just started prefetching. > > >> what you need is a speculative prefetch where you an tell teh processor "We >> will probably need the following address so start getting it while we go do >> other stuff". >> > > It does not have to be 'speculative' either. In this particular case > we have very good idea that we *will* need some data from ethernet > header and, probably, IP and TCP headers as well. We might as well tel > the hardware to start pulling data in without stalling the CPU. Intel > has instructions specifically for this purpose. I assume AMD has them > too. > > --Artem > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 23:18:35 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB86E1065672 for ; Mon, 7 Jul 2008 23:18:35 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (host-122-100-2-232.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 82EF98FC19 for ; Mon, 7 Jul 2008 23:18:35 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id BCBEA17D7E; Tue, 8 Jul 2008 09:18:32 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (138.21.96.58.exetel.com.au [58.96.21.138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 040A317D6C for ; Tue, 8 Jul 2008 09:18:24 +1000 (EST) Message-ID: <4872A3FE.706@modulus.org> Date: Tue, 08 Jul 2008 09:17:18 +1000 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: FreeBSD Net References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807072006.m67K6GTE020938@lava.sentex.ca> In-Reply-To: <200807072006.m67K6GTE020938@lava.sentex.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 23:18:35 -0000 Mike Tancsa wrote: > At 02:44 PM 7/7/2008, Paul wrote: >> Also my 82571 NIC supports multiple received queues and multiple >> transmit queues so why hasn't >> anyone written the driver to support this? It's not a 10gb card and >> it still supports it and it's widely > Intel actually maintains the driver. Not sure if there are plans or not, > but perhaps they can comment ? Last time Jack Vogel weighed in, I believe he said that the support for multiple queues and other performance enhancements are on the way. Some work had to be done to the networking infrastructure in FreeBSD first to allow this. - Andrew From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 23:21:59 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E455C106567D for ; Mon, 7 Jul 2008 23:21:59 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outN.internet-mail-service.net (outn.internet-mail-service.net [216.240.47.237]) by mx1.freebsd.org (Postfix) with ESMTP id C9A678FC0C for ; Mon, 7 Jul 2008 23:21:59 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 640A5242F; Mon, 7 Jul 2008 16:22:18 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 548992D6016; Mon, 7 Jul 2008 16:21:59 -0700 (PDT) Message-ID: <4872A516.4000306@elischer.org> Date: Mon, 07 Jul 2008 16:21:58 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Artem Belevich References: <4867420D.7090406@gtcomm.net> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2008 23:22:00 -0000 Artem Belevich wrote: >> Prefetching when you are waiting for the data isn't a help. > > Agreed. Got to start prefetch around ns > before you actually need the data and move on doing other things that > do not depend on the data you've just started prefetching. > >> what you need is a speculative prefetch where you an tell teh processor "We >> will probably need the following address so start getting it while we go do >> other stuff". > > It does not have to be 'speculative' either. "*Will*" is just a very definite subset of 'speculation' :-) > In this particular case > we have very good idea that we *will* need some data from ethernet > header and, probably, IP and TCP headers as well. We might as well tel > the hardware to start pulling data in without stalling the CPU. Intel > has instructions specifically for this purpose. I assume AMD has them > too. > > --Artem > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 01:07:36 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D0B8106564A; Tue, 8 Jul 2008 01:07:36 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id C09DD8FC15; Tue, 8 Jul 2008 01:07:35 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m6817YB1048522; Mon, 7 Jul 2008 21:07:34 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m6817XxO021966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2008 21:07:33 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807080107.m6817XxO021966@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 07 Jul 2008 21:07:33 -0400 To: Paul , Andre Oppermann From: Mike Tancsa In-Reply-To: <48726422.7050703@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 01:07:36 -0000 At 02:44 PM 7/7/2008, Paul wrote: >Also my 82571 NIC supports multiple received queues and multiple >transmit queues so why hasn't >anyone written the driver to support this? It's not a 10gb card and >it still supports it and it's widely >available and not too expensive either. The new 82575/6 chips >support even more queues and the >two port version will be out this month and the 4 port in october >(PCI-E cards). Motherboards are >already shipping with the 82576.. (82571 supports 2x/2x 575/6 >support 4x/4x) Actually, do any of your NICs attach via the igb driver ? ---Mike From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 01:20:40 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 870F31065675; Tue, 8 Jul 2008 01:20:40 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 424188FC20; Tue, 8 Jul 2008 01:20:40 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KG1oo-0004il-T9; Mon, 07 Jul 2008 21:16:34 -0400 Message-ID: <4872C161.6080105@gtcomm.net> Date: Mon, 07 Jul 2008 21:22:41 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Mike Tancsa References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> In-Reply-To: <200807080107.m6817XxO021966@lava.sentex.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Andre Oppermann Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 01:20:40 -0000 I read through the IGB driver, and it says 82575/6 only... which is the new chip Intel is releasing on the cards this month 2 port and october 4 port, but the chips are on some of the motherboards right now. Why can't it also use the 82571 ? doesn't make any sense.. I haven't tried it but just browsing the driver source doesn't look like it will work. Mike Tancsa wrote: > At 02:44 PM 7/7/2008, Paul wrote: > >> Also my 82571 NIC supports multiple received queues and multiple >> transmit queues so why hasn't >> anyone written the driver to support this? It's not a 10gb card and >> it still supports it and it's widely >> available and not too expensive either. The new 82575/6 chips >> support even more queues and the >> two port version will be out this month and the 4 port in october >> (PCI-E cards). Motherboards are >> already shipping with the 82576.. (82571 supports 2x/2x 575/6 >> support 4x/4x) > > > > > Actually, do any of your NICs attach via the igb driver ? > > ---Mike > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 03:32:54 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45E6A1065670 for ; Tue, 8 Jul 2008 03:32:54 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail14.syd.optusnet.com.au (mail14.syd.optusnet.com.au [211.29.132.195]) by mx1.freebsd.org (Postfix) with ESMTP id D0B108FC25 for ; Tue, 8 Jul 2008 03:32:47 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-215-175.belrs3.nsw.optusnet.com.au [122.106.215.175]) by mail14.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m683WeBZ011343 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 13:32:41 +1000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.2/8.14.2) with ESMTP id m683WeN6069225; Tue, 8 Jul 2008 13:32:40 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.2/8.14.2/Submit) id m683WeR9069224; Tue, 8 Jul 2008 13:32:40 +1000 (EST) (envelope-from peter) Date: Tue, 8 Jul 2008 13:32:40 +1000 From: Peter Jeremy To: Achim Message-ID: <20080708033239.GL62764@server.vk2pj.dyndns.org> References: <200807071615.40987.nettwork@gmx.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4zI0WCX1RcnW9Hbu" Content-Disposition: inline In-Reply-To: <200807071615.40987.nettwork@gmx.de> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-net@freebsd.org Subject: Re: smbmount / smbclient : strangely varying transfer speeds X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 03:32:54 -0000 --4zI0WCX1RcnW9Hbu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2008-Jul-07 16:15:40 +0000, Achim wrote: >Performance with a single client is degraded when the client is smbmount a= nd=20 >downloading.=20 >With a second transfer in any direction, performance becomes better, to ab= out=20 >3.5 resp. 8 MB/s depending on the second connection up- or downloading. >Unlike smbmount, single smbclient transfers yield acceptable results. Is this two transfers between a single server and single client or between a single server and two clients? In the former case, you might like to try a transfers involving two clients or two servers to try and identify which end is behaving oddly. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --4zI0WCX1RcnW9Hbu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkhy39cACgkQ/opHv/APuIeTEgCfcX1YOQSw1SNHOq/0bUUqQOMt arwAnAic7g028Mq+eBzzXb2X7ZLfTD03 =7a52 -----END PGP SIGNATURE----- --4zI0WCX1RcnW9Hbu-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 07:27:17 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 487E41065676 for ; Tue, 8 Jul 2008 07:27:17 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.230]) by mx1.freebsd.org (Postfix) with ESMTP id 16BBA8FC17 for ; Tue, 8 Jul 2008 07:27:11 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by rv-out-0506.google.com with SMTP id b25so3152631rvf.43 for ; Tue, 08 Jul 2008 00:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=m+c/e9RB/LQOcG9/dAcUYmsbtnNetLVQIg/vnzajRBE=; b=kIeNAMAB4m6Ba1/AJkn0RmbvGHv/1V5ULLalvbfhQGAdZN9hzVLm010i38RHaNuGjA AV/tyUDoCLMrQ1Dktz+MQlj4rFf5VTZO3NsBLZBawfsXdOUyJHpNm2bn6J+UtbLSXKnO zvZ4m7BnQEDd4pp8+qV3fxs2LZ3DBpwXmzs08= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=WB2ZHybcifW+uucfn1I9qiu+ER0k36oS3MhKeZ+Wk34P+XySWt6ZuW5Ncoa/SpwLen lflV/9f0ncwB0VOTLjgeCQtESI6SAVXEkpXL3S+ak3SLVx6CtPJKtlkTFFXg6DMwzICQ AfEPfN0UgUXmfYxzu32lCEq0w3ywyAX3DzMZU= Received: by 10.114.159.6 with SMTP id h6mr7047433wae.65.1215500528548; Tue, 08 Jul 2008 00:02:08 -0700 (PDT) Received: by 10.114.59.4 with HTTP; Tue, 8 Jul 2008 00:02:08 -0700 (PDT) Message-ID: Date: Tue, 8 Jul 2008 00:02:08 -0700 From: "Kip Macy" To: "Mike Tancsa" In-Reply-To: <200807080107.m6817XxO021966@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> Cc: FreeBSD Net , Andre Oppermann , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 07:27:17 -0000 On Mon, Jul 7, 2008 at 6:07 PM, Mike Tancsa wrote: > At 02:44 PM 7/7/2008, Paul wrote: > >> Also my 82571 NIC supports multiple received queues and multiple transmit >> queues so why hasn't >> anyone written the driver to support this? It's not a 10gb card and it >> still supports it and it's widely >> available and not too expensive either. The new 82575/6 chips support >> even more queues and the >> two port version will be out this month and the 4 port in october (PCI-E >> cards). Motherboards are >> already shipping with the 82576.. (82571 supports 2x/2x 575/6 support >> 4x/4x) > > > > > Actually, do any of your NICs attach via the igb driver ? > I have a pre-production card. With some bug fixes and some tuning of interrupt handling (custom stack - I've been asked to push the changes back in to CVS, I just don't have time right now) an otherwise unoptimized igb can forward 1.04Mpps from one port to another (1.04 Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 core system. -Kip From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 07:32:10 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6D3C106566C for ; Tue, 8 Jul 2008 07:32:10 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 951078FC1F for ; Tue, 8 Jul 2008 07:32:10 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wa-out-1112.google.com with SMTP id j4so1420237wah.3 for ; Tue, 08 Jul 2008 00:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=iYu0Pqecgdlg+gN8h2v4SEKbdC9un7VwJ6OJoV+ACv0=; b=K45s6wLSenCsRijR6pBGydvkBZ8kfcoHkapz/arZ6ng4JhYOrL4nQ5l/gXVVkw7vQt CUt+suR+/FWy0R083REYJihRR83NIMEyomFpGrXng3uMt4SP7C/mchy2cKHqd/Q+GUQ9 pGmz9Ct+gjLAtgfp89SqvZlB5Tjao9+t+shD4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=EbeJkGeVGQcYUd0LtaHJgEKPTKcetqqgNA+Q3rHfyXarR99R4oRzM7U1FrZUmiAHnP f0Mwkdr5OXi9JRLUexACHzlDrhlslZDVDRy4Idafkbh33Pq1l3BCZ6oiBbgkY1BPLkGl E5xAoAYusjQTbw95nGC23R20pMT2EL2EedkhA= Received: by 10.114.147.7 with SMTP id u7mr7050220wad.188.1215500670925; Tue, 08 Jul 2008 00:04:30 -0700 (PDT) Received: by 10.114.59.4 with HTTP; Tue, 8 Jul 2008 00:04:30 -0700 (PDT) Message-ID: Date: Tue, 8 Jul 2008 00:04:30 -0700 From: "Kip Macy" To: Paul In-Reply-To: <4872C161.6080105@gtcomm.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <4872C161.6080105@gtcomm.net> Cc: FreeBSD Net , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 07:32:10 -0000 On Mon, Jul 7, 2008 at 6:22 PM, Paul wrote: > I read through the IGB driver, and it says 82575/6 only... which is the new > chip Intel is releasing on the cards this month 2 port > and october 4 port, but the chips are on some of the motherboards right now. > Why can't it also use the 82571 ? doesn't make any sense.. I haven't tried > it but just browsing the driver source > doesn't look like it will work. The igb driver has been written to remove a lot of the cruft that has accumulated to work around deficiencies in earlier 8257x hardware. Although it supports "legacy" descriptor handling it has a new mode of descriptor handling that is ostensibly better. I don't have access to the data sheets for pre-zoar hardware so I'm not sure what it would take to support multiple queues on that hardware. -Kip From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 07:54:45 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E46551065671 for ; Tue, 8 Jul 2008 07:54:45 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id B63038FC18 for ; Tue, 8 Jul 2008 07:54:45 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 4358346B8C; Tue, 8 Jul 2008 03:54:45 -0400 (EDT) Date: Tue, 8 Jul 2008 08:54:45 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Artem Belevich In-Reply-To: Message-ID: <20080708085227.J31157@fledge.watson.org> References: <4867420D.7090406@gtcomm.net> <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 07:54:46 -0000 On Mon, 7 Jul 2008, Artem Belevich wrote: > As was already mentioned, we can't avoid all cache misses as there's data > that's recently been updated in memory via DMA and therefor kicked out of > cache. > > However, we may hide some of the latency penalty by prefetching > 'interesting' data early. I.e. we know that we want to access some ethernet > headers, so we may start pulling relevant data into cache early. Ideally, by > the time we need to access the field, it will already be in the cache. When > we're counting nanoseconds per packet this may bring some performance gain. There were some patches floating around for if_em to do a prefetch of the first bit of packet data on packets before handing them up the stack. My understanding is that they moved the hot spot earlier, but didn't make a huge difference because it doesn't really take that long to get to the point where you're processing the IP header in our current stack (a downside to optimization...). However, that's a pretty anecdotal story, and a proper study of the effects of prefetching would be most welcome. One thing that I'd really like to see someone look at is whether, by doing a bit of appropriately timed prefetching, we can move cache misses out from under hot locks that don't really relate to the data being prefetched. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 08:15:46 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07466106567D for ; Tue, 8 Jul 2008 08:15:45 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id 97F648FC22 for ; Tue, 8 Jul 2008 08:15:45 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 0C65D1B10EDC; Tue, 8 Jul 2008 10:15:42 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on malcho.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.6 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.2.4 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 9A7BB1B10EE0; Tue, 8 Jul 2008 10:15:39 +0200 (CEST) Message-ID: <4873222B.4080907@moneybookers.com> Date: Tue, 08 Jul 2008 11:15:39 +0300 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.14 (X11/20080616) MIME-Version: 1.0 To: Kip Macy References: <4867420D.7090406@gtcomm.net> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> In-Reply-To: Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.93, clamav-milter version 0.93 on blah.cmotd.com X-Virus-Status: Clean Cc: FreeBSD Net , Paul , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 08:15:46 -0000 Hi, Kip Macy wrote: > On Mon, Jul 7, 2008 at 6:07 PM, Mike Tancsa wrote: > >> At 02:44 PM 7/7/2008, Paul wrote: >> >> >>> Also my 82571 NIC supports multiple received queues and multiple transmit >>> queues so why hasn't >>> anyone written the driver to support this? It's not a 10gb card and it >>> still supports it and it's widely >>> available and not too expensive either. The new 82575/6 chips support >>> even more queues and the >>> two port version will be out this month and the 4 port in october (PCI-E >>> cards). Motherboards are >>> already shipping with the 82576.. (82571 supports 2x/2x 575/6 support >>> 4x/4x) >>> >> >> >> Actually, do any of your NICs attach via the igb driver ? >> >> > > I have a pre-production card. With some bug fixes and some tuning of > interrupt handling (custom stack - I've been asked to push the changes > back in to CVS, I just don't have time right now) an otherwise > unoptimized igb can forward 1.04Mpps from one port to another (1.04 > Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 > core system. > > Is this on 1gbps or on 10gbps NIC? > -Kip > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Best Wishes, Stefan Lambrev ICQ# 24134177 From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 08:19:11 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 881251065674 for ; Tue, 8 Jul 2008 08:19:11 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 54BA28FC26 for ; Tue, 8 Jul 2008 08:19:11 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wa-out-1112.google.com with SMTP id j4so1426069wah.3 for ; Tue, 08 Jul 2008 01:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=rwvTMbwkhSz+oBLAQAtb977c8ddpuxYfhMzG01dFdgA=; b=PwJlCJ9pP1Hgvj8Y19kvZG632jHWj8H63yEkN8RrwMRVbff0y/k0I+NXxCSfFQmaGB pGummu+H54bSI9w2785EIapMVbhZoDzmfizYyGIXppnQuSB4URL2gtak0ApsAb0m1qm3 pMKi7IKKy2gapPksfBeiWdrnHpPi9zD/0Slcs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=SNG/l9uDRShaUEOE9RgecUtguTVQn38FfeqoAJAvvcdId6rT4SzXNwBg0qmmCrLt8a wBGxS6tSjyDMekhWzgsotBDP2TbANSRGrwPY8q6QcJKnI4u5kzv24IiQ1BaOyeXuMnTM mjIVf9alHKVrtJQl5BAcfksqF34rifVtZL5cg= Received: by 10.114.66.2 with SMTP id o2mr7461462waa.124.1215505151101; Tue, 08 Jul 2008 01:19:11 -0700 (PDT) Received: by 10.114.59.4 with HTTP; Tue, 8 Jul 2008 01:19:10 -0700 (PDT) Message-ID: Date: Tue, 8 Jul 2008 01:19:10 -0700 From: "Kip Macy" To: "Stefan Lambrev" In-Reply-To: <4873222B.4080907@moneybookers.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <4873222B.4080907@moneybookers.com> Cc: FreeBSD Net , Paul , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 08:19:11 -0000 >> I have a pre-production card. With some bug fixes and some tuning of >> interrupt handling (custom stack - I've been asked to push the changes >> back in to CVS, I just don't have time right now) an otherwise >> unoptimized igb can forward 1.04Mpps from one port to another (1.04 >> Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 >> core system. >> >> > > Is this on 1gbps or on 10gbps NIC? >> Hi Stefan, The hardware that igb supports is just the latest revision of the hardware supported by em, i.e. it is 1gbps. Cheers, Kip From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 08:26:43 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F21E1065682; Tue, 8 Jul 2008 08:26:43 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 40DDB8FC1D; Tue, 8 Jul 2008 08:26:43 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KG8T6-00074M-QC; Tue, 08 Jul 2008 04:22:36 -0400 Message-ID: <4873253D.3070707@gtcomm.net> Date: Tue, 08 Jul 2008 04:28:45 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Kip Macy References: <4867420D.7090406@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <4873222B.4080907@moneybookers.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Mike Tancsa , Andre Oppermann , Stefan Lambrev Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 08:26:43 -0000 Will someone confirm if it will support the 82571EB ? I don't see a reason why not as it's very similar hardware and it's available now in large quantities so making 82571 part of igb I think would be a good idea. Kip Macy wrote: >>> I have a pre-production card. With some bug fixes and some tuning of >>> interrupt handling (custom stack - I've been asked to push the changes >>> back in to CVS, I just don't have time right now) an otherwise >>> unoptimized igb can forward 1.04Mpps from one port to another (1.04 >>> Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 >>> core system. >>> >>> >>> >> Is this on 1gbps or on 10gbps NIC? >> > > Hi Stefan, > The hardware that igb supports is just the latest revision of the > hardware supported by em, i.e. it is 1gbps. > > Cheers, > Kip > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 11:08:44 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4803106566C for ; Tue, 8 Jul 2008 11:08:44 +0000 (UTC) (envelope-from joe.kuan@itrinegy.com) Received: from mail.itrinegy.com (host81-143-90-145.in-addr.btopenworld.com [81.143.90.145]) by mx1.freebsd.org (Postfix) with ESMTP id 728198FC1C for ; Tue, 8 Jul 2008 11:08:44 +0000 (UTC) (envelope-from joe.kuan@itrinegy.com) Received: from [192.168.200.194] (192.168.200.194) by insvr01 (192.168.200.1) with Microsoft SMTP Server id 8.1.263.0; Tue, 8 Jul 2008 11:54:59 +0100 Message-ID: From: Joe Kuan To: Content-Type: text/plain; charset="US-ASCII"; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit MIME-Version: 1.0 (Apple Message framework v926) Date: Tue, 8 Jul 2008 11:57:49 +0100 X-Mailer: Apple Mail (2.926) Subject: Help: FreeBSD 6.3 - em driver & taskqueue & priority X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 11:08:45 -0000 Hi all, I have implemented an network application in kernel space and it is working fine. The application involves 3 network interfaces that FreeBSD 6.3 can forward mbuf between em0 and em1 in a rate 1.3 - 1.4 millions packets per second. Em2 is used for controlling the network application. The problem is that when em0 and em1 are transmitting in 1.3 - 1.4 millions packets per second, the em2 interface becomes irresponsive. However, my goal is to make the kernelised network application response as soon as a control packet arrives in em2, ie jumps the queue ahead of all the packets in em0 and em1. I think the problem lies on the priority set on the task structure are all the same for all the em devices. Am I heading in the right direction? If not, please advise me. Many thanks in advance Joe From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 11:47:52 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA24B106567A; Tue, 8 Jul 2008 11:47:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id 35BBC8FC1A; Tue, 8 Jul 2008 11:47:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m68Blj69028513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Jul 2008 21:47:47 +1000 Date: Tue, 8 Jul 2008 21:47:22 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Erik Trulsson In-Reply-To: <20080707131550.GA69202@owl.midgard.homeip.net> Message-ID: <20080708214624.W1168@besplex.bde.org> References: <486D35A0.4000302@gtcomm.net> <486DF1A3.9000409@gtcomm.net> <486E65E6.3060301@gtcomm.net> <4871DB8E.5070903@freebsd.org> <20080707191918.B4703@besplex.bde.org> <4871FB66.1060406@freebsd.org> <20080707213356.G7572@besplex.bde.org> <20080707131550.GA69202@owl.midgard.homeip.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Andre Oppermann , Ingo Flaschberger , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 11:47:52 -0000 On Mon, 7 Jul 2008, Erik Trulsson wrote: > On Mon, Jul 07, 2008 at 10:30:53PM +1000, Bruce Evans wrote: >> On Mon, 7 Jul 2008, Andre Oppermann wrote: >>> The theoretical maximum at 64byte frames is 1,488,100. I've looked >>> up my notes the 1.244Mpps number can be ajusted to 1.488Mpps. >> >> Where is the extra? I still get 1.644736 Mpps (10^9/(8*64+96)). >> 1.488095 is for 64 bits extra (10^9/(8*64+96+64)). > > A standard ethernet frame (on the wire) consists of: > 7 octets preamble > 1 octet Start Frame Delimiter > 6 octets destination address > 6 octets source address > 2 octets length/type > 46-1500 octets data (+padding if needed) > 4 octets Frame Check Sequence > > Followed by (at least) 96 bits interFrameGap, before the next frame starts. > > For minimal packet size this gives a maximum packet rate at 1Gbit/s of > 1e9/((7+1+6+6+2+46+4)*8+96)/ = 1488095 packets/second > > You probably missed the preamble and start frame delimiter in your > calculation. Thanks. Yes, that was it. Bruce From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 12:23:44 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 639351065686 for ; Tue, 8 Jul 2008 12:23:44 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id DB2328FC17 for ; Tue, 8 Jul 2008 12:23:43 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1KGBUE-0002wk-NX for freebsd-net@freebsd.org; Tue, 08 Jul 2008 11:35:58 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jul 2008 11:35:58 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jul 2008 11:35:58 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Ivan Voras Date: Tue, 08 Jul 2008 13:35:51 +0200 Lines: 46 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig0B4523A9C71CF936962836C0" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.14 (X11/20080505) In-Reply-To: X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Help: FreeBSD 6.3 - em driver & taskqueue & priority X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 12:23:44 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig0B4523A9C71CF936962836C0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Joe Kuan wrote: > Hi all, >=20 > I have implemented an network application in kernel space and it is > working fine. The application involves 3 network interfaces that FreeBS= D > 6.3 can forward mbuf between em0 and em1 in a rate 1.3 - 1.4 millions > packets per second. Em2 is used for controlling the network application= =2E >=20 > The problem is that when em0 and em1 are transmitting in 1.3 - 1.4 > millions packets per second, the em2 interface becomes irresponsive. > However, my goal is to make the kernelised network application response= > as soon as a control packet arrives in em2, ie jumps the queue ahead of= > all the packets in em0 and em1. >=20 > I think the problem lies on the priority set on the task structure > are all the same for all the em devices. Am I heading in the right > direction? A wild theory: are the NICs separate / individual and on separate buses? If they are not (e.g. two of them are on the same card or bus) it might be a hardware issue. --------------enig0B4523A9C71CF936962836C0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIc1EXldnAQVacBcgRAunlAKCUzOp1kvOnLws/mcrekwubfdTwwwCePerx myAhcDJPNzytmw4BuEuEMbY= =lehA -----END PGP SIGNATURE----- --------------enig0B4523A9C71CF936962836C0-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 12:26:38 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6D7A106566C for ; Tue, 8 Jul 2008 12:26:38 +0000 (UTC) (envelope-from joe.kuan@itrinegy.com) Received: from mail.itrinegy.com (host81-143-90-145.in-addr.btopenworld.com [81.143.90.145]) by mx1.freebsd.org (Postfix) with ESMTP id 4E1AC8FC23 for ; Tue, 8 Jul 2008 12:26:38 +0000 (UTC) (envelope-from joe.kuan@itrinegy.com) Received: from [192.168.200.194] (192.168.200.194) by insvr01 (192.168.200.1) with Microsoft SMTP Server id 8.1.263.0; Tue, 8 Jul 2008 13:23:46 +0100 Message-ID: <03FF3E77-8B0F-4AF2-91D5-F0062DD8C6A2@itrinegy.com> From: Joe Kuan To: Ivan Voras In-Reply-To: Content-Type: text/plain; charset="US-ASCII"; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit MIME-Version: 1.0 (Apple Message framework v926) Date: Tue, 8 Jul 2008 13:26:35 +0100 References: X-Mailer: Apple Mail (2.926) Cc: "freebsd-net@freebsd.org" Subject: Re: Help: FreeBSD 6.3 - em driver & taskqueue & priority X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 12:26:38 -0000 On 8 Jul 2008, at 12:35, Ivan Voras wrote: > Joe Kuan wrote: >> Hi all, >> >> I have implemented an network application in kernel space and it is >> working fine. The application involves 3 network interfaces that >> FreeBSD >> 6.3 can forward mbuf between em0 and em1 in a rate 1.3 - 1.4 millions >> packets per second. Em2 is used for controlling the network >> application. >> >> The problem is that when em0 and em1 are transmitting in 1.3 - 1.4 >> millions packets per second, the em2 interface becomes irresponsive. >> However, my goal is to make the kernelised network application >> response >> as soon as a control packet arrives in em2, ie jumps the queue >> ahead of >> all the packets in em0 and em1. >> >> I think the problem lies on the priority set on the task structure >> are all the same for all the em devices. Am I heading in the right >> direction? > > A wild theory: are the NICs separate / individual and on separate > buses? > If they are not (e.g. two of them are on the same card or bus) it > might > be a hardware issue. > em0 and em1 are on the same card. Em2 is on the separate card. Thanks Joe From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 13:16:24 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8E4F1065674 for ; Tue, 8 Jul 2008 13:16:24 +0000 (UTC) (envelope-from nettwork@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 19E3D8FC2E for ; Tue, 8 Jul 2008 13:16:23 +0000 (UTC) (envelope-from nettwork@gmx.de) Received: (qmail invoked by alias); 08 Jul 2008 13:16:21 -0000 Received: from e178213225.adsl.alicedsl.de (HELO [10.0.0.100]) [85.178.213.225] by mail.gmx.net (mp041) with SMTP; 08 Jul 2008 15:16:21 +0200 X-Authenticated: #46460896 X-Provags-ID: V01U2FsdGVkX18HPkp6iFAHOj/S89rBNJ2mgEwJxs/V7HyPbdXelp hVhrfUWX7xccKw From: Achim To: freebsd-net@freebsd.org Date: Tue, 8 Jul 2008 13:16:52 +0000 User-Agent: KMail/1.9.7 References: <200807071615.40987.nettwork@gmx.de> <20080708033239.GL62764@server.vk2pj.dyndns.org> In-Reply-To: <20080708033239.GL62764@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200807081316.52482.nettwork@gmx.de> X-Y-GMX-Trusted: 0 X-FuHaFi: 0.64 Subject: Re: smbmount / smbclient : strangely varying transfer speeds X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 13:16:24 -0000 On Tuesday 08 July 2008 03:32:40 Peter Jeremy wrote: > On 2008-Jul-07 16:15:40 +0000, Achim wrote: > >Performance with a single client is degraded when the client is smbmount > > and downloading. > >With a second transfer in any direction, performance becomes better, to > > about 3.5 resp. 8 MB/s depending on the second connection up- or > > downloading. Unlike smbmount, single smbclient transfers yield acceptable > > results. > > Is this two transfers between a single server and single client or > between a single server and two clients? In the former case, you > might like to try a transfers involving two clients or two servers > to try and identify which end is behaving oddly. Splendid Idea - As of yet, all test cases indeed were the same two physical machines, however if the client is a FBSD7 instead of the kubuntu one, comparable results are showing, so i suspect the server (FBSD7) to be the source of the oddness. I will try your suggestion as soon as the FBSD(Notebook) is back, should be tonight. Furthermore, there is a FBSD Dualboot on the kubuntu machine that i will test later to see if it shows the same behaviour. Funnily enough, if the second transfer is a very small-bandwidth one, like watching a movie instead of transferring it, the positive effect on the first transfer is also a LOT smaller than with a concurrent high-bandwidth access. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 13:40:19 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39BFB1065674; Tue, 8 Jul 2008 13:40:19 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id D7E418FC24; Tue, 8 Jul 2008 13:40:18 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 8B8471B10EFC; Tue, 8 Jul 2008 15:40:17 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on malcho.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.6 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.2.4 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 231571B10ED2; Tue, 8 Jul 2008 15:40:15 +0200 (CEST) Message-ID: <48736E3E.6090200@moneybookers.com> Date: Tue, 08 Jul 2008 16:40:14 +0300 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.14 (X11/20080616) MIME-Version: 1.0 To: Kip Macy References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <4872C161.6080105@gtcomm.net> In-Reply-To: Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.93, clamav-milter version 0.93 on blah.cmotd.com X-Virus-Status: Clean Cc: FreeBSD Net , Mike Tancsa , Andre Oppermann , Jack Vogel , Paul Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 13:40:19 -0000 Hi, Kip Macy wrote: > On Mon, Jul 7, 2008 at 6:22 PM, Paul wrote: > >> I read through the IGB driver, and it says 82575/6 only... which is the new >> chip Intel is releasing on the cards this month 2 port >> and october 4 port, but the chips are on some of the motherboards right now. >> Why can't it also use the 82571 ? doesn't make any sense.. I haven't tried >> it but just browsing the driver source >> doesn't look like it will work. >> > > The igb driver has been written to remove a lot of the cruft that has > accumulated to work around deficiencies in earlier 8257x hardware. > Although it supports "legacy" descriptor handling it has a new mode of > descriptor handling that is ostensibly better. I don't have access to > the data sheets for pre-zoar hardware so I'm not sure what it would > take to support multiple queues on that hardware. > May be we should ask Jack Vogel? He will have some news probably. > -Kip > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Best Wishes, Stefan Lambrev ICQ# 24134177 From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 16:34:50 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3CA261065678 for ; Tue, 8 Jul 2008 16:34:50 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id B37D18FC20 for ; Tue, 8 Jul 2008 16:34:49 +0000 (UTC) (envelope-from artemb@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1416235fgb.35 for ; Tue, 08 Jul 2008 09:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=RY55Q2hmv0SnI445CM2qjgOXbNCneXIPNRw4FrxRzvE=; b=l83cC7MeBHjTtagBAR9mS2EKoyMySANrCzJyZmd0lk2g911W80yLJVK2DAi6pUwCcx he0Hx90rcVf5vBrP/rP6bCTanVeN0Y8tg8r+aHM6hStlaxNCNNwk6zubRynNuQnVEIxA VIISINgxk9+T9vGFop3oXOQH15O05NTynTg0k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=PzWJru1DJ6yVRGXHbUVgoF7b1+8uppsGcCbDsYmuJZnZY+GWbUerWVKG35aBz98A6y JPxz6viCTfe52g6RD2miI7/NYW6O7MejP/YRCMYman5PwJrh5w+qdNV0PJvn0zKXRmaM mOAy/mOtzCGa3HLMeCXoyoKYU2LtfQcbMxEbk= Received: by 10.86.91.12 with SMTP id o12mr5754710fgb.1.1215534888241; Tue, 08 Jul 2008 09:34:48 -0700 (PDT) Received: by 10.86.83.19 with HTTP; Tue, 8 Jul 2008 09:34:48 -0700 (PDT) Message-ID: Date: Tue, 8 Jul 2008 09:34:48 -0700 From: "Artem Belevich" Sender: artemb@gmail.com To: "Robert Watson" In-Reply-To: <20080708085227.J31157@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <20080708085227.J31157@fledge.watson.org> X-Google-Sender-Auth: 40aab35622524a80 Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 16:34:50 -0000 On 7/8/08, Robert Watson wrote: > There were some patches floating around for if_em to do a prefetch of the > first bit of packet data on packets before handing them up the stack. My I found Andre Oppermann's optimization patch mentioned in july 2005 status report: http://lists.freebsd.org/pipermail/freebsd-announce/2005-July/001012.html http://www.nrg4u.com/freebsd/tcp_reass+prefetch-20041216.patch Is that the patch you had in mind? In the report Andre says: "Use [of prefetch] in both of these places show a very significant performance gain but not yet fully quantified." "very significant" bit looks promising. Unfortunately, it does not look like prefetch changes in the patch made it into official kernel. I wonder why. It should be easy enough to apply prefetch-related changes and see if/how it affects forwarding performance. --Artem From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 18:50:04 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 767D71065672 for ; Tue, 8 Jul 2008 18:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 62C118FC24 for ; Tue, 8 Jul 2008 18:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m68Io4Va087508 for ; Tue, 8 Jul 2008 18:50:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m68Io4qq087507; Tue, 8 Jul 2008 18:50:04 GMT (envelope-from gnats) Date: Tue, 8 Jul 2008 18:50:04 GMT Message-Id: <200807081850.m68Io4qq087507@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Francis Gendreau Cc: Subject: Re: kern/125195: verbrose dmesg from asus m3000n m3n as requested by Gavin Atkinson X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Francis Gendreau List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 18:50:04 -0000 The following reply was made to PR kern/125195; it has been noted by GNATS. From: Francis Gendreau To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/125195: verbrose dmesg from asus m3000n m3n as requested by Gavin Atkinson Date: Tue, 08 Jul 2008 11:33:22 -0400 verbose dmesg after hard power cycle: Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries Data TLB: 4 KB Pages, 4-way set associative, 128 entries Instruction TLB: 4 MB pages, fully associative, 2 entries 2nd-level cache: 1 MB, 8-way set associative, 64 byte line size 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size Data TLB: 4 MB Pages, 4-way set associative, 8 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size real memory = 527695872 (503 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009efff, 647168 bytes (158 pages) 0x0000000000100000 - 0x00000000003fffff, 3145728 bytes (768 pages) 0x0000000001028000 - 0x000000001ee22fff, 501198848 bytes (122363 pages) avail memory = 502419456 (479 MB) Table 'FACP' at 0x1f740200 Table 'OEMB' at 0x1f750040 MADT: No MADT table found APIC: Could not find any APICs. pnpbios: Found PnP BIOS data at 0xc00f2e00 pnpbios: Entry = f0000:39da Rev = 1.0 Other BIOS signatures found: wlan_amrr: wlan: <802.11 Link Layer> firmware: 'ipw_bss' version 130: 209190 bytes loaded at 0xc0d68738 firmware: 'ipw_ibss' version 130: 201138 bytes loaded at 0xc0d9d73c firmware: 'ipw_monitor' version 130: 196458 bytes loaded at 0xc0dd0748 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_buffersize=16384 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 ath_rate: version 1.2 nfslock: pseudo-device kbd: new array size 4 kbd1 at kbdmux0 io: mem: Pentium Pro MTRR support enabled null: random: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 19:59:27) ACPI: RSDP @ 0x0xf4b70/0x0014 (v 0 ACPIAM) ACPI: RSDT @ 0x0x1f740000/0x002C (v 1 A M I OEMRSDT 0x05000314 MSFT 0x00000097) ACPI: FACP @ 0x0x1f740200/0x0081 (v 2 A M I OEMFACP 0x05000314 MSFT 0x00000097) ACPI: DSDT @ 0x0x1f740300/0x7323 (v 1 0ABBD 0ABBD001 0x00000001 MSFT 0x0100000D) ACPI: FACS @ 0x0x1f750000/0x0040 ACPI: OEMB @ 0x0x1f750040/0x004D (v 1 A M I OEMBIOS 0x05000314 MSFT 0x00000097) npx0: INT 16 interface acpi0: on motherboard acpi0: [MPSAFE] acpi0: [ITHREAD] pci_open(1): mode 1 addr port (0x0cf8) is 0x8000005c pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=35808086) pcibios: No call entry point AcpiOsDerivePciId: \\_SB_.PCI0.P0P1.CBS0.CBSP -> bus 1 dev 5 func 0 acpi0: Power Button (fixed) acpi0: wakeup code va 0xccd3f000 pa 0x1000 atpic: Programming IRQ9 as level/low AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IROR -> bus 0 dev 31 func 0 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 1f700000 (3) failed ACPI timer: 1/1 1/1 1/0 1/1 1/1 1/1 1/0 1/1 1/1 1/1 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 5 6 7 11 12 Validation 0 11 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 11 12 Validation 0 255 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 12 Validation 0 4 N 0 4 12 After Disable 0 255 N 0 4 12 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 6 Validation 0 5 N 0 5 6 After Disable 0 255 N 0 5 6 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 6 11 Validation 0 11 N 0 6 11 After Disable 0 255 N 0 6 11 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 7 Validation 0 255 N 0 3 7 After Disable 0 255 N 0 3 7 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 4 7 Validation 0 255 N 0 4 7 After Disable 0 255 N 0 4 7 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 6 12 Validation 0 4 N 0 4 6 12 After Disable 0 255 N 0 4 6 12 cpu0: on acpi0 cpu0: switching to generic Cx mode est0: on cpu0 p4tcc0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 ACPI: Found matching pin for 0.2.INTA at func 0: 11 ACPI: Found matching pin for 0.31.INTA at func 1: 255 ACPI: Found matching pin for 0.31.INTB at func 5: 255 ACPI: Found matching pin for 0.31.INTB at func 6: 255 ACPI: Found matching pin for 0.29.INTA at func 0: 11 ACPI: Found matching pin for 0.29.INTB at func 1: 5 ACPI: Found matching pin for 0.29.INTC at func 2: 4 ACPI: Found matching pin for 0.29.INTD at func 7: 4 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x3580, revid=0x02 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3584, revid=0x02 domain=0, bus=0, slot=0, func=1 class=08-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3585, revid=0x02 domain=0, bus=0, slot=0, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 27, enabled map[14]: type Memory, range 32, base 0xffa80000, size 19, enabled map[18]: type I/O Port, range 32, base 0xdc00, size 3, enabled pcib0: matched entry for 0.2.INTA (src \\_SB_.LNKA:0) pcib0: slot 2 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=1 class=03-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xe8000000, size 27, enabled map[14]: type Memory, range 32, base 0xff980000, size 19, enabled found-> vendor=0x8086, dev=0x24c2, revid=0x03 domain=0, bus=0, slot=29, func=0 class=0c-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 map[20]: type I/O Port, range 32, base 0xd480, size 5, enabled pcib0: matched entry for 0.29.INTA (src \\_SB_.LNKA:0) pcib0: slot 29 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x24c4, revid=0x03 domain=0, bus=0, slot=29, func=1 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 map[20]: type I/O Port, range 32, base 0xd800, size 5, enabled pcib0: matched entry for 0.29.INTB (src \\_SB_.LNKD:0) pcib0: slot 29 INTB routed to irq 5 via \\_SB_.LNKD found-> vendor=0x8086, dev=0x24c7, revid=0x03 domain=0, bus=0, slot=29, func=2 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=4 map[20]: type I/O Port, range 32, base 0xd880, size 5, enabled pcib0: matched entry for 0.29.INTC (src \\_SB_.LNKC:0) pcib0: slot 29 INTC routed to irq 4 via \\_SB_.LNKC found-> vendor=0x8086, dev=0x24cd, revid=0x03 domain=0, bus=0, slot=29, func=7 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=d, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xffa7fc00, size 10, enabled pcib0: matched entry for 0.29.INTD (src \\_SB_.LNKH:0) pcib0: slot 29 INTD routed to irq 4 via \\_SB_.LNKH found-> vendor=0x8086, dev=0x2448, revid=0x83 domain=0, bus=0, slot=30, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x8080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x06 (1500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24cc, revid=0x03 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24ca, revid=0x03 domain=0, bus=0, slot=31, func=1 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled map[24]: type Memory, range 32, base 0, size 10, memory disabled found-> vendor=0x8086, dev=0x24c5, revid=0x03 domain=0, bus=0, slot=31, func=5 class=04-01-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe000, size 8, enabled map[14]: type I/O Port, range 32, base 0xe100, size 6, enabled map[18]: type Memory, range 32, base 0, size 9, memory disabled map[1c]: type Memory, range 32, base 0, size 8, memory disabled found-> vendor=0x8086, dev=0x24c6, revid=0x03 domain=0, bus=0, slot=31, func=6 class=07-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe200, size 8, enabled map[14]: type I/O Port, range 32, base 0xe300, size 7, enabled pci0: at device 0.1 (no driver attached) pci0: at device 0.3 (no driver attached) vgapci0: port 0xdc00-0xdc07 mem 0xf0000000-0xf7ffffff,0xffa80000-0xffafffff irq 11 at device 2.0 on pci0 agp0: on vgapci0 vgapci0: Reserved 0x8000000 bytes for rid 0x10 type 3 at 0xf0000000 vgapci0: Reserved 0x80000 bytes for rid 0x14 type 3 at 0xffa80000 agp0: detected 8060k stolen memory agp0: aperture size is 128M vgapci1: mem 0xe8000000-0xefffffff,0xff980000-0xff9fffff at device 2.1 on pci0 uhci0: port 0xd480-0xd49f irq 11 at device 29.0 on pci0 uhci0: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd480 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd800-0xd81f irq 5 at device 29.1 on pci0 uhci1: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd800 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xd880-0xd89f irq 4 at device 29.2 on pci0 uhci2: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd880 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xffa7fc00-0xffa7ffff irq 4 at device 29.7 on pci0 ehci0: Reserved 0x400 bytes for rid 0x10 type 3 at 0xffa7fc00 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered pcib1: at device 30.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xc000-0xcfff pcib1: memory decode 0xff700000-0xff7fffff pcib1: prefetched decode 0xdea00000-0xdeafffff pcib1: Subtractively decoded bridge. ACPI: Found matching pin for 1.8.INTA at func 0: 11 ACPI: Found matching pin for 1.5.INTA at func 0: 255 ACPI: Found matching pin for 1.5.INTB at func 1: 11 ACPI: Found matching pin for 1.4.INTA at func 0: 4 pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x8086, dev=0x1043, revid=0x04 domain=0, bus=1, slot=4, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0116, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x22 (8500 ns) intpin=a, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fd000, size 12, enabled pcib1: requested memory range 0xff7fd000-0xff7fdfff: good pcib1: matched entry for 1.4.INTA (src \\_SB_.LNKC:0) pcib1: slot 4 INTA routed to irq 4 via \\_SB_.LNKC found-> vendor=0x1180, dev=0x0475, revid=0xb8 domain=0, bus=1, slot=5, func=0 class=06-07-00, hdrtype=0x02, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x80 (32000 ns), maxlat=0x07 (1750 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0, size 12, enabled found-> vendor=0x1180, dev=0x0551, revid=0x00 domain=0, bus=1, slot=5, func=1 class=0c-00-10, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x04 (1000 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fe800, size 11, enabled pcib1: requested memory range 0xff7fe800-0xff7fefff: good pcib1: matched entry for 1.5.INTB (src \\_SB_.LNKA:0) pcib1: slot 5 INTB routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x103e, revid=0x83 domain=0, bus=1, slot=8, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0117, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xff7ff000, size 12, enabled pcib1: requested memory range 0xff7ff000-0xff7fffff: good map[14]: type I/O Port, range 32, base 0xcc00, size 6, enabled pcib1: requested I/O range 0xcc00-0xcc3f: in range pcib1: matched entry for 1.8.INTA (src \\_SB_.LNKE:0) pcib1: slot 8 INTA routed to irq 11 via \\_SB_.LNKE ipw0: mem 0xff7fd000-0xff7fdfff irq 4 at device 4.0 on pci1 ipw0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7fd000 ipw0: bpf attached ipw0: Ethernet address: 00:04:23:71:77:46 ipw0: bpf attached ipw0: bpf attached ipw0: [MPSAFE] ipw0: [ITHREAD] ipw0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps cbb0: at device 5.0 on pci1 pcib1: cbb0 requested memory range 0xff700000-0xff7fffff: good cbb0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xff700000 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pcib1: matched entry for 1.5.INTA (src \\_SB_.LNKB:0) pci_link1: Picked IRQ 9 with weight 0 pcib1: slot 5 INTA routed to irq 9 via \\_SB_.LNKB cbb0: [MPSAFE] cbb0: [ITHREAD] cbb0: PCI Configuration space: 0x00: 0x04751180 0x02100007 0x060700b8 0x00822000 0x10: 0xff700000 0x020000dc 0x20030201 0xfffff000 0x20: 0x00000000 0xfffff000 0x00000000 0xfffffffc 0x30: 0x00000000 0xfffffffc 0x00000000 0x07000109 0x40: 0x17441043 0x00000001 0x00000000 0x00000000 0x50: 0x00000000 0x00000000 0x00000000 0x00000000 0x60: 0x00000000 0x00000000 0x00000000 0x00000000 0x70: 0x00000000 0x00000000 0x00000000 0x00000000 0x80: 0x20a00001 0x00000000 0x04630463 0x00000000 0x90: 0x00000000 0x00000000 0x00000000 0x00000000 0xa0: 0x80000000 0x00000000 0x00000000 0x00000000 0xb0: 0x00000000 0x00000000 0x00000000 0x00000000 0xc0: 0x17441043 0x00000000 0x00000000 0x00000000 0xd0: 0x00000000 0x00000000 0x00000000 0xfe0a0001 0xe0: 0x24c04000 0x00000000 0x00000000 0x00000000 0xf0: 0x00000000 0x00000000 0x00000000 0x00000000 fwohci0: mem 0xff7fe800-0xff7fefff irq 11 at device 5.1 on pci1 fwohci0: Reserved 0x800 bytes for rid 0x10 type 3 at 0xff7fe800 fwohci0: [MPSAFE] fwohci0: [FILTER] fwohci0: OHCI version 1.0 (ROM=1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:e0:18:00:03:10:02:07 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:e0:18:10:02:07 fwe0: bpf attached fwe0: Ethernet address: 02:e0:18:10:02:07 fwip0: on firewire0 fwip0: bpf attached fwip0: Firewire address: 00:e0:18:00:03:10:02:07 @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x1374000 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode fxp0: port 0xcc00-0xcc3f mem 0xff7ff000-0xff7fffff irq 11 at device 8.0 on pci1 fxp0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7ff000 fxp0: using memory space register mapping fxp0: PCI IDs: 8086 103e 1043 1745 0083 fxp0: Dynamic Standby mode is disabled fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: MII without any PHY! device_attach: fxp0 attach returned 6 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xffa0 ata0: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0 atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6 ata0: reset tp1 mask=03 ostat0=50 ostat1=00 ata0: stat0=0x90 err=0x90 lsb=0x90 msb=0x90 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=00 devices=0x1 ata0: [MPSAFE] ata0: [ITHREAD] ata1: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170 atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376 ata1: reset tp1 mask=03 ostat0=50 ostat1=00 ata1: stat0=0x10 err=0x01 lsb=0x14 msb=0xeb ata1: stat1=0x00 err=0x01 lsb=0x7f msb=0x7f ata1: reset tp2 stat0=10 stat1=00 devices=0x4 ata1: [MPSAFE] ata1: [ITHREAD] pcm0: port 0xe000-0xe0ff,0xe100-0xe13f at device 31.5 on pci0 pcm0: Lazy allocation of 0x200 bytes rid 0x18 type 3 at 0x80000000 pcm0: Lazy allocation of 0x100 bytes rid 0x1c type 3 at 0x80000200 pcib0: matched entry for 0.31.INTB (src \\_SB_.LNKB:0) pcib0: slot 31 INTB routed to irq 9 via \\_SB_.LNKB pcm0: [MPSAFE] pcm0: [ITHREAD] pcm0: pcm0: Codec features headphone, 20 bit DAC, 20 bit ADC, 5 bit master volume, SigmaTel 3D Enhancement pcm0: Primary codec extended features variable rate PCM, reserved 1, AMAP, reserved 4 pcm0: ac97 codec dac ready count: 0 pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Mixer "speaker": pcm0: Mixer "line": pcm0: Mixer "mic": pcm0: Mixer "cd": pcm0: Mixer "rec": pcm0: Mixer "igain": pcm0: Mixer "ogain": pcm0: Mixer "line1": pcm0: Mixer "phin": pcm0: Mixer "phout": pcm0: Mixer "video": pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 1620000, 4000; 0xd4d6f000 -> 1620000 pcm0: sndbuf_setmap 162c000, 4000; 0xd4d73000 -> 162c000 pci0: at device 31.6 (no driver attached) acpi_button0: on acpi0 acpi_lid0: on acpi0 acpi_tz0: on acpi0 acpi_acad0: on acpi0 battery0: on acpi0 battery1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x41ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0-00, 2 buttons psm0: config:00000000, flags:00000008, packet size:3 psm0: syncmask:c0, syncbits:00 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0 port 0x2f8-0x2ff irq 3 drq 1 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff ex_isa_identify() ahc_isa_probe 12: ioport 0xcc00 alloc failed ahc_isa_probe 13: ioport 0xdc00 alloc failed ahc_isa_probe 14: ioport 0xec00 alloc failed ata: ata0 already exists; skipping it ata: ata1 already exists; skipping it atkbdc: atkbdc0 already exists; skipping it sio: sio0 already exists; skipping it pnp_identify: Trying Read_Port at 203 pnp_identify: Trying Read_Port at 243 pnp_identify: Trying Read_Port at 283 pnp_identify: Trying Read_Port at 2c3 pnp_identify: Trying Read_Port at 303 pnp_identify: Trying Read_Port at 343 pnp_identify: Trying Read_Port at 383 pnp_identify: Trying Read_Port at 3c3 PNP Identify complete sc: sc0 already exists; skipping it vga: vga0 already exists; skipping it isa_probe_children: disabling PnP devices isa_probe_children: probing non-PnP devices pmtimer0 on isa0 orm0: at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 adv0: not probed (disabled) aha0: not probed (disabled) aic0: not probed (disabled) bt0: not probed (disabled) cs0: not probed (disabled) ed0: not probed (disabled) fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fe0: not probed (disabled) ie0: not probed (disabled) le0: not probed (disabled) ppc0: parallel port found at 0x378 ppc0: using extended I/O port range ppc0: ECP SPP ECP+EPP SPP ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [MPSAFE] ppbus0: [ITHREAD] plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) sio1 failed to probe at port 0x2f8 irq 3 on isa0 sio2: not probed (disabled) sio3: not probed (disabled) sn0: not probed (disabled) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 vt0: not probed (disabled) isa_probe_children: probing PnP devices Device configuration finished. procfs registered Timecounter "TSC" frequency 600024956 Hz quality 800 Timecounters tick every 1.000 msec lo0: bpf attached hptrr: no controller detected. firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire acpi_acad0: acline initialization start ad0: setting PIO4 on ICH4 chip ad0: setting UDMA100 on ICH4 chip battery0: battery initialization start battery1: battery initialization start system power profile changed to 'economy' ad0: 38154MB at ata0-master UDMA100 ad0: 78140160 sectors [77520C/16H/63S] 16 sectors/interrupt 1 depth queue GEOM: new disk ad0 acpi_acad0: Off Line acpi_acad0: acline initialization done, tried 1 times battery0: battery initialization done, tried 1 times ad0: Intel check1 failed ad0: Adaptec check1 failed ad0: LSI (v3) check1 failed ad0: LSI (v2) check1 failed ad0: FreeBSD check1 failed ata1-master: pio=PIO4 wdma=WDMA2 udma=UDMA33 cable=40 wire acd0: setting PIO4 on ICH4 chip acd0: setting UDMA33 on ICH4 chip acd0: DVDROM drive at ata1 as master acd0: read 4125KB/s (4125KB/s), 512KB buffer, UDMA33 acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, packet acd0: Writes: acd0: Audio: play, 16 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc pcm0: measured ac97 link rate at 48017 Hz, will use 48000 Hz acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe0:ata1:0:0:0): Down reving Protocol Version from 2 to 0? (probe0:ata1:0:0:0): error 6 (probe0:ata1:0:0:0): Unretryable Error acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe6:sbp0:0:5:0): error 22 (probe6:sbp0:0:5:0): Unretryable Error (probe1:sbp0:0:0:0): error 22 (probe1:sbp0:0:0:0): Unretryable Error (probe2:sbp0:0:1:0): error 22 (probe2:sbp0:0:1:0): Unretryable Error (probe3:sbp0:0:2:0): error 22 (probe3:sbp0:0:2:0): Unretryable Error (probe4:sbp0:0:3:0): error 22 (probe4:sbp0:0:3:0): Unretryable Error (probe5:sbp0:0:4:0): error 22 (probe5:sbp0:0:4:0): Unretryable Error (probe7:sbp0:0:6:0): error 22 (probe7:sbp0:0:6:0): Unretryable Error pass0 at ata1 bus 0 target 0 lun 0 pass0: Removable CD-ROM SCSI-0 device pass0: 33.000MB/s transfers GEOM: new disk cd0 ATA PseudoRAID loade(cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error cd0 at ata1 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present d (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error Trying to mount root from ufs:/dev/ad0s1a start_init: trying /sbin/init drm0: on vgapci0 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm1: on vgapci1 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm0: [MPSAFE] drm0: [ITHREAD] drm0: [MPSAFE] drm0: [ITHREAD] Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...0 0 0 done All buffers synced. Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Preloaded elf kernel "/boot/kernel/kernel" at 0xc0e72000. Preloaded elf module "/boot/kernel/if_ipw.ko" at 0xc0e7214c. Preloaded elf module "/boot/kernel/snd_ich.ko" at 0xc0e721f8. Preloaded elf module "/boot/kernel/sound.ko" at 0xc0e722a4. Preloaded elf module "/boot/kernel/ipw_bss.ko" at 0xc0e72350. Preloaded elf module "/boot/kernel/ipw_ibss.ko" at 0xc0e723fc. Preloaded elf module "/boot/kernel/ipw_monitor.ko" at 0xc0e724ac. Preloaded elf module "/boot/kernel/atapicam.ko" at 0xc0e7255c. Preloaded elf module "/boot/kernel/acpi.ko" at 0xc0e7260c. Calibrating clock(s) ... i8254 clock: 1193167 Hz CLK_USE_I8254_CALIBRATION not specified - using default frequency Timecounter "i8254" frequency 1193182 Hz quality 0 Calibrating TSC clock ... TSC clock: 600023372 Hz CPU: Intel(R) Pentium(R) M processor 1400MHz (600.02-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x695 Stepping = 5 Features=0xa7e9fbbf Features2=0x180 Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries Data TLB: 4 KB Pages, 4-way set associative, 128 entries Instruction TLB: 4 MB pages, fully associative, 2 entries 2nd-level cache: 1 MB, 8-way set associative, 64 byte line size 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size Data TLB: 4 MB Pages, 4-way set associative, 8 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size real memory = 527695872 (503 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009efff, 647168 bytes (158 pages) 0x0000000000100000 - 0x00000000003fffff, 3145728 bytes (768 pages) 0x0000000001028000 - 0x000000001ee22fff, 501198848 bytes (122363 pages) avail memory = 502419456 (479 MB) Table 'FACP' at 0x1f740200 Table 'OEMB' at 0x1f750040 MADT: No MADT table found APIC: Could not find any APICs. pnpbios: Found PnP BIOS data at 0xc00f2e00 pnpbios: Entry = f0000:39da Rev = 1.0 Other BIOS signatures found: wlan_amrr: wlan: <802.11 Link Layer> firmware: 'ipw_bss' version 130: 209190 bytes loaded at 0xc0d68738 firmware: 'ipw_ibss' version 130: 201138 bytes loaded at 0xc0d9d73c firmware: 'ipw_monitor' version 130: 196458 bytes loaded at 0xc0dd0748 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_buffersize=16384 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 ath_rate: version 1.2 nfslock: pseudo-device kbd: new array size 4 kbd1 at kbdmux0 io: mem: Pentium Pro MTRR support enabled null: random: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 19:59:27) ACPI: RSDP @ 0x0xf4b70/0x0014 (v 0 ACPIAM) ACPI: RSDT @ 0x0x1f740000/0x002C (v 1 A M I OEMRSDT 0x05000314 MSFT 0x00000097) ACPI: FACP @ 0x0x1f740200/0x0081 (v 2 A M I OEMFACP 0x05000314 MSFT 0x00000097) ACPI: DSDT @ 0x0x1f740300/0x7323 (v 1 0ABBD 0ABBD001 0x00000001 MSFT 0x0100000D) ACPI: FACS @ 0x0x1f750000/0x0040 ACPI: OEMB @ 0x0x1f750040/0x004D (v 1 A M I OEMBIOS 0x05000314 MSFT 0x00000097) npx0: INT 16 interface acpi0: on motherboard acpi0: [MPSAFE] acpi0: [ITHREAD] pci_open(1): mode 1 addr port (0x0cf8) is 0x8000005c pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=35808086) pcibios: No call entry point AcpiOsDerivePciId: \\_SB_.PCI0.P0P1.CBS0.CBSP -> bus 1 dev 5 func 0 acpi0: Power Button (fixed) acpi0: wakeup code va 0xccd3f000 pa 0x1000 atpic: Programming IRQ9 as level/low AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IROR -> bus 0 dev 31 func 0 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 1f700000 (3) failed ACPI timer: 1/0 1/0 1/1 1/1 1/0 1/0 1/1 1/1 1/1 1/0 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 5 6 7 11 12 Validation 0 11 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 11 12 Validation 0 255 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 12 Validation 0 4 N 0 4 12 After Disable 0 255 N 0 4 12 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 6 Validation 0 5 N 0 5 6 After Disable 0 255 N 0 5 6 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 6 11 Validation 0 11 N 0 6 11 After Disable 0 255 N 0 6 11 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 7 Validation 0 255 N 0 3 7 After Disable 0 255 N 0 3 7 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 4 7 Validation 0 255 N 0 4 7 After Disable 0 255 N 0 4 7 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 6 12 Validation 0 4 N 0 4 6 12 After Disable 0 255 N 0 4 6 12 cpu0: on acpi0 cpu0: switching to generic Cx mode est0: on cpu0 p4tcc0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 ACPI: Found matching pin for 0.2.INTA at func 0: 11 ACPI: Found matching pin for 0.31.INTA at func 1: 255 ACPI: Found matching pin for 0.31.INTB at func 5: 255 ACPI: Found matching pin for 0.31.INTB at func 6: 255 ACPI: Found matching pin for 0.29.INTA at func 0: 11 ACPI: Found matching pin for 0.29.INTB at func 1: 5 ACPI: Found matching pin for 0.29.INTC at func 2: 4 ACPI: Found matching pin for 0.29.INTD at func 7: 4 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x3580, revid=0x02 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3584, revid=0x02 domain=0, bus=0, slot=0, func=1 class=08-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3585, revid=0x02 domain=0, bus=0, slot=0, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 27, enabled map[14]: type Memory, range 32, base 0xffa80000, size 19, enabled map[18]: type I/O Port, range 32, base 0xdc00, size 3, enabled pcib0: matched entry for 0.2.INTA (src \\_SB_.LNKA:0) pcib0: slot 2 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=1 class=03-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xe8000000, size 27, enabled map[14]: type Memory, range 32, base 0xff980000, size 19, enabled found-> vendor=0x8086, dev=0x24c2, revid=0x03 domain=0, bus=0, slot=29, func=0 class=0c-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 map[20]: type I/O Port, range 32, base 0xd480, size 5, enabled pcib0: matched entry for 0.29.INTA (src \\_SB_.LNKA:0) pcib0: slot 29 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x24c4, revid=0x03 domain=0, bus=0, slot=29, func=1 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 map[20]: type I/O Port, range 32, base 0xd800, size 5, enabled pcib0: matched entry for 0.29.INTB (src \\_SB_.LNKD:0) pcib0: slot 29 INTB routed to irq 5 via \\_SB_.LNKD found-> vendor=0x8086, dev=0x24c7, revid=0x03 domain=0, bus=0, slot=29, func=2 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=4 map[20]: type I/O Port, range 32, base 0xd880, size 5, enabled pcib0: matched entry for 0.29.INTC (src \\_SB_.LNKC:0) pcib0: slot 29 INTC routed to irq 4 via \\_SB_.LNKC found-> vendor=0x8086, dev=0x24cd, revid=0x03 domain=0, bus=0, slot=29, func=7 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=d, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xffa7fc00, size 10, enabled pcib0: matched entry for 0.29.INTD (src \\_SB_.LNKH:0) pcib0: slot 29 INTD routed to irq 4 via \\_SB_.LNKH found-> vendor=0x8086, dev=0x2448, revid=0x83 domain=0, bus=0, slot=30, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x8080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x06 (1500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24cc, revid=0x03 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24ca, revid=0x03 domain=0, bus=0, slot=31, func=1 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled map[24]: type Memory, range 32, base 0, size 10, memory disabled found-> vendor=0x8086, dev=0x24c5, revid=0x03 domain=0, bus=0, slot=31, func=5 class=04-01-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe000, size 8, enabled map[14]: type I/O Port, range 32, base 0xe100, size 6, enabled map[18]: type Memory, range 32, base 0, size 9, memory disabled map[1c]: type Memory, range 32, base 0, size 8, memory disabled found-> vendor=0x8086, dev=0x24c6, revid=0x03 domain=0, bus=0, slot=31, func=6 class=07-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe200, size 8, enabled map[14]: type I/O Port, range 32, base 0xe300, size 7, enabled pci0: at device 0.1 (no driver attached) pci0: at device 0.3 (no driver attached) vgapci0: port 0xdc00-0xdc07 mem 0xf0000000-0xf7ffffff,0xffa80000-0xffafffff irq 11 at device 2.0 on pci0 agp0: on vgapci0 vgapci0: Reserved 0x8000000 bytes for rid 0x10 type 3 at 0xf0000000 vgapci0: Reserved 0x80000 bytes for rid 0x14 type 3 at 0xffa80000 agp0: detected 8060k stolen memory agp0: aperture size is 128M vgapci1: mem 0xe8000000-0xefffffff,0xff980000-0xff9fffff at device 2.1 on pci0 uhci0: port 0xd480-0xd49f irq 11 at device 29.0 on pci0 uhci0: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd480 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd800-0xd81f irq 5 at device 29.1 on pci0 uhci1: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd800 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xd880-0xd89f irq 4 at device 29.2 on pci0 uhci2: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd880 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xffa7fc00-0xffa7ffff irq 4 at device 29.7 on pci0 ehci0: Reserved 0x400 bytes for rid 0x10 type 3 at 0xffa7fc00 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered pcib1: at device 30.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xc000-0xcfff pcib1: memory decode 0xff700000-0xff7fffff pcib1: prefetched decode 0xdea00000-0xdeafffff pcib1: Subtractively decoded bridge. ACPI: Found matching pin for 1.8.INTA at func 0: 11 ACPI: Found matching pin for 1.5.INTA at func 0: 255 ACPI: Found matching pin for 1.5.INTB at func 1: 11 ACPI: Found matching pin for 1.4.INTA at func 0: 4 pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x8086, dev=0x1043, revid=0x04 domain=0, bus=1, slot=4, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0116, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x22 (8500 ns) intpin=a, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fd000, size 12, enabled pcib1: requested memory range 0xff7fd000-0xff7fdfff: good pcib1: matched entry for 1.4.INTA (src \\_SB_.LNKC:0) pcib1: slot 4 INTA routed to irq 4 via \\_SB_.LNKC found-> vendor=0x1180, dev=0x0475, revid=0xb8 domain=0, bus=1, slot=5, func=0 class=06-07-00, hdrtype=0x02, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x80 (32000 ns), maxlat=0x07 (1750 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0, size 12, enabled found-> vendor=0x1180, dev=0x0551, revid=0x00 domain=0, bus=1, slot=5, func=1 class=0c-00-10, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x04 (1000 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fe800, size 11, enabled pcib1: requested memory range 0xff7fe800-0xff7fefff: good pcib1: matched entry for 1.5.INTB (src \\_SB_.LNKA:0) pcib1: slot 5 INTB routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x103e, revid=0x83 domain=0, bus=1, slot=8, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0117, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xff7ff000, size 12, enabled pcib1: requested memory range 0xff7ff000-0xff7fffff: good map[14]: type I/O Port, range 32, base 0xcc00, size 6, enabled pcib1: requested I/O range 0xcc00-0xcc3f: in range pcib1: matched entry for 1.8.INTA (src \\_SB_.LNKE:0) pcib1: slot 8 INTA routed to irq 11 via \\_SB_.LNKE ipw0: mem 0xff7fd000-0xff7fdfff irq 4 at device 4.0 on pci1 ipw0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7fd000 ipw0: bpf attached ipw0: Ethernet address: 00:04:23:71:77:46 ipw0: bpf attached ipw0: bpf attached ipw0: [MPSAFE] ipw0: [ITHREAD] ipw0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps cbb0: at device 5.0 on pci1 pcib1: cbb0 requested memory range 0xff700000-0xff7fffff: good cbb0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xff700000 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pcib1: matched entry for 1.5.INTA (src \\_SB_.LNKB:0) pci_link1: Picked IRQ 9 with weight 0 pcib1: slot 5 INTA routed to irq 9 via \\_SB_.LNKB cbb0: [MPSAFE] cbb0: [ITHREAD] cbb0: PCI Configuration space: 0x00: 0x04751180 0x02100007 0x060700b8 0x00822000 0x10: 0xff700000 0x020000dc 0x20030201 0xfffff000 0x20: 0x00000000 0xfffff000 0x00000000 0xfffffffc 0x30: 0x00000000 0xfffffffc 0x00000000 0x07000109 0x40: 0x17441043 0x00000001 0x00000000 0x00000000 0x50: 0x00000000 0x00000000 0x00000000 0x00000000 0x60: 0x00000000 0x00000000 0x00000000 0x00000000 0x70: 0x00000000 0x00000000 0x00000000 0x00000000 0x80: 0x20a00001 0x00000000 0x04630463 0x00000000 0x90: 0x00000000 0x00000000 0x00000000 0x00000000 0xa0: 0x80000000 0x00000000 0x00000000 0x00000000 0xb0: 0x00000000 0x00000000 0x00000000 0x00000000 0xc0: 0x17441043 0x00000000 0x00000000 0x00000000 0xd0: 0x00000000 0x00000000 0x00000000 0xfe0a0001 0xe0: 0x24c04000 0x00000000 0x00000000 0x00000000 0xf0: 0x00000000 0x00000000 0x00000000 0x00000000 fwohci0: mem 0xff7fe800-0xff7fefff irq 11 at device 5.1 on pci1 fwohci0: Reserved 0x800 bytes for rid 0x10 type 3 at 0xff7fe800 fwohci0: [MPSAFE] fwohci0: [FILTER] fwohci0: OHCI version 1.0 (ROM=1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:e0:18:00:03:10:02:07 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:e0:18:10:02:07 fwe0: bpf attached fwe0: Ethernet address: 02:e0:18:10:02:07 fwip0: on firewire0 fwip0: bpf attached fwip0: Firewire address: 00:e0:18:00:03:10:02:07 @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x1374000 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode fxp0: port 0xcc00-0xcc3f mem 0xff7ff000-0xff7fffff irq 11 at device 8.0 on pci1 fxp0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7ff000 fxp0: using memory space register mapping fxp0: PCI IDs: 8086 103e 1043 1745 0083 fxp0: Dynamic Standby mode is disabled fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: MII without any PHY! device_attach: fxp0 attach returned 6 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xffa0 ata0: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0 atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6 ata0: reset tp1 mask=03 ostat0=50 ostat1=00 ata0: stat0=0x90 err=0x90 lsb=0x90 msb=0x90 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=00 devices=0x1 ata0: [MPSAFE] ata0: [ITHREAD] ata1: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170 atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376 ata1: reset tp1 mask=03 ostat0=50 ostat1=00 ata1: stat0=0x10 err=0x01 lsb=0x14 msb=0xeb ata1: stat1=0x00 err=0x01 lsb=0x7f msb=0x7f ata1: reset tp2 stat0=10 stat1=00 devices=0x4 ata1: [MPSAFE] ata1: [ITHREAD] pcm0: port 0xe000-0xe0ff,0xe100-0xe13f at device 31.5 on pci0 pcm0: Lazy allocation of 0x200 bytes rid 0x18 type 3 at 0x80000000 pcm0: Lazy allocation of 0x100 bytes rid 0x1c type 3 at 0x80000200 pcib0: matched entry for 0.31.INTB (src \\_SB_.LNKB:0) pcib0: slot 31 INTB routed to irq 9 via \\_SB_.LNKB pcm0: [MPSAFE] pcm0: [ITHREAD] pcm0: pcm0: Codec features headphone, 20 bit DAC, 20 bit ADC, 5 bit master volume, SigmaTel 3D Enhancement pcm0: Primary codec extended features variable rate PCM, reserved 1, AMAP, reserved 4 pcm0: ac97 codec dac ready count: 0 pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Mixer "speaker": pcm0: Mixer "line": pcm0: Mixer "mic": pcm0: Mixer "cd": pcm0: Mixer "rec": pcm0: Mixer "igain": pcm0: Mixer "ogain": pcm0: Mixer "line1": pcm0: Mixer "phin": pcm0: Mixer "phout": pcm0: Mixer "video": pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 1620000, 4000; 0xd4d6f000 -> 1620000 pcm0: sndbuf_setmap 162c000, 4000; 0xd4d73000 -> 162c000 pci0: at device 31.6 (no driver attached) acpi_button0: on acpi0 acpi_lid0: on acpi0 acpi_tz0: on acpi0 acpi_acad0: on acpi0 battery0: on acpi0 battery1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x41ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0-00, 2 buttons psm0: config:00000000, flags:00000008, packet size:3 psm0: syncmask:c0, syncbits:00 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0 port 0x2f8-0x2ff irq 3 drq 1 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff ex_isa_identify() ahc_isa_probe 12: ioport 0xcc00 alloc failed ahc_isa_probe 13: ioport 0xdc00 alloc failed ahc_isa_probe 14: ioport 0xec00 alloc failed ata: ata0 already exists; skipping it ata: ata1 already exists; skipping it atkbdc: atkbdc0 already exists; skipping it sio: sio0 already exists; skipping it pnp_identify: Trying Read_Port at 203 pnp_identify: Trying Read_Port at 243 pnp_identify: Trying Read_Port at 283 pnp_identify: Trying Read_Port at 2c3 pnp_identify: Trying Read_Port at 303 pnp_identify: Trying Read_Port at 343 pnp_identify: Trying Read_Port at 383 pnp_identify: Trying Read_Port at 3c3 PNP Identify complete sc: sc0 already exists; skipping it vga: vga0 already exists; skipping it isa_probe_children: disabling PnP devices isa_probe_children: probing non-PnP devices pmtimer0 on isa0 orm0: at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 adv0: not probed (disabled) aha0: not probed (disabled) aic0: not probed (disabled) bt0: not probed (disabled) cs0: not probed (disabled) ed0: not probed (disabled) fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fe0: not probed (disabled) ie0: not probed (disabled) le0: not probed (disabled) ppc0: parallel port found at 0x378 ppc0: using extended I/O port range ppc0: ECP SPP ECP+EPP SPP ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [MPSAFE] ppbus0: [ITHREAD] plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) sio1 failed to probe at port 0x2f8 irq 3 on isa0 sio2: not probed (disabled) sio3: not probed (disabled) sn0: not probed (disabled) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 vt0: not probed (disabled) isa_probe_children: probing PnP devices Device configuration finished. procfs registered Timecounter "TSC" frequency 600023372 Hz quality 800 Timecounters tick every 1.000 msec lo0: bpf attachedfirewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) hptrr: no controller detected. acpi_acad0: acline initialization start battery0: battery initialization start battery1: battery initialization start ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire system power profile changed to 'economy' ad0: setting PIO4 on ICH4 chip acpi_acad0: Off Line acpi_acad0: acline initialization done, tried 1 times ad0: setting UDMA100 on ICH4 chip ad0: 38154MB at ata0-master UDMA100 ad0: 78140160 sectors [77520C/16H/63S] 16 sectors/interrupt 1 depth queue GEOM: new disk ad0 battery0: battery initialization done, tried 1 times ad0: Intel check1 failed ad0: Adaptec check1 failed ad0: LSI (v3) check1 failed ad0: LSI (v2) check1 failed ad0: FreeBSD check1 failed ata1-master: pio=PIO4 wdma=WDMA2 udma=UDMA33 cable=40 wire acd0: setting PIO4 on ICH4 chip acd0: setting UDMA33 on ICH4 chip acd0: DVDROM drive at ata1 as master acd0: read 4125KB/s (4125KB/s), 512KB buffer, UDMA33 acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, packet acd0: Writes: acd0: Audio: play, 16 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc pcm0: measured ac97 link rate at 48008 Hz, will use 48000 Hz acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe0:ata1:0:0:0): Down reving Protocol Version from 2 to 0? (probe0:ata1:0:0:0): error 6 (probe0:ata1:0:0:0): Unretryable Error acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe1:sbp0:0:0:0): error 22 (probe1:sbp0:0:0:0): Unretryable Error (probe2:sbp0:0:1:0): error 22 (probe2:sbp0:0:1:0): Unretryable Error (probe3:sbp0:0:2:0): error 22 (probe3:sbp0:0:2:0): Unretryable Error (probe4:sbp0:0:3:0): error 22 (probe4:sbp0:0:3:0): Unretryable Error (probe5:sbp0:0:4:0): error 22 (probe5:sbp0:0:4:0): Unretryable Error (probe6:sbp0:0:5:0): error 22 (probe6:sbp0:0:5:0): Unretryable Error (probe7:sbp0:0:6:0): error 22 (probe7:sbp0:0:6:0): Unretryable Error pass0 at ata1 bus 0 target 0 lun 0 pass0: Removable CD-ROM SCSI-0 device pass0: 33.000MB/s transfers GEOM: new disk cd0 ATA PseudoRAID load(cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error cd0 at ata1 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present ed (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error Trying to mount root from ufs:/dev/ad0s1a start_init: trying /sbin/init drm0: on vgapci0 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm1: on vgapci1 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm0: [MPSAFE] drm0: [ITHREAD] battery1: battery initialization failed, giving up umass0: on uhub3 umass0:3:0:-1: Attached to scbus3 (probe0:umass-sim0:0:0:0): error 22 (probe0:umass-sim0:0:0:0): Unretryable Error pass1 at umass-sim0 bus 0 target 0 lun 0 pass1: < > Removable Direct Access SCSI-2 device pass1: 40.000MB/s transfers GEOM: new disk da0 da0 at umass-sim0 bus 0 target 0 lun 0 da0: < > Removable Direct Access SCSI-2 device da0: 40.000MB/s transfers da0: 3102MB (6354432 512 byte sectors: 255H 63S/T 395C) GEOM_LABEL: Label for provider da0s1a is ufs/usbdrive. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 18:50:06 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98D391065672 for ; Tue, 8 Jul 2008 18:50:06 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 851778FC2D for ; Tue, 8 Jul 2008 18:50:06 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m68Io6RL087521 for ; Tue, 8 Jul 2008 18:50:06 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m68Io6ev087520; Tue, 8 Jul 2008 18:50:06 GMT (envelope-from gnats) Date: Tue, 8 Jul 2008 18:50:06 GMT Message-Id: <200807081850.m68Io6ev087520@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Francis Gendreau Cc: Subject: Re: kern/125195: verbrose dmesg from asus m3000n m3n as requested by Gavin Atkinson X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Francis Gendreau List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 18:50:06 -0000 The following reply was made to PR kern/125195; it has been noted by GNATS. From: Francis Gendreau To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/125195: verbrose dmesg from asus m3000n m3n as requested by Gavin Atkinson Date: Tue, 08 Jul 2008 11:33:22 -0400 verbose dmesg after hard power cycle: Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries Data TLB: 4 KB Pages, 4-way set associative, 128 entries Instruction TLB: 4 MB pages, fully associative, 2 entries 2nd-level cache: 1 MB, 8-way set associative, 64 byte line size 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size Data TLB: 4 MB Pages, 4-way set associative, 8 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size real memory = 527695872 (503 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009efff, 647168 bytes (158 pages) 0x0000000000100000 - 0x00000000003fffff, 3145728 bytes (768 pages) 0x0000000001028000 - 0x000000001ee22fff, 501198848 bytes (122363 pages) avail memory = 502419456 (479 MB) Table 'FACP' at 0x1f740200 Table 'OEMB' at 0x1f750040 MADT: No MADT table found APIC: Could not find any APICs. pnpbios: Found PnP BIOS data at 0xc00f2e00 pnpbios: Entry = f0000:39da Rev = 1.0 Other BIOS signatures found: wlan_amrr: wlan: <802.11 Link Layer> firmware: 'ipw_bss' version 130: 209190 bytes loaded at 0xc0d68738 firmware: 'ipw_ibss' version 130: 201138 bytes loaded at 0xc0d9d73c firmware: 'ipw_monitor' version 130: 196458 bytes loaded at 0xc0dd0748 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_buffersize=16384 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 ath_rate: version 1.2 nfslock: pseudo-device kbd: new array size 4 kbd1 at kbdmux0 io: mem: Pentium Pro MTRR support enabled null: random: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 19:59:27) ACPI: RSDP @ 0x0xf4b70/0x0014 (v 0 ACPIAM) ACPI: RSDT @ 0x0x1f740000/0x002C (v 1 A M I OEMRSDT 0x05000314 MSFT 0x00000097) ACPI: FACP @ 0x0x1f740200/0x0081 (v 2 A M I OEMFACP 0x05000314 MSFT 0x00000097) ACPI: DSDT @ 0x0x1f740300/0x7323 (v 1 0ABBD 0ABBD001 0x00000001 MSFT 0x0100000D) ACPI: FACS @ 0x0x1f750000/0x0040 ACPI: OEMB @ 0x0x1f750040/0x004D (v 1 A M I OEMBIOS 0x05000314 MSFT 0x00000097) npx0: INT 16 interface acpi0: on motherboard acpi0: [MPSAFE] acpi0: [ITHREAD] pci_open(1): mode 1 addr port (0x0cf8) is 0x8000005c pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=35808086) pcibios: No call entry point AcpiOsDerivePciId: \\_SB_.PCI0.P0P1.CBS0.CBSP -> bus 1 dev 5 func 0 acpi0: Power Button (fixed) acpi0: wakeup code va 0xccd3f000 pa 0x1000 atpic: Programming IRQ9 as level/low AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IROR -> bus 0 dev 31 func 0 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 1f700000 (3) failed ACPI timer: 1/1 1/1 1/0 1/1 1/1 1/1 1/0 1/1 1/1 1/1 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 5 6 7 11 12 Validation 0 11 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 11 12 Validation 0 255 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 12 Validation 0 4 N 0 4 12 After Disable 0 255 N 0 4 12 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 6 Validation 0 5 N 0 5 6 After Disable 0 255 N 0 5 6 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 6 11 Validation 0 11 N 0 6 11 After Disable 0 255 N 0 6 11 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 7 Validation 0 255 N 0 3 7 After Disable 0 255 N 0 3 7 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 4 7 Validation 0 255 N 0 4 7 After Disable 0 255 N 0 4 7 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 6 12 Validation 0 4 N 0 4 6 12 After Disable 0 255 N 0 4 6 12 cpu0: on acpi0 cpu0: switching to generic Cx mode est0: on cpu0 p4tcc0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 ACPI: Found matching pin for 0.2.INTA at func 0: 11 ACPI: Found matching pin for 0.31.INTA at func 1: 255 ACPI: Found matching pin for 0.31.INTB at func 5: 255 ACPI: Found matching pin for 0.31.INTB at func 6: 255 ACPI: Found matching pin for 0.29.INTA at func 0: 11 ACPI: Found matching pin for 0.29.INTB at func 1: 5 ACPI: Found matching pin for 0.29.INTC at func 2: 4 ACPI: Found matching pin for 0.29.INTD at func 7: 4 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x3580, revid=0x02 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3584, revid=0x02 domain=0, bus=0, slot=0, func=1 class=08-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3585, revid=0x02 domain=0, bus=0, slot=0, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 27, enabled map[14]: type Memory, range 32, base 0xffa80000, size 19, enabled map[18]: type I/O Port, range 32, base 0xdc00, size 3, enabled pcib0: matched entry for 0.2.INTA (src \\_SB_.LNKA:0) pcib0: slot 2 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=1 class=03-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xe8000000, size 27, enabled map[14]: type Memory, range 32, base 0xff980000, size 19, enabled found-> vendor=0x8086, dev=0x24c2, revid=0x03 domain=0, bus=0, slot=29, func=0 class=0c-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 map[20]: type I/O Port, range 32, base 0xd480, size 5, enabled pcib0: matched entry for 0.29.INTA (src \\_SB_.LNKA:0) pcib0: slot 29 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x24c4, revid=0x03 domain=0, bus=0, slot=29, func=1 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 map[20]: type I/O Port, range 32, base 0xd800, size 5, enabled pcib0: matched entry for 0.29.INTB (src \\_SB_.LNKD:0) pcib0: slot 29 INTB routed to irq 5 via \\_SB_.LNKD found-> vendor=0x8086, dev=0x24c7, revid=0x03 domain=0, bus=0, slot=29, func=2 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=4 map[20]: type I/O Port, range 32, base 0xd880, size 5, enabled pcib0: matched entry for 0.29.INTC (src \\_SB_.LNKC:0) pcib0: slot 29 INTC routed to irq 4 via \\_SB_.LNKC found-> vendor=0x8086, dev=0x24cd, revid=0x03 domain=0, bus=0, slot=29, func=7 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=d, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xffa7fc00, size 10, enabled pcib0: matched entry for 0.29.INTD (src \\_SB_.LNKH:0) pcib0: slot 29 INTD routed to irq 4 via \\_SB_.LNKH found-> vendor=0x8086, dev=0x2448, revid=0x83 domain=0, bus=0, slot=30, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x8080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x06 (1500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24cc, revid=0x03 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24ca, revid=0x03 domain=0, bus=0, slot=31, func=1 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled map[24]: type Memory, range 32, base 0, size 10, memory disabled found-> vendor=0x8086, dev=0x24c5, revid=0x03 domain=0, bus=0, slot=31, func=5 class=04-01-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe000, size 8, enabled map[14]: type I/O Port, range 32, base 0xe100, size 6, enabled map[18]: type Memory, range 32, base 0, size 9, memory disabled map[1c]: type Memory, range 32, base 0, size 8, memory disabled found-> vendor=0x8086, dev=0x24c6, revid=0x03 domain=0, bus=0, slot=31, func=6 class=07-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe200, size 8, enabled map[14]: type I/O Port, range 32, base 0xe300, size 7, enabled pci0: at device 0.1 (no driver attached) pci0: at device 0.3 (no driver attached) vgapci0: port 0xdc00-0xdc07 mem 0xf0000000-0xf7ffffff,0xffa80000-0xffafffff irq 11 at device 2.0 on pci0 agp0: on vgapci0 vgapci0: Reserved 0x8000000 bytes for rid 0x10 type 3 at 0xf0000000 vgapci0: Reserved 0x80000 bytes for rid 0x14 type 3 at 0xffa80000 agp0: detected 8060k stolen memory agp0: aperture size is 128M vgapci1: mem 0xe8000000-0xefffffff,0xff980000-0xff9fffff at device 2.1 on pci0 uhci0: port 0xd480-0xd49f irq 11 at device 29.0 on pci0 uhci0: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd480 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd800-0xd81f irq 5 at device 29.1 on pci0 uhci1: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd800 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xd880-0xd89f irq 4 at device 29.2 on pci0 uhci2: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd880 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xffa7fc00-0xffa7ffff irq 4 at device 29.7 on pci0 ehci0: Reserved 0x400 bytes for rid 0x10 type 3 at 0xffa7fc00 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered pcib1: at device 30.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xc000-0xcfff pcib1: memory decode 0xff700000-0xff7fffff pcib1: prefetched decode 0xdea00000-0xdeafffff pcib1: Subtractively decoded bridge. ACPI: Found matching pin for 1.8.INTA at func 0: 11 ACPI: Found matching pin for 1.5.INTA at func 0: 255 ACPI: Found matching pin for 1.5.INTB at func 1: 11 ACPI: Found matching pin for 1.4.INTA at func 0: 4 pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x8086, dev=0x1043, revid=0x04 domain=0, bus=1, slot=4, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0116, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x22 (8500 ns) intpin=a, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fd000, size 12, enabled pcib1: requested memory range 0xff7fd000-0xff7fdfff: good pcib1: matched entry for 1.4.INTA (src \\_SB_.LNKC:0) pcib1: slot 4 INTA routed to irq 4 via \\_SB_.LNKC found-> vendor=0x1180, dev=0x0475, revid=0xb8 domain=0, bus=1, slot=5, func=0 class=06-07-00, hdrtype=0x02, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x80 (32000 ns), maxlat=0x07 (1750 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0, size 12, enabled found-> vendor=0x1180, dev=0x0551, revid=0x00 domain=0, bus=1, slot=5, func=1 class=0c-00-10, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x04 (1000 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fe800, size 11, enabled pcib1: requested memory range 0xff7fe800-0xff7fefff: good pcib1: matched entry for 1.5.INTB (src \\_SB_.LNKA:0) pcib1: slot 5 INTB routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x103e, revid=0x83 domain=0, bus=1, slot=8, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0117, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xff7ff000, size 12, enabled pcib1: requested memory range 0xff7ff000-0xff7fffff: good map[14]: type I/O Port, range 32, base 0xcc00, size 6, enabled pcib1: requested I/O range 0xcc00-0xcc3f: in range pcib1: matched entry for 1.8.INTA (src \\_SB_.LNKE:0) pcib1: slot 8 INTA routed to irq 11 via \\_SB_.LNKE ipw0: mem 0xff7fd000-0xff7fdfff irq 4 at device 4.0 on pci1 ipw0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7fd000 ipw0: bpf attached ipw0: Ethernet address: 00:04:23:71:77:46 ipw0: bpf attached ipw0: bpf attached ipw0: [MPSAFE] ipw0: [ITHREAD] ipw0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps cbb0: at device 5.0 on pci1 pcib1: cbb0 requested memory range 0xff700000-0xff7fffff: good cbb0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xff700000 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pcib1: matched entry for 1.5.INTA (src \\_SB_.LNKB:0) pci_link1: Picked IRQ 9 with weight 0 pcib1: slot 5 INTA routed to irq 9 via \\_SB_.LNKB cbb0: [MPSAFE] cbb0: [ITHREAD] cbb0: PCI Configuration space: 0x00: 0x04751180 0x02100007 0x060700b8 0x00822000 0x10: 0xff700000 0x020000dc 0x20030201 0xfffff000 0x20: 0x00000000 0xfffff000 0x00000000 0xfffffffc 0x30: 0x00000000 0xfffffffc 0x00000000 0x07000109 0x40: 0x17441043 0x00000001 0x00000000 0x00000000 0x50: 0x00000000 0x00000000 0x00000000 0x00000000 0x60: 0x00000000 0x00000000 0x00000000 0x00000000 0x70: 0x00000000 0x00000000 0x00000000 0x00000000 0x80: 0x20a00001 0x00000000 0x04630463 0x00000000 0x90: 0x00000000 0x00000000 0x00000000 0x00000000 0xa0: 0x80000000 0x00000000 0x00000000 0x00000000 0xb0: 0x00000000 0x00000000 0x00000000 0x00000000 0xc0: 0x17441043 0x00000000 0x00000000 0x00000000 0xd0: 0x00000000 0x00000000 0x00000000 0xfe0a0001 0xe0: 0x24c04000 0x00000000 0x00000000 0x00000000 0xf0: 0x00000000 0x00000000 0x00000000 0x00000000 fwohci0: mem 0xff7fe800-0xff7fefff irq 11 at device 5.1 on pci1 fwohci0: Reserved 0x800 bytes for rid 0x10 type 3 at 0xff7fe800 fwohci0: [MPSAFE] fwohci0: [FILTER] fwohci0: OHCI version 1.0 (ROM=1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:e0:18:00:03:10:02:07 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:e0:18:10:02:07 fwe0: bpf attached fwe0: Ethernet address: 02:e0:18:10:02:07 fwip0: on firewire0 fwip0: bpf attached fwip0: Firewire address: 00:e0:18:00:03:10:02:07 @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x1374000 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode fxp0: port 0xcc00-0xcc3f mem 0xff7ff000-0xff7fffff irq 11 at device 8.0 on pci1 fxp0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7ff000 fxp0: using memory space register mapping fxp0: PCI IDs: 8086 103e 1043 1745 0083 fxp0: Dynamic Standby mode is disabled fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: MII without any PHY! device_attach: fxp0 attach returned 6 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xffa0 ata0: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0 atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6 ata0: reset tp1 mask=03 ostat0=50 ostat1=00 ata0: stat0=0x90 err=0x90 lsb=0x90 msb=0x90 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=00 devices=0x1 ata0: [MPSAFE] ata0: [ITHREAD] ata1: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170 atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376 ata1: reset tp1 mask=03 ostat0=50 ostat1=00 ata1: stat0=0x10 err=0x01 lsb=0x14 msb=0xeb ata1: stat1=0x00 err=0x01 lsb=0x7f msb=0x7f ata1: reset tp2 stat0=10 stat1=00 devices=0x4 ata1: [MPSAFE] ata1: [ITHREAD] pcm0: port 0xe000-0xe0ff,0xe100-0xe13f at device 31.5 on pci0 pcm0: Lazy allocation of 0x200 bytes rid 0x18 type 3 at 0x80000000 pcm0: Lazy allocation of 0x100 bytes rid 0x1c type 3 at 0x80000200 pcib0: matched entry for 0.31.INTB (src \\_SB_.LNKB:0) pcib0: slot 31 INTB routed to irq 9 via \\_SB_.LNKB pcm0: [MPSAFE] pcm0: [ITHREAD] pcm0: pcm0: Codec features headphone, 20 bit DAC, 20 bit ADC, 5 bit master volume, SigmaTel 3D Enhancement pcm0: Primary codec extended features variable rate PCM, reserved 1, AMAP, reserved 4 pcm0: ac97 codec dac ready count: 0 pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Mixer "speaker": pcm0: Mixer "line": pcm0: Mixer "mic": pcm0: Mixer "cd": pcm0: Mixer "rec": pcm0: Mixer "igain": pcm0: Mixer "ogain": pcm0: Mixer "line1": pcm0: Mixer "phin": pcm0: Mixer "phout": pcm0: Mixer "video": pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 1620000, 4000; 0xd4d6f000 -> 1620000 pcm0: sndbuf_setmap 162c000, 4000; 0xd4d73000 -> 162c000 pci0: at device 31.6 (no driver attached) acpi_button0: on acpi0 acpi_lid0: on acpi0 acpi_tz0: on acpi0 acpi_acad0: on acpi0 battery0: on acpi0 battery1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x41ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0-00, 2 buttons psm0: config:00000000, flags:00000008, packet size:3 psm0: syncmask:c0, syncbits:00 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0 port 0x2f8-0x2ff irq 3 drq 1 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff ex_isa_identify() ahc_isa_probe 12: ioport 0xcc00 alloc failed ahc_isa_probe 13: ioport 0xdc00 alloc failed ahc_isa_probe 14: ioport 0xec00 alloc failed ata: ata0 already exists; skipping it ata: ata1 already exists; skipping it atkbdc: atkbdc0 already exists; skipping it sio: sio0 already exists; skipping it pnp_identify: Trying Read_Port at 203 pnp_identify: Trying Read_Port at 243 pnp_identify: Trying Read_Port at 283 pnp_identify: Trying Read_Port at 2c3 pnp_identify: Trying Read_Port at 303 pnp_identify: Trying Read_Port at 343 pnp_identify: Trying Read_Port at 383 pnp_identify: Trying Read_Port at 3c3 PNP Identify complete sc: sc0 already exists; skipping it vga: vga0 already exists; skipping it isa_probe_children: disabling PnP devices isa_probe_children: probing non-PnP devices pmtimer0 on isa0 orm0: at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 adv0: not probed (disabled) aha0: not probed (disabled) aic0: not probed (disabled) bt0: not probed (disabled) cs0: not probed (disabled) ed0: not probed (disabled) fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fe0: not probed (disabled) ie0: not probed (disabled) le0: not probed (disabled) ppc0: parallel port found at 0x378 ppc0: using extended I/O port range ppc0: ECP SPP ECP+EPP SPP ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [MPSAFE] ppbus0: [ITHREAD] plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) sio1 failed to probe at port 0x2f8 irq 3 on isa0 sio2: not probed (disabled) sio3: not probed (disabled) sn0: not probed (disabled) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 vt0: not probed (disabled) isa_probe_children: probing PnP devices Device configuration finished. procfs registered Timecounter "TSC" frequency 600024956 Hz quality 800 Timecounters tick every 1.000 msec lo0: bpf attached hptrr: no controller detected. firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire acpi_acad0: acline initialization start ad0: setting PIO4 on ICH4 chip ad0: setting UDMA100 on ICH4 chip battery0: battery initialization start battery1: battery initialization start system power profile changed to 'economy' ad0: 38154MB at ata0-master UDMA100 ad0: 78140160 sectors [77520C/16H/63S] 16 sectors/interrupt 1 depth queue GEOM: new disk ad0 acpi_acad0: Off Line acpi_acad0: acline initialization done, tried 1 times battery0: battery initialization done, tried 1 times ad0: Intel check1 failed ad0: Adaptec check1 failed ad0: LSI (v3) check1 failed ad0: LSI (v2) check1 failed ad0: FreeBSD check1 failed ata1-master: pio=PIO4 wdma=WDMA2 udma=UDMA33 cable=40 wire acd0: setting PIO4 on ICH4 chip acd0: setting UDMA33 on ICH4 chip acd0: DVDROM drive at ata1 as master acd0: read 4125KB/s (4125KB/s), 512KB buffer, UDMA33 acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, packet acd0: Writes: acd0: Audio: play, 16 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc pcm0: measured ac97 link rate at 48017 Hz, will use 48000 Hz acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe0:ata1:0:0:0): Down reving Protocol Version from 2 to 0? (probe0:ata1:0:0:0): error 6 (probe0:ata1:0:0:0): Unretryable Error acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe6:sbp0:0:5:0): error 22 (probe6:sbp0:0:5:0): Unretryable Error (probe1:sbp0:0:0:0): error 22 (probe1:sbp0:0:0:0): Unretryable Error (probe2:sbp0:0:1:0): error 22 (probe2:sbp0:0:1:0): Unretryable Error (probe3:sbp0:0:2:0): error 22 (probe3:sbp0:0:2:0): Unretryable Error (probe4:sbp0:0:3:0): error 22 (probe4:sbp0:0:3:0): Unretryable Error (probe5:sbp0:0:4:0): error 22 (probe5:sbp0:0:4:0): Unretryable Error (probe7:sbp0:0:6:0): error 22 (probe7:sbp0:0:6:0): Unretryable Error pass0 at ata1 bus 0 target 0 lun 0 pass0: Removable CD-ROM SCSI-0 device pass0: 33.000MB/s transfers GEOM: new disk cd0 ATA PseudoRAID loade(cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error cd0 at ata1 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present d (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error Trying to mount root from ufs:/dev/ad0s1a start_init: trying /sbin/init drm0: on vgapci0 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm1: on vgapci1 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm0: [MPSAFE] drm0: [ITHREAD] drm0: [MPSAFE] drm0: [ITHREAD] Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...0 0 0 done All buffers synced. Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Preloaded elf kernel "/boot/kernel/kernel" at 0xc0e72000. Preloaded elf module "/boot/kernel/if_ipw.ko" at 0xc0e7214c. Preloaded elf module "/boot/kernel/snd_ich.ko" at 0xc0e721f8. Preloaded elf module "/boot/kernel/sound.ko" at 0xc0e722a4. Preloaded elf module "/boot/kernel/ipw_bss.ko" at 0xc0e72350. Preloaded elf module "/boot/kernel/ipw_ibss.ko" at 0xc0e723fc. Preloaded elf module "/boot/kernel/ipw_monitor.ko" at 0xc0e724ac. Preloaded elf module "/boot/kernel/atapicam.ko" at 0xc0e7255c. Preloaded elf module "/boot/kernel/acpi.ko" at 0xc0e7260c. Calibrating clock(s) ... i8254 clock: 1193167 Hz CLK_USE_I8254_CALIBRATION not specified - using default frequency Timecounter "i8254" frequency 1193182 Hz quality 0 Calibrating TSC clock ... TSC clock: 600023372 Hz CPU: Intel(R) Pentium(R) M processor 1400MHz (600.02-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x695 Stepping = 5 Features=0xa7e9fbbf Features2=0x180 Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries Data TLB: 4 KB Pages, 4-way set associative, 128 entries Instruction TLB: 4 MB pages, fully associative, 2 entries 2nd-level cache: 1 MB, 8-way set associative, 64 byte line size 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size Data TLB: 4 MB Pages, 4-way set associative, 8 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size real memory = 527695872 (503 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009efff, 647168 bytes (158 pages) 0x0000000000100000 - 0x00000000003fffff, 3145728 bytes (768 pages) 0x0000000001028000 - 0x000000001ee22fff, 501198848 bytes (122363 pages) avail memory = 502419456 (479 MB) Table 'FACP' at 0x1f740200 Table 'OEMB' at 0x1f750040 MADT: No MADT table found APIC: Could not find any APICs. pnpbios: Found PnP BIOS data at 0xc00f2e00 pnpbios: Entry = f0000:39da Rev = 1.0 Other BIOS signatures found: wlan_amrr: wlan: <802.11 Link Layer> firmware: 'ipw_bss' version 130: 209190 bytes loaded at 0xc0d68738 firmware: 'ipw_ibss' version 130: 201138 bytes loaded at 0xc0d9d73c firmware: 'ipw_monitor' version 130: 196458 bytes loaded at 0xc0dd0748 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_buffersize=16384 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 ath_rate: version 1.2 nfslock: pseudo-device kbd: new array size 4 kbd1 at kbdmux0 io: mem: Pentium Pro MTRR support enabled null: random: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 19:59:27) ACPI: RSDP @ 0x0xf4b70/0x0014 (v 0 ACPIAM) ACPI: RSDT @ 0x0x1f740000/0x002C (v 1 A M I OEMRSDT 0x05000314 MSFT 0x00000097) ACPI: FACP @ 0x0x1f740200/0x0081 (v 2 A M I OEMFACP 0x05000314 MSFT 0x00000097) ACPI: DSDT @ 0x0x1f740300/0x7323 (v 1 0ABBD 0ABBD001 0x00000001 MSFT 0x0100000D) ACPI: FACS @ 0x0x1f750000/0x0040 ACPI: OEMB @ 0x0x1f750040/0x004D (v 1 A M I OEMBIOS 0x05000314 MSFT 0x00000097) npx0: INT 16 interface acpi0: on motherboard acpi0: [MPSAFE] acpi0: [ITHREAD] pci_open(1): mode 1 addr port (0x0cf8) is 0x8000005c pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=35808086) pcibios: No call entry point AcpiOsDerivePciId: \\_SB_.PCI0.P0P1.CBS0.CBSP -> bus 1 dev 5 func 0 acpi0: Power Button (fixed) acpi0: wakeup code va 0xccd3f000 pa 0x1000 atpic: Programming IRQ9 as level/low AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IROR -> bus 0 dev 31 func 0 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 1f700000 (3) failed ACPI timer: 1/0 1/0 1/1 1/1 1/0 1/0 1/1 1/1 1/1 1/0 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 5 6 7 11 12 Validation 0 11 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 11 12 Validation 0 255 N 0 3 4 5 6 7 11 12 After Disable 0 255 N 0 3 4 5 6 7 11 12 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 12 Validation 0 4 N 0 4 12 After Disable 0 255 N 0 4 12 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 6 Validation 0 5 N 0 5 6 After Disable 0 255 N 0 5 6 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 6 11 Validation 0 11 N 0 6 11 After Disable 0 255 N 0 6 11 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 7 Validation 0 255 N 0 3 7 After Disable 0 255 N 0 3 7 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 4 7 Validation 0 255 N 0 4 7 After Disable 0 255 N 0 4 7 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 4 N 0 4 6 12 Validation 0 4 N 0 4 6 12 After Disable 0 255 N 0 4 6 12 cpu0: on acpi0 cpu0: switching to generic Cx mode est0: on cpu0 p4tcc0: on cpu0 pcib0: port 0xcf8-0xcff on acpi0 ACPI: Found matching pin for 0.2.INTA at func 0: 11 ACPI: Found matching pin for 0.31.INTA at func 1: 255 ACPI: Found matching pin for 0.31.INTB at func 5: 255 ACPI: Found matching pin for 0.31.INTB at func 6: 255 ACPI: Found matching pin for 0.29.INTA at func 0: 11 ACPI: Found matching pin for 0.29.INTB at func 1: 5 ACPI: Found matching pin for 0.29.INTC at func 2: 4 ACPI: Found matching pin for 0.29.INTD at func 7: 4 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x3580, revid=0x02 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3584, revid=0x02 domain=0, bus=0, slot=0, func=1 class=08-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3585, revid=0x02 domain=0, bus=0, slot=0, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 27, enabled map[14]: type Memory, range 32, base 0xffa80000, size 19, enabled map[18]: type I/O Port, range 32, base 0xdc00, size 3, enabled pcib0: matched entry for 0.2.INTA (src \\_SB_.LNKA:0) pcib0: slot 2 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x3582, revid=0x02 domain=0, bus=0, slot=2, func=1 class=03-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 1 supports D0 D1 D3 current D0 map[10]: type Prefetchable Memory, range 32, base 0xe8000000, size 27, enabled map[14]: type Memory, range 32, base 0xff980000, size 19, enabled found-> vendor=0x8086, dev=0x24c2, revid=0x03 domain=0, bus=0, slot=29, func=0 class=0c-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 map[20]: type I/O Port, range 32, base 0xd480, size 5, enabled pcib0: matched entry for 0.29.INTA (src \\_SB_.LNKA:0) pcib0: slot 29 INTA routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x24c4, revid=0x03 domain=0, bus=0, slot=29, func=1 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 map[20]: type I/O Port, range 32, base 0xd800, size 5, enabled pcib0: matched entry for 0.29.INTB (src \\_SB_.LNKD:0) pcib0: slot 29 INTB routed to irq 5 via \\_SB_.LNKD found-> vendor=0x8086, dev=0x24c7, revid=0x03 domain=0, bus=0, slot=29, func=2 class=0c-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=4 map[20]: type I/O Port, range 32, base 0xd880, size 5, enabled pcib0: matched entry for 0.29.INTC (src \\_SB_.LNKC:0) pcib0: slot 29 INTC routed to irq 4 via \\_SB_.LNKC found-> vendor=0x8086, dev=0x24cd, revid=0x03 domain=0, bus=0, slot=29, func=7 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=d, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xffa7fc00, size 10, enabled pcib0: matched entry for 0.29.INTD (src \\_SB_.LNKH:0) pcib0: slot 29 INTD routed to irq 4 via \\_SB_.LNKH found-> vendor=0x8086, dev=0x2448, revid=0x83 domain=0, bus=0, slot=30, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x8080, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x06 (1500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24cc, revid=0x03 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x24ca, revid=0x03 domain=0, bus=0, slot=31, func=1 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled map[24]: type Memory, range 32, base 0, size 10, memory disabled found-> vendor=0x8086, dev=0x24c5, revid=0x03 domain=0, bus=0, slot=31, func=5 class=04-01-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe000, size 8, enabled map[14]: type I/O Port, range 32, base 0xe100, size 6, enabled map[18]: type Memory, range 32, base 0, size 9, memory disabled map[1c]: type Memory, range 32, base 0, size 8, memory disabled found-> vendor=0x8086, dev=0x24c6, revid=0x03 domain=0, bus=0, slot=31, func=6 class=07-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xe200, size 8, enabled map[14]: type I/O Port, range 32, base 0xe300, size 7, enabled pci0: at device 0.1 (no driver attached) pci0: at device 0.3 (no driver attached) vgapci0: port 0xdc00-0xdc07 mem 0xf0000000-0xf7ffffff,0xffa80000-0xffafffff irq 11 at device 2.0 on pci0 agp0: on vgapci0 vgapci0: Reserved 0x8000000 bytes for rid 0x10 type 3 at 0xf0000000 vgapci0: Reserved 0x80000 bytes for rid 0x14 type 3 at 0xffa80000 agp0: detected 8060k stolen memory agp0: aperture size is 128M vgapci1: mem 0xe8000000-0xefffffff,0xff980000-0xff9fffff at device 2.1 on pci0 uhci0: port 0xd480-0xd49f irq 11 at device 29.0 on pci0 uhci0: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd480 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd800-0xd81f irq 5 at device 29.1 on pci0 uhci1: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd800 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xd880-0xd89f irq 4 at device 29.2 on pci0 uhci2: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd880 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xffa7fc00-0xffa7ffff irq 4 at device 29.7 on pci0 ehci0: Reserved 0x400 bytes for rid 0x10 type 3 at 0xffa7fc00 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered pcib1: at device 30.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xc000-0xcfff pcib1: memory decode 0xff700000-0xff7fffff pcib1: prefetched decode 0xdea00000-0xdeafffff pcib1: Subtractively decoded bridge. ACPI: Found matching pin for 1.8.INTA at func 0: 11 ACPI: Found matching pin for 1.5.INTA at func 0: 255 ACPI: Found matching pin for 1.5.INTB at func 1: 11 ACPI: Found matching pin for 1.4.INTA at func 0: 4 pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x8086, dev=0x1043, revid=0x04 domain=0, bus=1, slot=4, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0116, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x22 (8500 ns) intpin=a, irq=4 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fd000, size 12, enabled pcib1: requested memory range 0xff7fd000-0xff7fdfff: good pcib1: matched entry for 1.4.INTA (src \\_SB_.LNKC:0) pcib1: slot 4 INTA routed to irq 4 via \\_SB_.LNKC found-> vendor=0x1180, dev=0x0475, revid=0xb8 domain=0, bus=1, slot=5, func=0 class=06-07-00, hdrtype=0x02, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x80 (32000 ns), maxlat=0x07 (1750 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0, size 12, enabled found-> vendor=0x1180, dev=0x0551, revid=0x00 domain=0, bus=1, slot=5, func=1 class=0c-00-10, hdrtype=0x00, mfdev=1 cmdreg=0x0106, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x40 (1920 ns), mingnt=0x02 (500 ns), maxlat=0x04 (1000 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xff7fe800, size 11, enabled pcib1: requested memory range 0xff7fe800-0xff7fefff: good pcib1: matched entry for 1.5.INTB (src \\_SB_.LNKA:0) pcib1: slot 5 INTB routed to irq 11 via \\_SB_.LNKA found-> vendor=0x8086, dev=0x103e, revid=0x83 domain=0, bus=1, slot=8, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0117, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xff7ff000, size 12, enabled pcib1: requested memory range 0xff7ff000-0xff7fffff: good map[14]: type I/O Port, range 32, base 0xcc00, size 6, enabled pcib1: requested I/O range 0xcc00-0xcc3f: in range pcib1: matched entry for 1.8.INTA (src \\_SB_.LNKE:0) pcib1: slot 8 INTA routed to irq 11 via \\_SB_.LNKE ipw0: mem 0xff7fd000-0xff7fdfff irq 4 at device 4.0 on pci1 ipw0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7fd000 ipw0: bpf attached ipw0: Ethernet address: 00:04:23:71:77:46 ipw0: bpf attached ipw0: bpf attached ipw0: [MPSAFE] ipw0: [ITHREAD] ipw0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps cbb0: at device 5.0 on pci1 pcib1: cbb0 requested memory range 0xff700000-0xff7fffff: good cbb0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xff700000 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pcib1: matched entry for 1.5.INTA (src \\_SB_.LNKB:0) pci_link1: Picked IRQ 9 with weight 0 pcib1: slot 5 INTA routed to irq 9 via \\_SB_.LNKB cbb0: [MPSAFE] cbb0: [ITHREAD] cbb0: PCI Configuration space: 0x00: 0x04751180 0x02100007 0x060700b8 0x00822000 0x10: 0xff700000 0x020000dc 0x20030201 0xfffff000 0x20: 0x00000000 0xfffff000 0x00000000 0xfffffffc 0x30: 0x00000000 0xfffffffc 0x00000000 0x07000109 0x40: 0x17441043 0x00000001 0x00000000 0x00000000 0x50: 0x00000000 0x00000000 0x00000000 0x00000000 0x60: 0x00000000 0x00000000 0x00000000 0x00000000 0x70: 0x00000000 0x00000000 0x00000000 0x00000000 0x80: 0x20a00001 0x00000000 0x04630463 0x00000000 0x90: 0x00000000 0x00000000 0x00000000 0x00000000 0xa0: 0x80000000 0x00000000 0x00000000 0x00000000 0xb0: 0x00000000 0x00000000 0x00000000 0x00000000 0xc0: 0x17441043 0x00000000 0x00000000 0x00000000 0xd0: 0x00000000 0x00000000 0x00000000 0xfe0a0001 0xe0: 0x24c04000 0x00000000 0x00000000 0x00000000 0xf0: 0x00000000 0x00000000 0x00000000 0x00000000 fwohci0: mem 0xff7fe800-0xff7fefff irq 11 at device 5.1 on pci1 fwohci0: Reserved 0x800 bytes for rid 0x10 type 3 at 0xff7fe800 fwohci0: [MPSAFE] fwohci0: [FILTER] fwohci0: OHCI version 1.0 (ROM=1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:e0:18:00:03:10:02:07 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:e0:18:10:02:07 fwe0: bpf attached fwe0: Ethernet address: 02:e0:18:10:02:07 fwip0: on firewire0 fwip0: bpf attached fwip0: Firewire address: 00:e0:18:00:03:10:02:07 @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x1374000 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode fxp0: port 0xcc00-0xcc3f mem 0xff7ff000-0xff7fffff irq 11 at device 8.0 on pci1 fxp0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xff7ff000 fxp0: using memory space register mapping fxp0: PCI IDs: 8086 103e 1043 1745 0083 fxp0: Dynamic Standby mode is disabled fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: fxp_miibus_readreg: timed out fxp0: MII without any PHY! device_attach: fxp0 attach returned 6 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xffa0 ata0: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0 atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6 ata0: reset tp1 mask=03 ostat0=50 ostat1=00 ata0: stat0=0x90 err=0x90 lsb=0x90 msb=0x90 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=00 devices=0x1 ata0: [MPSAFE] ata0: [ITHREAD] ata1: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170 atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376 ata1: reset tp1 mask=03 ostat0=50 ostat1=00 ata1: stat0=0x10 err=0x01 lsb=0x14 msb=0xeb ata1: stat1=0x00 err=0x01 lsb=0x7f msb=0x7f ata1: reset tp2 stat0=10 stat1=00 devices=0x4 ata1: [MPSAFE] ata1: [ITHREAD] pcm0: port 0xe000-0xe0ff,0xe100-0xe13f at device 31.5 on pci0 pcm0: Lazy allocation of 0x200 bytes rid 0x18 type 3 at 0x80000000 pcm0: Lazy allocation of 0x100 bytes rid 0x1c type 3 at 0x80000200 pcib0: matched entry for 0.31.INTB (src \\_SB_.LNKB:0) pcib0: slot 31 INTB routed to irq 9 via \\_SB_.LNKB pcm0: [MPSAFE] pcm0: [ITHREAD] pcm0: pcm0: Codec features headphone, 20 bit DAC, 20 bit ADC, 5 bit master volume, SigmaTel 3D Enhancement pcm0: Primary codec extended features variable rate PCM, reserved 1, AMAP, reserved 4 pcm0: ac97 codec dac ready count: 0 pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Mixer "speaker": pcm0: Mixer "line": pcm0: Mixer "mic": pcm0: Mixer "cd": pcm0: Mixer "rec": pcm0: Mixer "igain": pcm0: Mixer "ogain": pcm0: Mixer "line1": pcm0: Mixer "phin": pcm0: Mixer "phout": pcm0: Mixer "video": pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 1620000, 4000; 0xd4d6f000 -> 1620000 pcm0: sndbuf_setmap 162c000, 4000; 0xd4d73000 -> 162c000 pci0: at device 31.6 (no driver attached) acpi_button0: on acpi0 acpi_lid0: on acpi0 acpi_tz0: on acpi0 acpi_acad0: on acpi0 battery0: on acpi0 battery1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x41ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0-00, 2 buttons psm0: config:00000000, flags:00000008, packet size:3 psm0: syncmask:c0, syncbits:00 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: irq maps: 0 0 0 0 sio0 port 0x2f8-0x2ff irq 3 drq 1 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff unknown: status reg test failed ff ex_isa_identify() ahc_isa_probe 12: ioport 0xcc00 alloc failed ahc_isa_probe 13: ioport 0xdc00 alloc failed ahc_isa_probe 14: ioport 0xec00 alloc failed ata: ata0 already exists; skipping it ata: ata1 already exists; skipping it atkbdc: atkbdc0 already exists; skipping it sio: sio0 already exists; skipping it pnp_identify: Trying Read_Port at 203 pnp_identify: Trying Read_Port at 243 pnp_identify: Trying Read_Port at 283 pnp_identify: Trying Read_Port at 2c3 pnp_identify: Trying Read_Port at 303 pnp_identify: Trying Read_Port at 343 pnp_identify: Trying Read_Port at 383 pnp_identify: Trying Read_Port at 3c3 PNP Identify complete sc: sc0 already exists; skipping it vga: vga0 already exists; skipping it isa_probe_children: disabling PnP devices isa_probe_children: probing non-PnP devices pmtimer0 on isa0 orm0: at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 adv0: not probed (disabled) aha0: not probed (disabled) aic0: not probed (disabled) bt0: not probed (disabled) cs0: not probed (disabled) ed0: not probed (disabled) fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fe0: not probed (disabled) ie0: not probed (disabled) le0: not probed (disabled) ppc0: parallel port found at 0x378 ppc0: using extended I/O port range ppc0: ECP SPP ECP+EPP SPP ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [MPSAFE] ppbus0: [ITHREAD] plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) sio1 failed to probe at port 0x2f8 irq 3 on isa0 sio2: not probed (disabled) sio3: not probed (disabled) sn0: not probed (disabled) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 vt0: not probed (disabled) isa_probe_children: probing PnP devices Device configuration finished. procfs registered Timecounter "TSC" frequency 600023372 Hz quality 800 Timecounters tick every 1.000 msec lo0: bpf attachedfirewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) hptrr: no controller detected. acpi_acad0: acline initialization start battery0: battery initialization start battery1: battery initialization start ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire system power profile changed to 'economy' ad0: setting PIO4 on ICH4 chip acpi_acad0: Off Line acpi_acad0: acline initialization done, tried 1 times ad0: setting UDMA100 on ICH4 chip ad0: 38154MB at ata0-master UDMA100 ad0: 78140160 sectors [77520C/16H/63S] 16 sectors/interrupt 1 depth queue GEOM: new disk ad0 battery0: battery initialization done, tried 1 times ad0: Intel check1 failed ad0: Adaptec check1 failed ad0: LSI (v3) check1 failed ad0: LSI (v2) check1 failed ad0: FreeBSD check1 failed ata1-master: pio=PIO4 wdma=WDMA2 udma=UDMA33 cable=40 wire acd0: setting PIO4 on ICH4 chip acd0: setting UDMA33 on ICH4 chip acd0: DVDROM drive at ata1 as master acd0: read 4125KB/s (4125KB/s), 512KB buffer, UDMA33 acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, packet acd0: Writes: acd0: Audio: play, 16 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc pcm0: measured ac97 link rate at 48008 Hz, will use 48000 Hz acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe0:ata1:0:0:0): Down reving Protocol Version from 2 to 0? (probe0:ata1:0:0:0): error 6 (probe0:ata1:0:0:0): Unretryable Error acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata1:0:0:0): error 22 (probe0:ata1:0:0:0): Unretryable Error (probe1:sbp0:0:0:0): error 22 (probe1:sbp0:0:0:0): Unretryable Error (probe2:sbp0:0:1:0): error 22 (probe2:sbp0:0:1:0): Unretryable Error (probe3:sbp0:0:2:0): error 22 (probe3:sbp0:0:2:0): Unretryable Error (probe4:sbp0:0:3:0): error 22 (probe4:sbp0:0:3:0): Unretryable Error (probe5:sbp0:0:4:0): error 22 (probe5:sbp0:0:4:0): Unretryable Error (probe6:sbp0:0:5:0): error 22 (probe6:sbp0:0:5:0): Unretryable Error (probe7:sbp0:0:6:0): error 22 (probe7:sbp0:0:6:0): Unretryable Error pass0 at ata1 bus 0 target 0 lun 0 pass0: Removable CD-ROM SCSI-0 device pass0: 33.000MB/s transfers GEOM: new disk cd0 ATA PseudoRAID load(cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error cd0 at ata1 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present ed (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error (cd0:ata1:0:0:0): error 6 (cd0:ata1:0:0:0): Unretryable Error Trying to mount root from ufs:/dev/ad0s1a start_init: trying /sbin/init drm0: on vgapci0 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm1: on vgapci1 info: [drm] AGP at 0xf0000000 128MB info: [drm] Initialized i915 1.5.0 20060119 drm0: [MPSAFE] drm0: [ITHREAD] battery1: battery initialization failed, giving up umass0: on uhub3 umass0:3:0:-1: Attached to scbus3 (probe0:umass-sim0:0:0:0): error 22 (probe0:umass-sim0:0:0:0): Unretryable Error pass1 at umass-sim0 bus 0 target 0 lun 0 pass1: < > Removable Direct Access SCSI-2 device pass1: 40.000MB/s transfers GEOM: new disk da0 da0 at umass-sim0 bus 0 target 0 lun 0 da0: < > Removable Direct Access SCSI-2 device da0: 40.000MB/s transfers da0: 3102MB (6354432 512 byte sectors: 255H 63S/T 395C) GEOM_LABEL: Label for provider da0s1a is ufs/usbdrive. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 20:57:37 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDA37106566B; Tue, 8 Jul 2008 20:57:37 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id BF50A8FC18; Tue, 8 Jul 2008 20:57:37 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KGKBi-0005Cz-Mq; Tue, 08 Jul 2008 16:53:26 -0400 Message-ID: <4873D539.9060107@gtcomm.net> Date: Tue, 08 Jul 2008 16:59:37 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Brian McGinty References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> In-Reply-To: <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kip Macy , FreeBSD Net , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 20:57:38 -0000 But this is probably no routing table, and single source and dst ips or very limited number of ips and ports. the entire problem with Linux is the route cache, try and generate random source ips and random source/dst ports and it won't even do 100kpps without problems. I would like to log into the machine and see 1.4Mpps going through 3 nics :) Brian McGinty wrote: >> I have a pre-production card. With some bug fixes and some tuning of >> interrupt handling (custom stack - I've been asked to push the changes >> back in to CVS, I just don't have time right now) an otherwise >> unoptimized igb can forward 1.04Mpps from one port to another (1.04 >> Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 >> core system. >> > > I have a 8 core system running stock Linux that easily does line rate > (ie, 1.488 Mpps) on 3 (82575) interfaces. Ie, 3 * 1.48 Mpps! > > Cheers, > Brian. > > >> -Kip >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 21:06:19 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 915B3106567A for ; Tue, 8 Jul 2008 21:06:19 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.181]) by mx1.freebsd.org (Postfix) with ESMTP id 19D788FC17 for ; Tue, 8 Jul 2008 21:06:18 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wa-out-1112.google.com with SMTP id j4so1543589wah.3 for ; Tue, 08 Jul 2008 14:06:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=C6pSsOlCJbmxG7Z4tKFgNyBRw+nRJ9JDESmM9E1q4hw=; b=IXctnL37pgLyvrgmLEvoZAbGB3nAY3ACvFGKoTkT17ZKUAK5SM8N6bBhafMeLrMEjR CeLH/A+MuWsuePpv3j4PRaFERqNidJpjhp5N9U6cGxKrUuzonhWs+f+jy5PSMbSeWMd7 pAsIWIR00IJ78WWigCXdgDUlKqs7tWFQ1Jkwc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=ZZuHfLU6akuqt5kH2SVCJVb5yF+iHrc3WnvripTaQGYHJtLDWgRawas4WFu8R9cARl lsi7Zl5G6yuPwunFtcxc6cP+7hXqFqIUzZ8Ti930VylncgdVOqR1GuSuS6g1pJlgo5dx aUMg61y++uQgRYL/7CGoCNamiXSqkjiovGo24= Received: by 10.114.144.1 with SMTP id r1mr8475638wad.97.1215551177914; Tue, 08 Jul 2008 14:06:17 -0700 (PDT) Received: by 10.114.59.4 with HTTP; Tue, 8 Jul 2008 14:06:17 -0700 (PDT) Message-ID: Date: Tue, 8 Jul 2008 14:06:17 -0700 From: "Kip Macy" To: "Brian McGinty" In-Reply-To: <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> Cc: FreeBSD Net , Paul , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 21:06:19 -0000 On Tue, Jul 8, 2008 at 1:46 PM, Brian McGinty wrote: >> I have a pre-production card. With some bug fixes and some tuning of >> interrupt handling (custom stack - I've been asked to push the changes >> back in to CVS, I just don't have time right now) an otherwise >> unoptimized igb can forward 1.04Mpps from one port to another (1.04 >> Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 >> core system. > > I have a 8 core system running stock Linux that easily does line rate > (ie, 1.488 Mpps) on 3 (82575) interfaces. Ie, 3 * 1.48 Mpps! Hi Brian I very much doubt that this is ceteris paribus. This is 384 random IPs -> 384 random IP addresses with a flow lookup for each packet. Also, I've read through igb on Linux - it has a lot of optimizations that the FreeBSD driver lacks and I have yet to implement. Thanks, Kip From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 21:13:49 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B93C1065671 for ; Tue, 8 Jul 2008 21:13:49 +0000 (UTC) (envelope-from brian.mcginty@gmail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.168]) by mx1.freebsd.org (Postfix) with ESMTP id CBD7B8FC16 for ; Tue, 8 Jul 2008 21:13:48 +0000 (UTC) (envelope-from brian.mcginty@gmail.com) Received: by wf-out-1314.google.com with SMTP id 24so2485634wfg.7 for ; Tue, 08 Jul 2008 14:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=h4koFv8oGJ5TSOZbG8/Ly6S+sjwDstfEOGGN60eU6mg=; b=Sa+oD2NoSkwNj2RSwacy3Hr3s5l2nfzps35ee9kh5BRBUdDJrjwS5juhJ0ilomzU8x Jgv753XZI9RZI/4S8ThFNoq1WU+7JfnNA0iv/+JPAtShJl5R1T1Ex0sOQnZin9TuOBSk RD/TIk3vjSmDfzr6lWALjpLh7TCF6RDf5WqhM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=nLH0ifIiIK5iZ5d71a0J4xGESzkJLGUvI5h0smexiZmi6oRVgb9mJIJtT7shkDfpM0 mIyfnkLkJDVP1bIj3Jt6CxWgYq1k0iEJ0oc6vNejQp0n1/WAbq/oSCez2AGJH+xE3CLs FlTxgMpre7cF5LJjC4QehJwpwv1Liu3ATZEM0= Received: by 10.142.223.20 with SMTP id v20mr1970240wfg.81.1215549991747; Tue, 08 Jul 2008 13:46:31 -0700 (PDT) Received: by 10.142.188.14 with HTTP; Tue, 8 Jul 2008 13:46:31 -0700 (PDT) Message-ID: <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> Date: Tue, 8 Jul 2008 13:46:31 -0700 From: "Brian McGinty" To: "Kip Macy" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> Cc: FreeBSD Net , Paul , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2008 21:13:49 -0000 > I have a pre-production card. With some bug fixes and some tuning of > interrupt handling (custom stack - I've been asked to push the changes > back in to CVS, I just don't have time right now) an otherwise > unoptimized igb can forward 1.04Mpps from one port to another (1.04 > Mpps in on igb0 and 1.04 Mpps out on igb1) using 3.5 cores on an 8 > core system. I have a 8 core system running stock Linux that easily does line rate (ie, 1.488 Mpps) on 3 (82575) interfaces. Ie, 3 * 1.48 Mpps! Cheers, Brian. > > -Kip > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 05:30:24 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C848B106566B for ; Wed, 9 Jul 2008 05:30:24 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id 5037E8FC12 for ; Wed, 9 Jul 2008 05:30:24 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m695UETn030620 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jul 2008 15:30:16 +1000 Date: Wed, 9 Jul 2008 15:30:14 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Peter Jeremy In-Reply-To: <20080707221257.GH62764@server.vk2pj.dyndns.org> Message-ID: <20080709142008.H26105@delplex.bde.org> References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> <20080707221257.GH62764@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , Julian Elischer Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 05:30:24 -0000 On Tue, 8 Jul 2008, Peter Jeremy wrote: > On 2008-Jul-07 13:25:13 -0700, Julian Elischer wrote: >> what you need is a speculative prefetch where you an tell teh >> processor "We will probably need the following address so start >> getting it while we go do other stuff". > > This looks like the PREFETCH instructions that exist in at least amd64 > and SPARC. Unfortunately, their optimal use is very implementation- > dependent and the AMD documentation suggests that incorrect use can > degrade performance. I use the following hacks to test these in my version of bge in ~5.2: % Index: dev/bge/if_bge.c % =================================================================== % RCS file: /home/ncvs/src/sys/dev/bge/if_bge.c,v % retrieving revision 1.84 % diff -u -2 -r1.84 if_bge.c % --- dev/bge/if_bge.c 12 Mar 2005 06:51:25 -0000 1.84 % +++ dev/bge/if_bge.c 8 Jul 2008 04:49:12 -0000 % @@ -2690,4 +2845,11 @@ % */ % % +int bge_prefetch = 1; % +int bge_nprefetchnta = 0; % +int bge_nprefetch = 0x40; % +int bge_nprefetchw = 0; % +int bge_nprefetch0 = 0; % +int bge_nprefetch1 = 0; % +int bge_nprefetch2 = 0; % static void % bge_rxeof(sc) % @@ -2789,4 +2960,35 @@ % #endif % eh = mtod(m, struct ether_header *); % + if (bge_prefetch) { % + struct cl { % + char cl_data[64]; /* XXX */ % + } *clp; % + int i, j; % + % + /* XXX misalignment is likely. */ % + clp = mtod(m, struct cl *); % +#ifdef __i386__ /* XXX actually 3dnow */ % + for (i = 0, j = 0; i < bge_nprefetchnta; % + i += sizeof(*clp), j++) % + __asm("prefetchnta %0" : : "m" (clp[j])); % + for (i = 0, j = 0; i < bge_nprefetch; % + i += sizeof(*clp), j++) % + __asm("prefetch %0" : : "m" (clp[j])); % + for (i = 0, j = 0; i < bge_nprefetchw; % + i += sizeof(*clp), j++) % + __asm("prefetchw %0" : : "m" (clp[j])); % +#endif % +#ifdef __amd64__ % + for (i = 0, j = 0; i < bge_nprefetch0; % + i += sizeof(*clp), j++) % + __asm("prefetch0 %0" : : "m" (clp[j])); % + for (i = 0, j = 0; i < bge_nprefetch1; % + i += sizeof(*clp), j++) % + __asm("prefetch1 %0" : : "m" (clp[j])); % + for (i = 0, j = 0; i < bge_nprefetch2; % + i += sizeof(*clp), j++) % + __asm("prefetch2 %0" : : "m" (clp[j])); % +#endif % + } % m->m_pkthdr.len = m->m_len = cur_rx->bge_len - ETHER_CRC_LEN; % m->m_pkthdr.rcvif = ifp; % Index: net/if_ethersubr.c % =================================================================== % RCS file: /home/ncvs/src/sys/net/if_ethersubr.c,v % retrieving revision 1.174 % diff -u -2 -r1.174 if_ethersubr.c % --- net/if_ethersubr.c 24 Jun 2004 12:31:44 -0000 1.174 % +++ net/if_ethersubr.c 7 Jul 2008 18:31:13 -0000 % @@ -479,4 +479,5 @@ % * mbuf chain m with the ethernet header at the front. % */ % +int monearly = 0; % static void % ether_input(struct ifnet *ifp, struct mbuf *m) % @@ -485,4 +486,12 @@ % u_short etype; % % + if (monearly && ifp->if_flags & IFF_MONITOR) { % + /* % + * Interface marked for monitoring; discard packet. % + */ % + m_freem(m); % + return; % + } % + % /* % * Do consistency checks to verify assumptions The results were underwhelming and contrary to Andre's assertion that the primary bottleneck (apart from PCI32) is hardware-related cache misses (I think it is software-related cache misses). I previously reported that fixing monitor mode avoids 1 cache miss and thus saves 5% CPU. Plain prefetch forces this cache miss (but no other hardware-related ones, since there are no other hardware-related ones in upper layers) to occur asynchronously and always occur. However, it only saves 2% in unfixed normal mode and in unfixed monitor mode (in fixed monitor mode, it makes little difference except to not avoid the cache miss -- since the cache miss is asynchronous it doesn't affect %CPU much). Even 5% is a relatively uninteresting savings, since the non-hardware related CPU overhead is 10 times as much as that. I'm testing only receive of udp packets with a payload of 5 bytes (padded), so the whole packet fits in 64 bytes and there is only 1 hardware-related cache miss per packet to avoid or prefetch. The precise size is 60 (64 - CRC_size I think). m->m_data is always misaligned at an offset of 2 bytes from a 64-byte cache line boundary, prefetching 64 bytes at this address is not quite right, but since the 60 bytes all fit in 1 cache line, the prefetch fetches enough. prefetchnta as in Andre's old patch (16 Dec 2004) didn't seem to work. I also prefetch as soon as possible in the driver interrupt handler where Andre's old patch prefetches in ether_input() where this is almost certainly too late. The difference between the 5% and the 2% saings may be due to it also being too late in the driver interrupt handler. Someone mentioned not caring about latency. Doing something else to wait for all the prefetches made by the interrupt handler to complete might help here, but only if you could find something useful to do (hard), and I think latency would just increase the slowness in most cases since significant latency would require long queues and the long queues would bust caches (starting with discarding all the prefetches). Andre's old patch uses a hard-coded prefetch size of 74 (76 after source alignment and 128 after rounding up) where mine uses a parameter of 64 (66 after virtual source alignment and 64 after rounding down). This would cause an unnecessary extra cache miss for small packets. It too only tries to prefetch the packet header, but allows for tcp and tcp options so a small packet's headers alone are larger than 64 bytes. The extra cache miss for never-accessed data shouldn' cost much since it uses prefetchnta. (All of my tests are on an Athlon64 where prefetchnta actually works, unlike on AthlonXP. But actually working might be responsible for it not being very effective here. To work, it must not be too aggressive or it will cost too much for never-accessed data.) Timings (some repeated), all for ttcp receiving on bge0 at 397 kpps: -monitor: 35% idle (8.0-CURRENT) 14 cm/p monitor: 83% idle (8.0-CURRENT) 6 cm/p +monitor: 85% idle (8.0-CURRENT) 5 cm/p -monitor: 17% idle (~5.2) 19 cm/p 17-19 monitor: 66% idle (~5.2) 8 cm/p 66-68 +monitor: 71% idle (~5.2) 7 cm/p 70-75 cm/p = k8-dc-misses (bge0 system) +monitor is monitor mode with the exit moved to the top of ether_input(). Patch for ~5.2 now included. Results with prefetch not actually shown above since I forgot half of the details. cm/p was unchanged except for +monitor it is increased (by the unused prefetch). %idle decreased by 1-2% (less in -current where there is less slop) except for +monitor. Note that -current has many improvements over ~5.2 in both %CPU and cache misses for receiving. But for sending, -current gives a 10% lower rate for the same CPU (100%) though it reduces cache misses. Simplified or improved patches for -current: % diff -c2 ./dev/bge/if_bge.c~ ./dev/bge/if_bge.c % *** ./dev/bge/if_bge.c~ Fri May 16 16:39:01 2008 % --- ./dev/bge/if_bge.c Tue Jul 8 07:58:52 2008 % *************** % *** 3017,3020 **** % --- 3133,3137 ---- % */ % % + int bge_prefetch = 1; % static void % bge_rxeof(struct bge_softc *sc) % *************** % *** 3126,3129 **** % --- 3252,3257 ---- % m->m_pkthdr.len = m->m_len = cur_rx->bge_len - ETHER_CRC_LEN; % m->m_pkthdr.rcvif = ifp; % + if (bge_prefetch) % + __asm("prefetch %0" : : "m" (*mtod(m, char *))); % % if (ifp->if_capenable & IFCAP_RXCSUM) { % diff -c2 ./net/if_ethersubr.c~ ./net/if_ethersubr.c % *** ./net/if_ethersubr.c~ Fri May 16 16:41:45 2008 % --- ./net/if_ethersubr.c Tue Jul 8 07:55:14 2008 % *************** % *** 509,512 **** % --- 507,511 ---- % * mbuf chain m with the ethernet header at the front. % */ % + int broken_monitor = 0; % static void % ether_input(struct ifnet *ifp, struct mbuf *m) % *************** % *** 546,550 **** % } % eh = mtod(m, struct ether_header *); % - etype = ntohs(eh->ether_type); % if (m->m_pkthdr.rcvif == NULL) { % if_printf(ifp, "discard frame w/o interface pointer\n"); % --- 545,548 ---- % *************** % *** 560,564 **** % #endif % % ! if (ETHER_IS_MULTICAST(eh->ether_dhost)) { % if (ETHER_IS_BROADCAST(eh->ether_dhost)) % m->m_flags |= M_BCAST; % --- 558,564 ---- % #endif % % ! if (((ifp->if_flags & IFF_MONITOR) == 0 || broken_monitor) && % ! ETHER_IS_MULTICAST(eh->ether_dhost)) { % ! /* XXX bpf might need this even in monitor mode. */ % if (ETHER_IS_BROADCAST(eh->ether_dhost)) % m->m_flags |= M_BCAST; % *************** % *** 616,619 **** % --- 616,620 ---- % * TODO: Deal with Q-in-Q frames, but not arbitrary nesting levels. % */ % + etype = ntohs(eh->ether_type); % if ((m->m_flags & M_VLANTAG) == 0 && etype == ETHERTYPE_VLAN) { % struct ether_vlan_header *evl; Bruce From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 08:50:30 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91418106568E; Wed, 9 Jul 2008 08:50:30 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx07.syd.optusnet.com.au (fallbackmx07.syd.optusnet.com.au [211.29.132.9]) by mx1.freebsd.org (Postfix) with ESMTP id 2C2148FC44; Wed, 9 Jul 2008 08:50:30 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by fallbackmx07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m656gD4e024931; Sat, 5 Jul 2008 16:42:13 +1000 Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m656g9vR017221 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 5 Jul 2008 16:42:11 +1000 Date: Sat, 5 Jul 2008 16:42:09 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: John Baldwin In-Reply-To: <200807041748.m64HmZur018637@svn.freebsd.org> Message-ID: <20080705161831.F13262@delplex.bde.org> References: <200807041748.m64HmZur018637@svn.freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: net@freebsd.org Subject: Re: svn commit: r180256 - head/sys/dev/arl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 08:50:30 -0000 On Fri, 4 Jul 2008, John Baldwin wrote: > Author: jhb > Date: Fri Jul 4 17:48:34 2008 > New Revision: 180256 > URL: http://svn.freebsd.org/changeset/base/180256 > > Log: > Make arl(4) MPSAFE: > ... > - ifp->if_snd.ifq_maxlen = IFQ_MAXLEN; > + IFQ_SET_MAXLEN(&ifp->if_snd, IFQ_MAXLEN); Why do we obfuscate setting of ifq_maxlen using a macro, especially when the setting is to a wrong default value? The macro was introduced with ALTQ changes, but seems to have never done anything different for ALTQ. ALTQ also introduced an ifq_drv_maxlen field, but the macro provides no help for managing this. Drivers that support ALTQ end up with 2 settings of ifq_*maxlen, one direct one for ifq_drv_maxlen and one obfuscated one of ifq_maxlen. arl apparently doesn't support ALTQ, and you didn't fix this -- it still doesn't set ifq_drv_maxlen. if_attach() uses the correct default value of ifqmaxlen if the driver leaves ifp->if_snd.ifq_maxlen set to 0, but prints a bogus warning about this. Non-driver code under net/ still mostly doesn't use the obfuscation, but uses IFQ_MAXLEN and ifqmaxlen almost perfectly randomly to have about 50% of each. Since ifqmaxlen isn't a tuneable or sysctl, and is statically initialized to IFQ_MAXLEN, not using only makes a difference if someone iniitalizes it diffently using a debugger, so these bugs are normally just spelling errors. IFQ_MAXLEN is also too small for 1Gbps or even 100Nbps hardware devices, so only drivers for old hardware and some software drivers can use it anyway. Bruce From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 12:14:20 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85CE31065671 for ; Wed, 9 Jul 2008 12:14:20 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5B7438FC1E for ; Wed, 9 Jul 2008 12:14:20 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 9C59946C4B; Wed, 9 Jul 2008 08:14:19 -0400 (EDT) Date: Wed, 9 Jul 2008 13:14:19 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20080705161831.F13262@delplex.bde.org> Message-ID: <20080709131101.S8639@fledge.watson.org> References: <200807041748.m64HmZur018637@svn.freebsd.org> <20080705161831.F13262@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: John Baldwin , net@freebsd.org Subject: Re: svn commit: r180256 - head/sys/dev/arl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 12:14:20 -0000 On Sat, 5 Jul 2008, Bruce Evans wrote: > On Fri, 4 Jul 2008, John Baldwin wrote: Since ifqmaxlen isn't a tuneable or > sysctl, and is statically initialized to IFQ_MAXLEN, not using only makes a > difference if someone iniitalizes it diffently using a debugger, so these > bugs are normally just spelling errors. IFQ_MAXLEN is also too small for > 1Gbps or even 100Nbps hardware devices, so only drivers for old hardware and > some software drivers can use it anyway. I was actually thinking about this this morning -- Paul Saab pointed out to me that, on Linux, you can run-time tune the transmit queue limit using ifconfig(8). I think doing something similar would, if nothing else, make it easier to understand the impact of our current queue settings in testing. And, just to put it on the table in e-mail, since I know it has come up a lot at developer summits: the ALTQ infrastructure is decreasingly compatible with current network devices, which often have quite large queues (descriptor rings) in hardware, or where there are multiple transmit queues. One possibility I've been considering is making the whole ifq subsystem a library to device drivers, rather than a required interface to transmit. This would allow the device driver to instantiate more than one if there are multiple hardware queues that need to be represented, or, for example, allow synthetic encapsulation interfaces (such as vlan) to avoid queueing entirely and directly dispatch to the lower layer interface without requiring a mandatory enqueue/dequeue step. I've started hacking on this every now and then, but it requires a lot of code to be touched -- it's something we do need to address before 8.0, however. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 14:33:52 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A35C31065671 for ; Wed, 9 Jul 2008 14:33:52 +0000 (UTC) (envelope-from sclark46@earthlink.net) Received: from elasmtp-dupuy.atl.sa.earthlink.net (elasmtp-dupuy.atl.sa.earthlink.net [209.86.89.62]) by mx1.freebsd.org (Postfix) with ESMTP id 760C48FC2C for ; Wed, 9 Jul 2008 14:33:52 +0000 (UTC) (envelope-from sclark46@earthlink.net) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=bRFyh/J1EqOREbsGtF/0bXnX+sj6/NOXPqcNpBMhsAcBnDxSDFPlJr8KFNtNbR5z; h=Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [24.144.77.185] (helo=joker.seclark.com) by elasmtp-dupuy.atl.sa.earthlink.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KGaOU-0003je-Ov for freebsd-net@freebsd.org; Wed, 09 Jul 2008 10:11:42 -0400 Message-ID: <4874C71D.5000204@earthlink.net> Date: Wed, 09 Jul 2008 10:11:41 -0400 From: Stephen Clark User-Agent: Thunderbird 2.0.0.14 (X11/20080501) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ELNK-Trace: a437fbc6971e80f61aa676d7e74259b7b3291a7d08dfec79ddf1086972d61ebe0525a9644b6e934c350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 24.144.77.185 Subject: 6.3-p2 gre X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sclark46@earthlink.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 14:33:52 -0000 Hello List, I am running ospf over a gre/vpn tunnel. When I run tcpdump on the gre interface ospf stops working. I see the following errors in the ospfd log. 2008/07/09 10:05:02 OSPF: *** sendmsg in ospf_write failed to 224.0.0.5, id 0, off 0, len 68, interface gre1, mtu 1412: Network is down 2008/07/09 10:05:12 OSPF: *** sendmsg in ospf_write failed to 224.0.0.5, id 0, off 0, len 68, interface gre1, mtu 1412: Network is down if I then do ifconfig gre1 down;ifconfig gre1 up ospf start workding again. this does not happen on freebsd 4.x Also if I use the -p flag when running tcpdump things seem to be OK and ospf continues to work normally. Why would putting the gre interface in promiscuous mode cause this problem? Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 15:13:10 2008 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87AD5106568A; Wed, 9 Jul 2008 15:13:10 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.freebsd.org (Postfix) with ESMTP id 23BC98FC19; Wed, 9 Jul 2008 15:13:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m69FD558018806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 10 Jul 2008 01:13:08 +1000 Date: Thu, 10 Jul 2008 01:13:05 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Robert Watson In-Reply-To: <20080709131101.S8639@fledge.watson.org> Message-ID: <20080710002759.Q27395@delplex.bde.org> References: <200807041748.m64HmZur018637@svn.freebsd.org> <20080705161831.F13262@delplex.bde.org> <20080709131101.S8639@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: John Baldwin , net@FreeBSD.org Subject: Re: svn commit: r180256 - head/sys/dev/arl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 15:13:10 -0000 On Wed, 9 Jul 2008, Robert Watson wrote: > On Sat, 5 Jul 2008, Bruce Evans wrote: > >> On Fri, 4 Jul 2008, John Baldwin wrote: Since ifqmaxlen isn't a tuneable or >> sysctl, and is statically initialized to IFQ_MAXLEN, not using only makes a >> difference if someone iniitalizes it diffently using a debugger, so these >> bugs are normally just spelling errors. IFQ_MAXLEN is also too small for >> 1Gbps or even 100Nbps hardware devices, so only drivers for old hardware >> and some software drivers can use it anyway. > > I was actually thinking about this this morning -- Paul Saab pointed out to > me that, on Linux, you can run-time tune the transmit queue limit using > ifconfig(8). I think doing something similar would, if nothing else, make it > easier to understand the impact of our current queue settings in testing. Yes, the control should really be per-device. However, I don't like the bloat for dynamic everything in every driver. However2, I use a hack (a per-driver global possibly-set by ddb at boot time) to optionally enlarge the tx queue for all drivers that I touch. It was in editing this and having to change it for ALTQ and its unnecessary macro that I noticed the bogusness if IFQ_MAXLEN and the ALTQ macro. > And, just to put it on the table in e-mail, since I know it has come up a lot > at developer summits: the ALTQ infrastructure is decreasingly compatible with > current network devices, which often have quite large queues (descriptor > rings) in hardware, or where there are multiple transmit queues. One Hardware queues are never large :-). 512 is common, but enlargement gives ~20000. 20000 is too large for most purposes but rarely matters. I don't use ALTQ, and just notice that very rarely, latency can be enormous if the queue length builds up to he maximum. > possibility I've been considering is making the whole ifq subsystem a library > to device drivers, rather than a required interface to transmit. This would > allow the device driver to instantiate more than one if there are multiple > hardware queues that need to be represented, or, for example, allow synthetic > encapsulation interfaces (such as vlan) to avoid queueing entirely and > directly dispatch to the lower layer interface without requiring a mandatory > enqueue/dequeue step. I've started hacking on this every now and then, but > it requires a lot of code to be touched -- it's something we do need to > address before 8.0, however. Could this be more efficient? I think direct dispatch wouldn't work well. It didn't help as much as hoped for rx, and tx is predictable so perfect scheduling of it is possible (only dispatch in bulk in order to be more efficient). Also, the current implementation gives necessary watermark stuff almost automatically -- the queue split gives a virtual low watermark at the split point, and this reduces the chance of the combined queue running dry. Bruce From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 15:22:34 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EF2D1065677 for ; Wed, 9 Jul 2008 15:22:34 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from thor-new.fsklaw.com (thor-new.fsklaw.com [64.174.116.34]) by mx1.freebsd.org (Postfix) with ESMTP id 574018FC25 for ; Wed, 9 Jul 2008 15:22:34 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from localhost (localhost [127.0.0.1]) by thor-new.fsklaw.com (Postfix) with ESMTP id 852DB16BC48B; Wed, 9 Jul 2008 08:22:33 -0700 (PDT) Received: from thor-new.fsklaw.com ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 05590-10; Wed, 9 Jul 2008 08:22:30 -0700 (PDT) Received: from cor (unknown [192.168.61.119]) by thor-new.fsklaw.com (Postfix) with ESMTP id E74FB16BC45B; Wed, 9 Jul 2008 08:22:26 -0700 (PDT) Received: from 192.168.62.153 (SquirrelMail authenticated user zaphod) by cor with HTTP; Wed, 9 Jul 2008 08:21:06 -0700 (PDT) Message-ID: <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> In-Reply-To: <200807040155.m641tl8s000607@lava.sentex.ca> References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> Date: Wed, 9 Jul 2008 08:21:06 -0700 (PDT) From: zaphod@fsklaw.com To: "Mike Tancsa" , freebsd-net@freebsd.org User-Agent: SquirrelMail/1.4.15 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: by amavisd-new at fsklaw.com Cc: Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 15:22:34 -0000 > At 03:15 PM 7/3/2008, zaphod@fsklaw.com wrote: >>I have a real poser, and I ccan't solve it. >> >>Currently I have a ipsec vpn tunneling 14 servers through a central >> server. >> >>I would like to restructure this so that each server talks to each other >>directly, rather than passing everything through a single server. >> >>However, on every other machine I cannot get a second tunnel to come up. >>Not a gre or gif tunnel. And yet I have 14 on the central machine. > > You would need a lot of policies on each of the boxes (14) but there > is no reason it should not work. Do each of the sites have a unique > subnet ? Do they have static IP addresses ? > > > An easier solution might be to use something like OpenVPN which > allows all the boxes to auth and route through a single server, but > they can also talk to each other with a single config option. > > ---Mike Mike, thanks for the response. I agree it should work. But it's not. With respect to the next two questions, yes and yes. I'm not a huge fan of OpenVPN, but the bigger issue is that the gif tunnels come up at boot up. As well as routes. Given the client server nature of OpenVPN it is suitable, because if a server reboots, I'm not certain a client would auto re-connect. But I have done no testing. And If I can't reesolve this I may have to. Cheers, Zaphod > > > From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 15:45:41 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4692B106566B for ; Wed, 9 Jul 2008 15:45:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 1053D8FC21 for ; Wed, 9 Jul 2008 15:45:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m69Fjcxr017720; Wed, 9 Jul 2008 11:45:39 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m69FjcP4031350 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jul 2008 11:45:38 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807091545.m69FjcP4031350@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 09 Jul 2008 11:45:35 -0400 To: zaphod@fsklaw.com, freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 15:45:41 -0000 At 11:21 AM 7/9/2008, zaphod@fsklaw.com wrote: >I agree it should work. But it's not. With respect to the next two >questions, yes and yes. Can you post some of the configs you are using for 3 of the sites so we can perhaps spot the problem(s) you are having ? I have a similar setup with 5 sites, all talking to each other via IPSEC tunnels. Its a lot of policies, but they work just fine. >I'm not a huge fan of OpenVPN, but the bigger issue is that the gif >tunnels come up at boot up. As well as routes. Given the client server >nature of OpenVPN it is suitable, because if a server reboots, I'm not >certain a client would auto re-connect. We have ~ 400 sites running OpenVPN across Canada that all reconnect just fine after reboots / power cycles etc. We dont let the clients talk to each other, but that would just be a config change to allow that to work. ---Mike From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 16:50:50 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A01141065671 for ; Wed, 9 Jul 2008 16:50:50 +0000 (UTC) (envelope-from sclark46@earthlink.net) Received: from elasmtp-dupuy.atl.sa.earthlink.net (elasmtp-dupuy.atl.sa.earthlink.net [209.86.89.62]) by mx1.freebsd.org (Postfix) with ESMTP id 571168FC0A for ; Wed, 9 Jul 2008 16:50:50 +0000 (UTC) (envelope-from sclark46@earthlink.net) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=LCVU9JlP8wV2LrKhYrrgbNBxec+4tnb3C8TlM2dPtza9/Gkvy8VpOAqSFLszCsU7; h=Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [24.144.77.185] (helo=joker.seclark.com) by elasmtp-dupuy.atl.sa.earthlink.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KGcsS-00010T-Qu; Wed, 09 Jul 2008 12:50:48 -0400 Message-ID: <4874EC67.6020104@earthlink.net> Date: Wed, 09 Jul 2008 12:50:47 -0400 From: Stephen Clark User-Agent: Thunderbird 2.0.0.14 (X11/20080501) MIME-Version: 1.0 To: Mike Tancsa References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> In-Reply-To: <200807091545.m69FjcP4031350@lava.sentex.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ELNK-Trace: a437fbc6971e80f61aa676d7e74259b7b3291a7d08dfec79a31246b52a858aa729f8df4fb8dad033350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 24.144.77.185 Cc: freebsd-net@freebsd.org, zaphod@fsklaw.com Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sclark46@earthlink.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 16:50:50 -0000 Mike Tancsa wrote: > At 11:21 AM 7/9/2008, zaphod@fsklaw.com wrote: > >> I agree it should work. But it's not. With respect to the next two >> questions, yes and yes. > > Can you post some of the configs you are using for 3 of the sites so we > can perhaps spot the problem(s) you are having ? I have a similar setup > with 5 sites, all talking to each other via IPSEC tunnels. Its a lot of > policies, but they work just fine. > > > > >> I'm not a huge fan of OpenVPN, but the bigger issue is that the gif >> tunnels come up at boot up. As well as routes. Given the client server >> nature of OpenVPN it is suitable, because if a server reboots, I'm not >> certain a client would auto re-connect. > > We have ~ 400 sites running OpenVPN across Canada that all reconnect > just fine after reboots / power cycles etc. We dont let the clients > talk to each other, but that would just be a config change to allow that > to work. > > ---Mike > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Hi, I do this also - having both multiple gre/vpn tunnels to do ospf. Using freebsd 4.x and 6.1 Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 17:17:33 2008 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 408BF106564A; Wed, 9 Jul 2008 17:17:33 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 13ABB8FC0A; Wed, 9 Jul 2008 17:17:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 7929146C66; Wed, 9 Jul 2008 13:17:32 -0400 (EDT) Date: Wed, 9 Jul 2008 18:17:32 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20080710002759.Q27395@delplex.bde.org> Message-ID: <20080709181531.Q8639@fledge.watson.org> References: <200807041748.m64HmZur018637@svn.freebsd.org> <20080705161831.F13262@delplex.bde.org> <20080709131101.S8639@fledge.watson.org> <20080710002759.Q27395@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: John Baldwin , net@FreeBSD.org Subject: Re: svn commit: r180256 - head/sys/dev/arl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 17:17:33 -0000 On Thu, 10 Jul 2008, Bruce Evans wrote: >> possibility I've been considering is making the whole ifq subsystem a >> library to device drivers, rather than a required interface to transmit. >> This would allow the device driver to instantiate more than one if there >> are multiple hardware queues that need to be represented, or, for example, >> allow synthetic encapsulation interfaces (such as vlan) to avoid queueing >> entirely and directly dispatch to the lower layer interface without >> requiring a mandatory enqueue/dequeue step. I've started hacking on this >> every now and then, but it requires a lot of code to be touched -- it's >> something we do need to address before 8.0, however. > > Could this be more efficient? > > I think direct dispatch wouldn't work well. It didn't help as much as hoped > for rx, and tx is predictable so perfect scheduling of it is possible (only > dispatch in bulk in order to be more efficient). Also, the current > implementation gives necessary watermark stuff almost automatically -- the > queue split gives a virtual low watermark at the split point, and this > reduces the chance of the combined queue running dry. In most cases, what I have in mind would simply be a rearrangement rather than a functional change. However, for vlans, I think it would significantly lower overhead without really modifying queueing behavior: notice that we enqueue it at the VLAN layer just to dequeue it a few instructions later so that we can enqueue it a layer lower. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 17:31:38 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B82731065678 for ; Wed, 9 Jul 2008 17:31:38 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from thor-new.fsklaw.com (thor-new.fsklaw.com [64.174.116.34]) by mx1.freebsd.org (Postfix) with ESMTP id 7BA148FC16 for ; Wed, 9 Jul 2008 17:31:38 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from localhost (localhost [127.0.0.1]) by thor-new.fsklaw.com (Postfix) with ESMTP id D7E8516C1AE1; Wed, 9 Jul 2008 10:31:37 -0700 (PDT) Received: from thor-new.fsklaw.com ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08065-08; Wed, 9 Jul 2008 10:31:34 -0700 (PDT) Received: from cor (unknown [192.168.61.119]) by thor-new.fsklaw.com (Postfix) with ESMTP id E267F16C1AB4; Wed, 9 Jul 2008 10:31:33 -0700 (PDT) Received: from 192.168.62.153 (SquirrelMail authenticated user zaphod) by cor with HTTP; Wed, 9 Jul 2008 10:30:09 -0700 (PDT) Message-ID: In-Reply-To: <200807091545.m69FjcP4031350@lava.sentex.ca> References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> Date: Wed, 9 Jul 2008 10:30:09 -0700 (PDT) From: zaphod@fsklaw.com To: "Mike Tancsa" , freebsd-net@freebsd.org User-Agent: SquirrelMail/1.4.15 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: by amavisd-new at fsklaw.com Cc: Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 17:31:38 -0000 > At 11:21 AM 7/9/2008, zaphod@fsklaw.com wrote: > >>I agree it should work. But it's not. With respect to the next two >>questions, yes and yes. > > Can you post some of the configs you are using for 3 of the sites so > we can perhaps spot the problem(s) you are having ? I have a similar > setup with 5 sites, all talking to each other via IPSEC tunnels. Its > a lot of policies, but they work just fine. > > > > >>I'm not a huge fan of OpenVPN, but the bigger issue is that the gif >>tunnels come up at boot up. As well as routes. Given the client server >>nature of OpenVPN it is suitable, because if a server reboots, I'm not >>certain a client would auto re-connect. > > We have ~ 400 sites running OpenVPN across Canada that all reconnect > just fine after reboots / power cycles etc. We dont let the clients > talk to each other, but that would just be a config change to allow > that to work. > > ---Mike > Last first. Well that's good info on OpenVPN. As to the first, I'm not even at the ipsec stage yet. I'm just trying to get tunnels up. I wrote a couple of shell scripts to bring them up for testing. Server1 orange# more mkgif #/bin/sh ifconfig gif1 create ifconfig gif1 1.1.1.1 2.2.2.2 ifconfig gif1 inet 192.168.72.1 192.168.70.1 netmask 255.255.255.0 ifconfig gif1 tunnel 1.1.1.1 2.2.2.2 ifconfig gif1 mtu 1500 route change 192.168.70.0 192.168.70.1 255.255.255.0 route change 192.168.71.0 192.168.70.1 255.255.255.0 Server2 to# more mkgif #/bin/sh ifconfig gif1 create ifconfig gif1 2.2.2.2 1.1.1.1 ifconfig gif1 inet 192.168.70.1 192.168.72.1 netmask 255.255.255.0 ifconfig gif1 tunnel 2.2.2.2 1.1.1.1 ifconfig gif1 mtu 1500 route change 192.168.72.0 192.168.72.1 255.255.255.0 Seems pretty straight forward a tunnel. But nothing heads out. Can't ping a thing. I even tried a gre, when I did that I got a ping error. Unfortunately I can't find my note on the exact error. Cheers, Zaphod > > From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 17:49:21 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B80901065675 for ; Wed, 9 Jul 2008 17:49:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outO.internet-mail-service.net (outo.internet-mail-service.net [216.240.47.238]) by mx1.freebsd.org (Postfix) with ESMTP id 9A2668FC19 for ; Wed, 9 Jul 2008 17:49:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 383DF23F8; Wed, 9 Jul 2008 10:49:22 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id BFA082D6022; Wed, 9 Jul 2008 10:49:19 -0700 (PDT) Message-ID: <4874FA1F.40209@elischer.org> Date: Wed, 09 Jul 2008 10:49:19 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: zaphod@fsklaw.com References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Mike Tancsa Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 17:49:21 -0000 zaphod@fsklaw.com wrote: >> At 11:21 AM 7/9/2008, zaphod@fsklaw.com wrote: >> >>> I agree it should work. But it's not. With respect to the next two >>> questions, yes and yes. >> Can you post some of the configs you are using for 3 of the sites so >> we can perhaps spot the problem(s) you are having ? I have a similar >> setup with 5 sites, all talking to each other via IPSEC tunnels. Its >> a lot of policies, but they work just fine. >> >> >> >> >>> I'm not a huge fan of OpenVPN, but the bigger issue is that the gif >>> tunnels come up at boot up. As well as routes. Given the client server >>> nature of OpenVPN it is suitable, because if a server reboots, I'm not >>> certain a client would auto re-connect. >> We have ~ 400 sites running OpenVPN across Canada that all reconnect >> just fine after reboots / power cycles etc. We dont let the clients >> talk to each other, but that would just be a config change to allow >> that to work. >> >> ---Mike >> > Last first. Well that's good info on OpenVPN. > > As to the first, I'm not even at the ipsec stage yet. I'm just trying to > get tunnels up. I wrote a couple of shell scripts to bring them up for > testing. > > Server1 > > orange# more mkgif > #/bin/sh > ifconfig gif1 create > ifconfig gif1 1.1.1.1 2.2.2.2 ^^^^ what's that for? since you over-ride it in the next line vvvvv > ifconfig gif1 inet 192.168.72.1 192.168.70.1 netmask 255.255.255.0 (PTP links don't have netmasks) > ifconfig gif1 tunnel 1.1.1.1 2.2.2.2 > ifconfig gif1 mtu 1500 > route change 192.168.70.0 192.168.70.1 255.255.255.0 > route change 192.168.71.0 192.168.70.1 255.255.255.0 > > Server2 > to# more mkgif > #/bin/sh > ifconfig gif1 create > ifconfig gif1 2.2.2.2 1.1.1.1 > ifconfig gif1 inet 192.168.70.1 192.168.72.1 netmask 255.255.255.0 > ifconfig gif1 tunnel 2.2.2.2 1.1.1.1 > ifconfig gif1 mtu 1500 > route change 192.168.72.0 192.168.72.1 255.255.255.0 > > Seems pretty straight forward a tunnel. But nothing heads out. Can't ping > a thing. > > I even tried a gre, when I did that I got a ping error. Unfortunately I > can't find my note on the exact error. > > Cheers, > > Zaphod >> > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 18:04:35 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2ACDD1065673 for ; Wed, 9 Jul 2008 18:04:35 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id C87438FC16 for ; Wed, 9 Jul 2008 18:04:34 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m69I4WXj050236; Wed, 9 Jul 2008 14:04:32 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m69I4VOh031916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jul 2008 14:04:31 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807091804.m69I4VOh031916@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 09 Jul 2008 14:04:29 -0400 To: zaphod@fsklaw.com, freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 18:04:35 -0000 At 01:30 PM 7/9/2008, zaphod@fsklaw.com wrote: >Seems pretty straight forward a tunnel. But nothing heads out. Can't ping >a thing. I think your tunnel endpoints are overlapping your remote subnets. The GIF tunnel IP addresses are not supposed to be on the same internal LAN. If server 1's public IP is 1.1.1.1 and server 2 is 2.2.2.2 and server1's internet network is 192.168.1.0/24 and server2's inside network is 192.168.2.0/24 This should work. #!/bin/sh #server1 to connect to server2 MEOUTSIDE=1.1.1.1 MEINSIDE=10.10.69.1 REMOTEOUTSIDE=2.2.2.2 REMOTEINSIDE=10.10.69.2 REMOTENET=192.168.2.0/24 /sbin/ifconfig gif1 create tunnel $MEOUTSIDE $REMOTEOUTSIDE /sbin/ifconfig gif1 $MEINSIDE netmask 255.255.255.252 $REMOTEINSIDE /sbin/route delete $REMOTENET /sbin/route add $REMOTENET $REMOTEINSIDE #!/bin/sh #server2 script to connect to server1 MEOUTSIDE=2.2.2.2 MEINSIDE=10.10.69.2 REMOTEOUTSIDE=1.1.1.1 REMOTEINSIDE=10.10.69.1 REMOTENET=192.168.1.0/24 /sbin/ifconfig gif1 create tunnel $MEOUTSIDE $REMOTEOUTSIDE /sbin/ifconfig gif1 $MEINSIDE netmask 255.255.255.252 $REMOTEINSIDE /sbin/route delete $REMOTENET /sbin/route add $REMOTENET $REMOTEINSIDE Also, dont confuse using GIF and IPSEC. To create some IPSEC tunnels, you dont need gif or gre interfaces. The policies will do that for you. ---Mike >Server1 > >orange# more mkgif >#/bin/sh >ifconfig gif1 create >ifconfig gif1 1.1.1.1 2.2.2.2 >ifconfig gif1 inet 192.168.72.1 192.168.70.1 netmask 255.255.255.0 >ifconfig gif1 tunnel 1.1.1.1 2.2.2.2 >ifconfig gif1 mtu 1500 >route change 192.168.70.0 192.168.70.1 255.255.255.0 >route change 192.168.71.0 192.168.70.1 255.255.255.0 > >Server2 >to# more mkgif >#/bin/sh >ifconfig gif1 create >ifconfig gif1 2.2.2.2 1.1.1.1 >ifconfig gif1 inet 192.168.70.1 192.168.72.1 netmask 255.255.255.0 >ifconfig gif1 tunnel 2.2.2.2 1.1.1.1 >ifconfig gif1 mtu 1500 >route change 192.168.72.0 192.168.72.1 255.255.255.0 From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 18:04:57 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E718106567A for ; Wed, 9 Jul 2008 18:04:57 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from thor-new.fsklaw.com (thor-new.fsklaw.com [64.174.116.34]) by mx1.freebsd.org (Postfix) with ESMTP id 1C3178FC17 for ; Wed, 9 Jul 2008 18:04:57 +0000 (UTC) (envelope-from zaphod@fsklaw.com) Received: from localhost (localhost [127.0.0.1]) by thor-new.fsklaw.com (Postfix) with ESMTP id E2E5316C35E7; Wed, 9 Jul 2008 11:04:56 -0700 (PDT) Received: from thor-new.fsklaw.com ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08995-05; Wed, 9 Jul 2008 11:04:53 -0700 (PDT) Received: from cor (unknown [192.168.61.119]) by thor-new.fsklaw.com (Postfix) with ESMTP id 0F7C916C35B6; Wed, 9 Jul 2008 11:04:53 -0700 (PDT) Received: from 192.168.62.153 (SquirrelMail authenticated user zaphod) by cor with HTTP; Wed, 9 Jul 2008 11:03:28 -0700 (PDT) Message-ID: <3d2c56c963f5fc5f6732548548068f69.squirrel@cor> In-Reply-To: <4874FA1F.40209@elischer.org> References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> <4874FA1F.40209@elischer.org> Date: Wed, 9 Jul 2008 11:03:28 -0700 (PDT) From: zaphod@fsklaw.com To: "Julian Elischer" User-Agent: SquirrelMail/1.4.15 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: by amavisd-new at fsklaw.com Cc: freebsd-net@freebsd.org, zaphod@fsklaw.com, Mike Tancsa Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 18:04:57 -0000 > zaphod@fsklaw.com wrote: >>> At 11:21 AM 7/9/2008, zaphod@fsklaw.com wrote: >>> >>>> I agree it should work. But it's not. With respect to the next two >>>> questions, yes and yes. >>> Can you post some of the configs you are using for 3 of the sites so >>> we can perhaps spot the problem(s) you are having ? I have a similar >>> setup with 5 sites, all talking to each other via IPSEC tunnels. Its >>> a lot of policies, but they work just fine. >>> >>> >>> >>> >>>> I'm not a huge fan of OpenVPN, but the bigger issue is that the gif >>>> tunnels come up at boot up. As well as routes. Given the client >>>> server >>>> nature of OpenVPN it is suitable, because if a server reboots, I'm not >>>> certain a client would auto re-connect. >>> We have ~ 400 sites running OpenVPN across Canada that all reconnect >>> just fine after reboots / power cycles etc. We dont let the clients >>> talk to each other, but that would just be a config change to allow >>> that to work. >>> >>> ---Mike >>> >> Last first. Well that's good info on OpenVPN. >> >> As to the first, I'm not even at the ipsec stage yet. I'm just trying >> to >> get tunnels up. I wrote a couple of shell scripts to bring them up for >> testing. >> >> Server1 >> >> orange# more mkgif >> #/bin/sh >> ifconfig gif1 create >> ifconfig gif1 1.1.1.1 2.2.2.2 > > ^^^^ what's that for? Well added that as I was googling the problem someone had said to do it so I tried it. Wasn't there initially. Doesn't work with or without. > since you over-ride it in the next line vvvvv > > >> ifconfig gif1 inet 192.168.72.1 192.168.70.1 netmask 255.255.255.0 > > (PTP links don't have netmasks) > snip: Got it from the manual # ifconfig gif0 create # ifconfig gif0 tunnel A.B.C.D W.X.Y.Z # ifconfig gif0 inet 192.168.1.1 192.168.2.1 netmask 0xffffffff I'll try it without. Cheers, Zaphod From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 18:26:46 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C565A1065678 for ; Wed, 9 Jul 2008 18:26:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 87CF38FC30 for ; Wed, 9 Jul 2008 18:26:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m69IQivq055384; Wed, 9 Jul 2008 14:26:44 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m69IQiKR032020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jul 2008 14:26:44 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807091826.m69IQiKR032020@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 09 Jul 2008 14:26:41 -0400 To: zaphod@fsklaw.com, freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: <7.1.0.9.0.20080709133535.2396cea8@sentex.net> References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> <7.1.0.9.0.20080709133535.2396cea8@sentex.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 18:26:46 -0000 At 02:04 PM 7/9/2008, Mike Tancsa wrote: >Also, dont confuse using GIF and IPSEC. To create some IPSEC >tunnels, you dont need gif or gre interfaces. The policies will do >that for you. Here is a simple example that just uses IPSEC tunnels with a static key. You dont need any gif/gre stuff. Dont use this in production, use IPSEC-TOOLS from the ports to do dynamic keying. To test the tunnel, assuming the inside interface of the freebsd boxes are .1 ping -S 192.168.1.1 192.168.1.2 #/bin/sh server1 MEOUTSIDE=1.1.1.1 MEINSIDE=192.168.1.0/24 REMOTEOUTSIDE=2.2.2.2 REMOTEINSIDE=192.168.5.0/24 IPSECKEY=ZA6PkrlNH6BN11SG1rCa8dxa setkey -c < Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B32CA106564A; Wed, 9 Jul 2008 18:50:33 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 156868FC1D; Wed, 9 Jul 2008 18:50:33 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m69IoWe1062333; Wed, 9 Jul 2008 18:50:32 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m69IoWJX062329; Wed, 9 Jul 2008 18:50:32 GMT (envelope-from linimon) Date: Wed, 9 Jul 2008 18:50:32 GMT Message-Id: <200807091850.m69IoWJX062329@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/125442: CARP combined with LAGG causes system panic - V7.0 RELEASE#2.0/amd64 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 18:50:33 -0000 Synopsis: CARP combined with LAGG causes system panic - V7.0 RELEASE#2.0/amd64 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Jul 9 18:48:59 UTC 2008 Responsible-Changed-Why: Over to maintainer(s). Note to submitter: we are going to need the stack trace. http://www.freebsd.org/cgi/query-pr.cgi?pr=125442 From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 21:26:54 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C909B1065671 for ; Wed, 9 Jul 2008 21:26:54 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id A2FC28FC14 for ; Wed, 9 Jul 2008 21:26:54 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id m69LQqjQ076213 for ; Wed, 9 Jul 2008 14:26:54 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id m69LPqfd086756 for ; Wed, 9 Jul 2008 14:25:52 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (209.249.190.254.available.above.net [209.249.190.254] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id m69LPqgm040183 for ; Wed, 9 Jul 2008 14:25:52 -0700 (PDT) (envelope-from gnn@neville-neil.com) Date: Wed, 09 Jul 2008 17:25:51 -0400 Message-ID: From: gnn@freebsd.org To: net@freebsd.org User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.1.50 (i386-apple-darwin8.11.1) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Canit-CHI2: 0.50 X-Bayes-Prob: 0.5 (Score 0, tokens from: ) X-Spam-Score: 0.10 () [Tag at 5.00] COMBINED_FROM X-CanItPRO-Stream: default X-Canit-Stats-ID: 902236 - a23f3521c173 X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: Subject: What's the deal with hardware checksum and net.inet.udp.checksum? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2008 21:26:54 -0000 I would assume that if a card, say the em, has hardware TX checksum that the UDP checksum could be calculated by the hardware, but this seems not to be the case. The manual pages are unhelpful in this regard. Thanks, George From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 10:43:23 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E734B106567D; Thu, 10 Jul 2008 10:43:23 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id C7BC98FC2C; Thu, 10 Jul 2008 10:43:23 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 7615E46BB4; Thu, 10 Jul 2008 06:43:23 -0400 (EDT) Date: Thu, 10 Jul 2008 11:43:23 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: gnn@freebsd.org In-Reply-To: Message-ID: <20080710114028.T34050@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: net@freebsd.org Subject: Re: What's the deal with hardware checksum and net.inet.udp.checksum? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 10:43:24 -0000 On Wed, 9 Jul 2008, gnn@freebsd.org wrote: > I would assume that if a card, say the em, has hardware TX checksum that the > UDP checksum could be calculated by the hardware, but this seems not to be > the case. The manual pages are unhelpful in this regard. On the whole, they should be generated in hardware as long as it's not administratively disabled with ifconfig, and as long as there aren't know bugs in the hardware for the rev you're using. Just for example, hardware checksumming is disabled in software for quite a few early 1gbps cards due to bugs in the hardware causing rather nasty side effects. What specific problem are you seeing? We do do a software checksum of the pseudo-header, but the UDP data should be checksummed by hardware. (The usual test for hardware checksum being enabled on transmit is to tcpdump the interface and see tcpdump reporting lots of bad checksums, as the BPF capture happens before hardware checksumming is run -- in principle on the receive side that shouldn't happen!) Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 12:25:05 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0970B1065672 for ; Thu, 10 Jul 2008 12:25:05 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 994F18FC1F for ; Thu, 10 Jul 2008 12:25:04 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: (qmail 39203 invoked by uid 89); 10 Jul 2008 12:27:45 -0000 Received: from unknown (HELO ?IPv6:2607:f118::5?) (steve@ibctech.ca@2607:f118::5) by 2607:f118::b6 with ESMTPA; 10 Jul 2008 12:27:45 -0000 Message-ID: <4875FFA1.9010608@ibctech.ca> Date: Thu, 10 Jul 2008 08:25:05 -0400 From: Steve Bertrand User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: zaphod@fsklaw.com References: <8f7879db41dbaecc479a017110e8f32f.squirrel@cor> <200807040155.m641tl8s000607@lava.sentex.ca> <7904ac587e71a42fb86c2bbe77bde0ae.squirrel@cor> <200807091545.m69FjcP4031350@lava.sentex.ca> <4874FA1F.40209@elischer.org> <3d2c56c963f5fc5f6732548548068f69.squirrel@cor> In-Reply-To: <3d2c56c963f5fc5f6732548548068f69.squirrel@cor> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Julian Elischer , Mike Tancsa Subject: Re: Tunneling issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 12:25:05 -0000 zaphod@fsklaw.com wrote: >>> ifconfig gif1 inet 192.168.72.1 192.168.70.1 netmask 255.255.255.0 Above you are assigning a /24 netmask. > Got it from the manual > > > # ifconfig gif0 create > # ifconfig gif0 tunnel A.B.C.D W.X.Y.Z > # ifconfig gif0 inet 192.168.1.1 192.168.2.1 netmask 0xffffffff In your example from the manual, it is applying a /32 netmask ie: 255.255.255.255. Aside from that, do you see ICMP inbound via tcpdump on the machine that you are trying to ping? Is the traffic making it to the destination, but not the return trip? Steve From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 14:54:14 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47F73106566B for ; Thu, 10 Jul 2008 14:54:14 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: from web76615.mail.sg1.yahoo.com (web76615.mail.sg1.yahoo.com [124.108.123.71]) by mx1.freebsd.org (Postfix) with SMTP id A06E08FC1B for ; Thu, 10 Jul 2008 14:54:13 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: (qmail 52671 invoked by uid 60001); 10 Jul 2008 14:27:29 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.ph; h=Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=GBv1NMJ2SxOJ2HXKP0+1PbU6/PSzsYY7TQVegCbh1S9Z9uYrC8W/VSpHFXYYN29v2YkLhdLIhGaipucG95NGCnrYQnQzlIj+8KlcBkLU2GRxIyMj39sj1fDpqqShiov/fYosFs22qTHE9WS8VqZYj1kkz+JKCs4RK/62kmZwYoQ=; Received: from [58.71.34.137] by web76615.mail.sg1.yahoo.com via HTTP; Thu, 10 Jul 2008 22:27:28 SGT X-Mailer: YahooMailWebService/0.7.199 Date: Thu, 10 Jul 2008 22:27:28 +0800 (SGT) From: Archimedes Gaviola To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID: <44627.52285.qm@web76615.mail.sg1.yahoo.com> Subject: [Regarding Packet Error] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: a_gaviola@yahoo.com.ph List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 14:54:14 -0000 Hi, I'm running a FreeBSD-6.2 RELEASE system on an Intel Gigabit copper NIC=20 but when it receive packets, I encounter some errors with netstat. Below is the netstat dump of my system. As what you can see, there is no dropping of packets received but only errors at the input level of the interface. Does anyone have any idea of what's going on at the entry of the packets? What could be the possible causes of packet errors? Or what network data has=20 been collected in the netstat when it displays error? Can someone=20 explained how FreeBSD process packets when it enters the network system in the kernel? This is to know how my system behaves before doing any step for troubleshooting. # netstat -I em0 -w 1 -d input (em0) output packets errs bytes packets errs bytes colls drops 16233 1410 24570946 13785 0 965516 0 0 16065 1262 24318048 13210 0 930566 0 0 19091 1152 28897958 15965 0 1119340 0 0 16105 1341 24380062 13827 0 975404 0 0 15999 1242 24215216 14073 0 991380 0 0 16314 1526 24695034 13531 0 946110 0 0 16319 1134 24699760 12704 0 886526 0 0 16320 1253 24698795 13752 0 966382 0 0 15900 1467 24065394 13529 0 953136 0 0 16014 1285 24239412 13018 0 912128 0 0 16219 1253 24549782 13003 0 911444 0 0 16205 1245 24528586 13504 0 947206 0 0 16003 1430 24222790 14052 0 997390 0 0 16035 1381 24269784 12641 0 881562 0 0 15909 1165 24074754 13608 0 956804 0 0 15961 1511 24151964 13501 0 953862 0 0 16217 1285 24542454 13245 0 925960 0 0 16308 1272 24683106 12444 0 861054 0 0 16323 1004 24707389 12764 0 888100 0 0 16148 1096 24442256 12663 0 888614 0 0 16534 1202 25026660 13348 0 941722 0 0 Thank you. =0A=0A=0A Get your preferred Email name!=0ANow you can @ymail.com and = @rocketmail.com. =0Ahttp://mail.promotions.yahoo.com/newdomains/ph/ From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 15:46:47 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DF621065672 for ; Thu, 10 Jul 2008 15:46:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 243DE8FC1A for ; Thu, 10 Jul 2008 15:46:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.2/8.14.2) with ESMTP id m6AFkjLF051183; Thu, 10 Jul 2008 11:46:45 -0400 (EDT) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id m6AFkiAw036967 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 10 Jul 2008 11:46:44 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200807101546.m6AFkiAw036967@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 10 Jul 2008 11:46:43 -0400 To: a_gaviola@yahoo.com.ph, freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: <44627.52285.qm@web76615.mail.sg1.yahoo.com> References: <44627.52285.qm@web76615.mail.sg1.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: Re: [Regarding Packet Error] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 15:46:47 -0000 At 10:27 AM 7/10/2008, Archimedes Gaviola wrote: >Hi, > >I'm running a FreeBSD-6.2 RELEASE system on an Intel Gigabit copper NIC >but when it receive packets, There have been a number of bug fixes to the intel nics since 6.2R. Can you at least update to 6.3R or RELENG_6 ? Also, check the stats for your nics sysctl dev.em.0.stats=1 it will dump them to syslog Does the switchport your are connected to show any errors ? ---Mike From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 19:54:00 2008 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B280106566C; Thu, 10 Jul 2008 19:54:00 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id 111608FC1C; Thu, 10 Jul 2008 19:54:00 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id m6AJrx2e038753; Thu, 10 Jul 2008 12:53:59 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id m6AJqC1Z059439; Thu, 10 Jul 2008 12:52:13 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (209.249.190.254.available.above.net [209.249.190.254] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id m6AJqCTN035108; Thu, 10 Jul 2008 12:52:12 -0700 (PDT) (envelope-from gnn@neville-neil.com) Date: Thu, 10 Jul 2008 15:52:11 -0400 Message-ID: From: gnn@FreeBSD.org To: Robert Watson In-Reply-To: <20080710114028.T34050@fledge.watson.org> References: <20080710114028.T34050@fledge.watson.org> User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.1.50 (i386-apple-darwin8.11.1) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Canit-CHI2: 0.50 X-Bayes-Prob: 0.5 (Score 0, tokens from: ) X-Spam-Score: 0.40 () [Tag at 5.00] COMBINED_FROM,PORN_RP_NASTY X-CanItPRO-Stream: default X-Canit-Stats-ID: 911374 - e7c0111dbdb5 X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: net@FreeBSD.org Subject: Re: What's the deal with hardware checksum and net.inet.udp.checksum? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 19:54:00 -0000 At Thu, 10 Jul 2008 11:43:23 +0100 (BST), rwatson wrote: > > On Wed, 9 Jul 2008, gnn@freebsd.org wrote: > > > I would assume that if a card, say the em, has hardware TX checksum that the > > UDP checksum could be calculated by the hardware, but this seems not to be > > the case. The manual pages are unhelpful in this regard. > > On the whole, they should be generated in hardware as long as it's > not administratively disabled with ifconfig, and as long as there > aren't know bugs in the hardware for the rev you're using. Just for > example, hardware checksumming is disabled in software for quite a > few early 1gbps cards due to bugs in the hardware causing rather > nasty side effects. What specific problem are you seeing? We do do > a software checksum of the pseudo-header, but the UDP data should be > checksummed by hardware. > > (The usual test for hardware checksum being enabled on transmit is > to tcpdump the interface and see tcpdump reporting lots of bad > checksums, as the BPF capture happens before hardware checksumming > is run -- in principle on the receive side that shouldn't happen!) > If the sysctl it turned off on the transmitter then the receiving machine sees UDP checksums of 0. Best, George From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 21:06:33 2008 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 002171065672; Thu, 10 Jul 2008 21:06:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id CF22E8FC1F; Thu, 10 Jul 2008 21:06:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 647D446B49; Thu, 10 Jul 2008 17:06:32 -0400 (EDT) Date: Thu, 10 Jul 2008 22:06:32 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: gnn@FreeBSD.org In-Reply-To: Message-ID: <20080710220201.K34050@fledge.watson.org> References: <20080710114028.T34050@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: net@FreeBSD.org Subject: Re: What's the deal with hardware checksum and net.inet.udp.checksum? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 21:06:33 -0000 On Thu, 10 Jul 2008, gnn@FreeBSD.org wrote: > If the sysctl it turned off on the transmitter then the receiving machine > sees UDP checksums of 0. Right. If you disable UDP checksumming, we don't generate checksums (hardware or software) in udp_output(): /* * Set up checksum and output datagram. */ if (udp_cksum) { if (inp->inp_flags & INP_ONESBCAST) faddr.s_addr = INADDR_BROADCAST; ui->ui_sum = in_pseudo(ui->ui_src.s_addr, faddr.s_addr, htons((u_short)len + sizeof(struct udphdr) + IPPROTO_UDP)); m->m_pkthdr.csum_flags = CSUM_UDP; m->m_pkthdr.csum_data = offsetof(struct udphdr, uh_sum); } else ui->ui_sum = 0; You can disable hardware checksums using the -txcsum flag on ifconfig for each interface -- once the above-generated mbuf header gets to the IP layer and the route out an interface is available, we on-demand generate checksum in software if hardware checksums aren't available or are administratively disabled. Vis ip_output(): m->m_pkthdr.csum_flags |= CSUM_IP; sw_csum = m->m_pkthdr.csum_flags & ~ifp->if_hwassist; if (sw_csum & CSUM_DELAY_DATA) { in_delayed_cksum(m); sw_csum &= ~CSUM_DELAY_DATA; } m->m_pkthdr.csum_flags &= ifp->if_hwassist; It's possible to imagine adding a global sysctl that has slightly different policy implications, such as globally disabling hardware checksums, or not generating full checksums if the interface doesn't support hardware checksums rather than generating them. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 05:12:08 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5335106564A for ; Fri, 11 Jul 2008 05:12:08 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 8BDF18FC1A for ; Fri, 11 Jul 2008 05:12:08 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-197-4.hsd1.fl.comcast.net ([76.108.197.4] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KHAr0-00010l-M0; Fri, 11 Jul 2008 01:07:34 -0400 Message-ID: <4876EC16.5070209@gtcomm.net> Date: Fri, 11 Jul 2008 01:13:58 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Bruce Evans References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> <20080707221257.GH62764@server.vk2pj.dyndns.org> <20080709142008.H26105@delplex.bde.org> In-Reply-To: <20080709142008.H26105@delplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Peter Jeremy , Julian Elischer , FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 05:12:08 -0000 I tested Linux in bridge configuration with the same machine and it CPUed out at about 600kpps through the bridge.. That's a bit low :/ Soft interrupt using all the cpu. Same opteron 2222, 82571EB Pci express NIC. Tried SMP/ non-smp , load balanced irqs, etc.. Good news is using iptables only adds a few percentage onto the CPU usage. But still, what's with that.. So far FreeBSD got the highest pps rating for forwarding. I haven't tried bridge mode. Ipfw probably takes a big hit in that too though. Looking for an 82575 to test.. Paul From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 05:35:26 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB48C1065679 for ; Fri, 11 Jul 2008 05:35:26 +0000 (UTC) (envelope-from artis.caune@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.240]) by mx1.freebsd.org (Postfix) with ESMTP id 98FAB8FC1A for ; Fri, 11 Jul 2008 05:35:26 +0000 (UTC) (envelope-from artis.caune@gmail.com) Received: by an-out-0708.google.com with SMTP id b33so875744ana.13 for ; Thu, 10 Jul 2008 22:35:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=RpAz93Rrqu61YrWK6ZWxpN+MuoEkcXLbXWgilbNQoCU=; b=SnRbmNwG8yXruJpZtzF0aXn4bz251CR9REp2V3I0u8XIWkTC4aXcAB6j3BAug79Mlc 1Tpc2zRm7BF6PyNLIlj8X/PVZkIi7HD9jpyMm8Mcr1SLnU7OvtGjLW/8DGiKi8O3/gX2 0zie3xpy7ZsY6LV1YMvErDcGAzLGEXbwFCrto= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=q66qB9LAzbmwtQFUUPPim8/2z/iYhFy+4O730sXwKOumOGEkYKKEpEo4B5rvxRZfo7 Z5SDQzhO8NGkK6x9hq50/0rdccKw8AJPp8cQevPWTH9VxfZ94dB2w05ZZKCXvgvOACRj V6RmgC4WIuI1V5X+QwaNsLpqqdxyyGkgP+6oA= Received: by 10.100.95.19 with SMTP id s19mr7876980anb.65.1215752852296; Thu, 10 Jul 2008 22:07:32 -0700 (PDT) Received: by 10.100.253.17 with HTTP; Thu, 10 Jul 2008 22:07:32 -0700 (PDT) Message-ID: <9e20d71e0807102207s7cbebb8fiad965ef46eaf84d@mail.gmail.com> Date: Fri, 11 Jul 2008 08:07:32 +0300 From: "Artis Caune" To: a_gaviola@yahoo.com.ph In-Reply-To: <44627.52285.qm@web76615.mail.sg1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <44627.52285.qm@web76615.mail.sg1.yahoo.com> Cc: freebsd-net@freebsd.org Subject: Re: [Regarding Packet Error] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 05:35:26 -0000 On Thu, Jul 10, 2008 at 5:27 PM, Archimedes Gaviola wrote: > Hi, > > I'm running a FreeBSD-6.2 RELEASE system on an Intel Gigabit copper NIC > but when it receive packets, I encounter some errors with netstat. Below is > the netstat dump of my system. As what you can see, there is no dropping of > packets received but only errors at the input level of the interface. Does > anyone have any idea of what's going on at the entry of the packets? What > could be the possible causes of packet errors? Or what network data has > been collected in the netstat when it displays error? Can someone > explained how FreeBSD process packets when it enters the network system > in the kernel? This is to know how my system behaves before doing any step > for troubleshooting. have you enabled polling on interface? try to disable if yes. is it pci or pci64/x interface? try to set the following variables in /boot/loader.conf and reboot: hw.em.rxd=4096 hw.em.txd=4096 From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 07:47:06 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19BEF1065675 for ; Fri, 11 Jul 2008 07:47:06 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: from web76608.mail.sg1.yahoo.com (web76608.mail.sg1.yahoo.com [124.108.123.64]) by mx1.freebsd.org (Postfix) with SMTP id 727DE8FC37 for ; Fri, 11 Jul 2008 07:47:05 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: (qmail 2190 invoked by uid 60001); 11 Jul 2008 07:47:03 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.ph; h=Received:X-Mailer:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=Fk9FAz/DFet4F4zKdwR7NZdRIf3OesYR79Xscft5v3GP3lefmlsOHaGfoNgCiVvF3Flrtjns1gWyaukQCO2zlqowrqlsweNrTDVT6vejlY/OzvIwsH+MCsrF4RQkBXNJLP3ichJ0SMCxpxkXGXIqjv5xO7YfCxWoT/2x14jcrZk=; Received: from [58.71.34.138] by web76608.mail.sg1.yahoo.com via HTTP; Fri, 11 Jul 2008 15:47:02 SGT X-Mailer: YahooMailWebService/0.7.199 Date: Fri, 11 Jul 2008 15:47:02 +0800 (SGT) From: Archimedes Gaviola To: freebsd-net@freebsd.org, Mike Tancsa In-Reply-To: <200807101546.m6AFkiAw036967@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID: <948463.97751.qm@web76608.mail.sg1.yahoo.com> Cc: Subject: Re: [Regarding Packet Error] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: a_gaviola@yahoo.com.ph List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 07:47:06 -0000 --- On Thu, 7/10/08, Mike Tancsa wrote: > From: Mike Tancsa > Subject: Re: [Regarding Packet Error] > To: a_gaviola@yahoo.com.ph, freebsd-net@freebsd.org > Date: Thursday, 10 July, 2008, 11:46 PM > At 10:27 AM 7/10/2008, Archimedes Gaviola wrote: > >Hi, > > > >I'm running a FreeBSD-6.2 RELEASE system on an > Intel Gigabit copper NIC > >but when it receive packets, >=20 > There have been a number of bug fixes to the intel nics > since=20 > 6.2R. Can you at least update to 6.3R or RELENG_6 ? =20 [Archimedes] Yes, I'll try newer versions of FreeBSD with 6.3 RELEASE and 7= .0 RELEASE if errors will still occur. Also, > check the=20 > stats for your nics >=20 > sysctl dev.em.0.stats=3D1 >=20 > it will dump them to syslog [Archimedes] Here's the log. Jul 11 07:01:18 freebsd62 kernel: em0: Excessive collisions =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Sequence errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Defer count =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Missed Packets =3D 294797 Jul 11 07:01:18 freebsd62 kernel: em0: Receive No Buffers =3D 1377122 Jul 11 07:01:18 freebsd62 kernel: em0: Receive Length Errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Receive errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Crc errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Alignment errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: Carrier extension errors =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: RX overruns =3D 24092 Jul 11 07:01:18 freebsd62 kernel: em0: watchdog timeouts =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: XON Rcvd =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: XON Xmtd =3D 26434 Jul 11 07:01:18 freebsd62 kernel: em0: XOFF Rcvd =3D 0 Jul 11 07:01:18 freebsd62 kernel: em0: XOFF Xmtd =3D 319731 Jul 11 07:01:18 freebsd62 kernel: em0: Good Packets Rcvd =3D 10486744 Jul 11 07:01:18 freebsd62 kernel: em0: Good Packets Xmtd =3D 7311565 >=20 > Does the switchport your are connected to show any errors ? >=20 [Archimedes] No, it doesn't show any errors on my Gigabit LinkSys switch. Thanks, Archimedes =0A=0A=0A Yahoo! Toolbar is now powered with Search Assist.Download it= now!=0Ahttp://ph.toolbar.yahoo.com/ From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 08:04:07 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89FE2106567F for ; Fri, 11 Jul 2008 08:04:07 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: from web76615.mail.sg1.yahoo.com (web76615.mail.sg1.yahoo.com [124.108.123.71]) by mx1.freebsd.org (Postfix) with SMTP id E36C48FC1B for ; Fri, 11 Jul 2008 08:04:06 +0000 (UTC) (envelope-from a_gaviola@yahoo.com.ph) Received: (qmail 89288 invoked by uid 60001); 11 Jul 2008 08:04:04 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.ph; h=Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=b9dJuTCBsYL+vkuOi2xrofd54HM4OftA08dz937l+V2ZNAKMfSraw+jwZ0R59VGcwQg6lXjuO7cOdvD6mg1JAOiZV6C8uaiPbabbz0SLOEMRl9AiwqTt22dS4FnTWELGcIuDHe7R8Lz2/n8Gj5+PbkchDZ5cDuD6yv5BD9Qpjqc=; Received: from [58.71.34.137] by web76615.mail.sg1.yahoo.com via HTTP; Fri, 11 Jul 2008 16:04:03 SGT X-Mailer: YahooMailWebService/0.7.199 Date: Fri, 11 Jul 2008 16:04:03 +0800 (SGT) From: Archimedes Gaviola To: Artis Caune In-Reply-To: <9e20d71e0807102207s7cbebb8fiad965ef46eaf84d@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID: <212607.89062.qm@web76615.mail.sg1.yahoo.com> Cc: freebsd-net@freebsd.org Subject: Re: [Regarding Packet Error] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: a_gaviola@yahoo.com.ph List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 08:04:07 -0000 --- On Fri, 7/11/08, Artis Caune wrote: > From: Artis Caune > Subject: Re: [Regarding Packet Error] > To: a_gaviola@yahoo.com.ph > Cc: freebsd-net@freebsd.org > Date: Friday, 11 July, 2008, 1:07 PM > On Thu, Jul 10, 2008 at 5:27 PM, Archimedes Gaviola > wrote: > > Hi, > > > > I'm running a FreeBSD-6.2 RELEASE system on an > Intel Gigabit copper NIC > > but when it receive packets, I encounter some errors > with netstat. Below is > > the netstat dump of my system. As what you can see, > there is no dropping of > > packets received but only errors at the input level of > the interface. Does > > anyone have any idea of what's going on at the > entry of the packets? What > > could be the possible causes of packet errors? Or what > network data has > > been collected in the netstat when it displays error? > Can someone > > explained how FreeBSD process packets when it enters > the network system > > in the kernel? This is to know how my system behaves > before doing any step > > for troubleshooting. >=20 > have you enabled polling on interface? try to disable if > yes. [Archimedes] No, device polling is not enabled to my system. > is it pci or pci64/x interface? [Archimedes] This is an Intel PCI-X copper NIC http://www.intel.com/network= /connectivity/resources/doc_library/data_sheets/pro1000mt_sa.pdf =20 >=20 > try to set the following variables in /boot/loader.conf and > reboot: > hw.em.rxd=3D4096 > hw.em.txd=3D4096 [Archimedes] Yes, I need to try these options also.=20 Thanks, Archimedes=0A=0A=0A Yahoo! Toolbar is now powered with Search Assist.D= ownload it now!=0Ahttp://ph.toolbar.yahoo.com/ From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 10:09:21 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23F3D1065670 for ; Fri, 11 Jul 2008 10:09:20 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id AC2F58FC0C for ; Fri, 11 Jul 2008 10:09:20 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 51B661B10EE8; Fri, 11 Jul 2008 12:09:19 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on malcho.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.6 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.2.4 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 204561B10EE0; Fri, 11 Jul 2008 12:09:17 +0200 (CEST) Message-ID: <4877314C.5000106@moneybookers.com> Date: Fri, 11 Jul 2008 13:09:16 +0300 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.14 (X11/20080616) MIME-Version: 1.0 To: Paul References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> <20080707221257.GH62764@server.vk2pj.dyndns.org> <20080709142008.H26105@delplex.bde.org> <4876EC16.5070209@gtcomm.net> In-Reply-To: <4876EC16.5070209@gtcomm.net> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.93, clamav-milter version 0.93 on blah.cmotd.com X-Virus-Status: Clean Cc: Peter Jeremy , Julian Elischer , FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 10:09:21 -0000 Hi Paul, Paul wrote: > I tested Linux in bridge configuration with the same machine and it > CPUed out at about 600kpps through the bridge.. 600kpps incoming or 600kpps incoming+ outgoing ? > That's a bit low :/ Soft interrupt using all the cpu. Same opteron > 2222, 82571EB Pci express NIC. > Tried SMP/ non-smp , load balanced irqs, etc.. Does hwpmc work out of the box (FreeBSD) with those CPUs? > > Good news is using iptables only adds a few percentage onto the CPU > usage. But still, what's with that.. > So far FreeBSD got the highest pps rating for forwarding. I haven't > tried bridge mode. Ipfw probably takes a big hit in that too though. > > Looking for an 82575 to test.. P.S. It was a nice chat, but what we can expect from the future? Any plans, patches etc? Someone suggested to install 8-current and test with it as this is the "fast" way to have something included in FreeBSD. I can do this - I can install 8-current, patch it and put it under load and report results, but need patches :) I guess Paul is in the same situation .. -- Best Wishes, Stefan Lambrev ICQ# 24134177 From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 15:15:08 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0150C106564A for ; Fri, 11 Jul 2008 15:15:08 +0000 (UTC) (envelope-from bart@it-ss.be) Received: from authrelay.it-ss.be (authrelay.it-ss.be [195.28.164.225]) by mx1.freebsd.org (Postfix) with ESMTP id 72EFB8FC1C for ; Fri, 11 Jul 2008 15:15:07 +0000 (UTC) (envelope-from bart@it-ss.be) Received: from bartwrkstxp (178.146-136-217.adsl-dyn.isp.belgacom.be [217.136.146.178]) (authenticated bits=0) by mx10.it-ss.be (8.13.6/8.13.6) with ESMTP id m6BFEvTX015922; Fri, 11 Jul 2008 17:14:59 +0200 Message-ID: <003d01c8e368$e3213f50$020b000a@bartwrkstxp> From: "Bart Van Kerckhove" To: "Stefan Lambrev" , "Paul" References: <2d3001c8def1$f4309b90$020b000a@bartwrkstxp> <486FFF70.3090402@gtcomm.net> <48701921.7090107@gtcomm.net> <4871E618.1080500@freebsd.org> <20080708002228.G680@besplex.bde.org> <48724238.2020103@freebsd.org> <20080708034304.R21502@delplex.bde.org> <20080708045135.V1022@besplex.bde.org> <48727BA9.6020702@elischer.org> <20080707221257.GH62764@server.vk2pj.dyndns.org> <20080709142008.H26105@delplex.bde.org><4876EC16.5070209@gtcomm.net> <4877314C.5000106@moneybookers.com> Date: Fri, 11 Jul 2008 17:12:10 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Scanned-By: MIMEDefang 2.63 on 195.28.164.225 Cc: Peter Jeremy , Julian Elischer , FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 15:15:08 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> Good news is using iptables only adds a few percentage onto the CPU >> usage. But still, what's with that.. >> So far FreeBSD got the highest pps rating for forwarding. I haven't >> tried bridge mode. Ipfw probably takes a big hit in that too though. >> Looking for an 82575 to test.. > > P.S. It was a nice chat, but what we can expect from the future? Any > plans, patches etc? > Someone suggested to install 8-current and test with it as this is the > "fast" way to have something included in FreeBSD. > I can do this - I can install 8-current, patch it and put it under > load and report results, but need patches :) > I guess Paul is in the same situation .. I'm in the same situation as well. Would anyone be interested in very specific work aimed at improving IP forwarding? I would happily put out a bounty for this, and I'm quite sure I'm not alone. PS Paul: idd you get around to testing C2D ? Kind regards, Met vriendelijke groet / With kind regards, Bart Van Kerckhove http://friet.net/pgp.txt "There are 10 kinds of ppl; those who read binary and those who don't" -----BEGIN PGP SIGNATURE----- iQA/AwUBSHdqOQoIFchBM0BKEQJSPQCfQKKgD8+xrX088+o0IKmPDdDD0XoAnAv+ SqgNdjkKsEstDYqnFDNUQuK3 =ft58 -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 20:18:05 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 155841065674; Fri, 11 Jul 2008 20:18:05 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 157C48FC1F; Fri, 11 Jul 2008 20:18:05 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (gavin@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m6BKI4ZM086740; Fri, 11 Jul 2008 20:18:04 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m6BKI4oo086736; Fri, 11 Jul 2008 20:18:04 GMT (envelope-from gavin) Date: Fri, 11 Jul 2008 20:18:04 GMT Message-Id: <200807112018.m6BKI4oo086736@freefall.freebsd.org> To: chelton30@gmail.com, gavin@FreeBSD.org, freebsd-i386@FreeBSD.org, freebsd-net@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/125502: [ral] ifconfig ral0 scan produces no output unless in shared mode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 20:18:05 -0000 Old Synopsis: ifconfig ral0 scan produces no output New Synopsis: [ral] ifconfig ral0 scan produces no output unless in shared mode State-Changed-From-To: open->feedback State-Changed-By: gavin State-Changed-When: Fri Jul 11 20:11:15 UTC 2008 State-Changed-Why: To submitter: We'll probably need more details from you before there's any chance of diagnosing the problem. To start with, could you try using the wlandebug tool, from /usr/src/tools/tools/net80211 and see if that reveals anything obvious? # wlandebug -i ral0 +scan+auth+debug+assoc net.wlan.0.debug: 0 => 0xc80000 Responsible-Changed-From-To: freebsd-i386->freebsd-net Responsible-Changed-By: gavin Responsible-Changed-When: Fri Jul 11 20:11:15 UTC 2008 Responsible-Changed-Why: Over to maintainer(s) http://www.freebsd.org/cgi/query-pr.cgi?pr=125502 From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 20:52:08 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D47810656E4 for ; Fri, 11 Jul 2008 20:52:08 +0000 (UTC) (envelope-from robin@icir.org) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) by mx1.freebsd.org (Postfix) with ESMTP id 6864C8FC0C for ; Fri, 11 Jul 2008 20:52:08 +0000 (UTC) (envelope-from robin@icir.org) Received: from empire.ICSI.Berkeley.EDU (empire.ICSI.Berkeley.EDU [192.150.186.169]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id m6BKRgZD025281; Fri, 11 Jul 2008 13:27:42 -0700 (PDT) Received: by empire.ICSI.Berkeley.EDU (Postfix, from userid 502) id C4B59E0F8CD; Fri, 11 Jul 2008 13:27:37 -0700 (PDT) Date: Fri, 11 Jul 2008 13:27:37 -0700 From: Robin Sommer To: freebsd-net@freebsd.org Message-ID: <20080711202737.GB27418@icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) Cc: Subject: BPF problems on FreeBSD 7.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 20:52:08 -0000 Hi all, we're seeing some strange effects with our libpcap-based application (the Bro network intrusion detection system) on a FreeBSD 7-RELEASE system. As the application has always been running fine on 6.x, we're wondering whether this might be triggered by any of the changes that went into 7. The problem is that the Bro process, after running fine for a few hours or so, regularly stalls completely; the process seems to enter some odd state, using 0% CPU and with top showing only an empty field in the STATE column. We saw this effect with a Neterion network card and first thought it might be a driver problem. After switching to an Intel card, we see something slightly different: now the process doesn't stall completely anymore but it still gets to some point at which it stops receiving packets from libpcap. We haven't yet seen these problems with any other libpcap application. The only difference between Bro and most other libpcap applications that I can think of right now, is that Bro is using select() on the file descriptor. However, with a small test applicaton which mimics Bro's way of using libpcap, we couldn't reproduce the problem so far either. With the Neterion card, we have also tried disabling LRO and MSI explicitly but to no avail. Again, this is all with a Bro installation that works fine when running FreeBSD 6.x (we haven't run 6.x on the same boxes but we see the problems on two separate machines running FreeBSD 7). I'm wondering whether anybody here has seen something similar or might have an idea where to start looking for the cause. Any ideas? Thanks, Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin@icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 21:20:07 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 364231065670 for ; Fri, 11 Jul 2008 21:20:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 33B068FC08 for ; Fri, 11 Jul 2008 21:20:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m6BLK75w091851 for ; Fri, 11 Jul 2008 21:20:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m6BLK7Bh091849; Fri, 11 Jul 2008 21:20:07 GMT (envelope-from gnats) Date: Fri, 11 Jul 2008 21:20:07 GMT Message-Id: <200807112120.m6BLK7Bh091849@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Cc: Subject: Re: kern/92090: [bge] bge0: watchdog timeout -- resetting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark.Favas@csiro.au List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 21:20:07 -0000 The following reply was made to PR kern/92090; it has been noted by GNATS. From: To: , Cc: Subject: Re: kern/92090: [bge] bge0: watchdog timeout -- resetting Date: Sat, 12 Jul 2008 05:13:08 +0800 --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D17EXWAMBX01nexu_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I can confirm that this problem still exists in FreeBSD7.0-STABLE. Hardware is Dell PowerEdge 2550 with on-board bge interface. And yes, the p= roblem is most reliably triggered by csup. FreeBSD bienvenue 7.0-STABLE FreeBSD 7.0-STABLE #1: Mon Jul 7 16:14:42 WST= 2008 root@bienvenue:/usr/obj/usr/src/sys/BIENVENUE i386 Jul 11 03:50:46 bienvenue kernel: bge0: watchdog timeout -- resetting Jul 11 03:50:46 bienvenue kernel: bge0: link state changed to DOWN Jul 11 03:50:48 bienvenue kernel: bge0: link state changed to UP Jul 12 03:55:09 bienvenue kernel: bge0: watchdog timeout -- resetting Jul 12 03:55:09 bienvenue kernel: bge0: link state changed to DOWN Jul 12 03:55:11 bienvenue kernel: bge0: link state changed to UP --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D17EXWAMBX01nexu_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
I can confirm that this problem still exists in FreeBSD7.0-STABLE.
 
Hardware is Dell PowerEdge 2550 with on-board bge interface. And yes, = the problem is most reliably triggered by csup.
 
FreeBSD bienvenue 7.0-STABLE FreeBSD 7.0-STABLE #1: Mon Jul  7 16= :14:42 WST 2008     root@bienvenue:/usr/obj/usr/src/sys= /BIENVENUE  i386
 
Jul 11 03:50:46 bienvenue kernel: bge0: watchdog timeout -- resetting<= /div>
Jul 11 03:50:46 bienvenue kernel: bge0: link state changed to DOWN
Jul 11 03:50:48 bienvenue kernel: bge0: link state changed to UP
Jul 12 03:55:09 bienvenue kernel: bge0: watchdog timeout -- resetting<= /div>
Jul 12 03:55:09 bienvenue kernel: bge0: link state changed to DOWN
Jul 12 03:55:11 bienvenue kernel: bge0: link state changed to UP
 
 
 
 --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D17EXWAMBX01nexu_-- From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 21:40:08 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81C741065674 for ; Fri, 11 Jul 2008 21:40:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 79ABB8FC16 for ; Fri, 11 Jul 2008 21:40:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m6BLe8Oi094259 for ; Fri, 11 Jul 2008 21:40:08 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m6BLe8Ao094258; Fri, 11 Jul 2008 21:40:08 GMT (envelope-from gnats) Date: Fri, 11 Jul 2008 21:40:08 GMT Message-Id: <200807112140.m6BLe8Ao094258@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Cc: Subject: Re: kern/123347: [bge] bge1: watchdog timeout -- linkstate changed to DOWN X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark.Favas@csiro.au List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2008 21:40:08 -0000 The following reply was made to PR kern/123347; it has been noted by GNATS. From: To: , Cc: Subject: Re: kern/123347: [bge] bge1: watchdog timeout -- linkstate changed to DOWN Date: Sat, 12 Jul 2008 05:03:21 +0800 --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D16EXWAMBX01nexu_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I am currently seeing the same issue with a single bge interface on FreeBSD= 7.0-STABLE. pciconf -lv | grep -A 3 bge bge0@pci0:1:8:0: class=3D0x020000 card=3D0x00d11028 chip=3D0x164414e= 4 rev=3D0x10 hdr=3D0x00 vendor =3D 'Broadcom Corporation' device =3D 'BCM5751F NetXtreme Gigabit Ethernet Controller' class =3D network Jul 11 03:50:46 bienvenue kernel: bge0: watchdog timeout -- resetting Jul 11 03:50:46 bienvenue kernel: bge0: link state changed to DOWN Jul 11 03:50:48 bienvenue kernel: bge0: link state changed to UP Jul 12 03:55:09 bienvenue kernel: bge0: watchdog timeout -- resetting Jul 12 03:55:09 bienvenue kernel: bge0: link state changed to DOWN Jul 12 03:55:11 bienvenue kernel: bge0: link state changed to UP FreeBSD bienvenue 7.0-STABLE FreeBSD 7.0-STABLE #1: Mon Jul 7 16:14:42 WST= 2008 root@bienvenue:/usr/obj/usr/src/sys/BIENVENUE i386 --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D16EXWAMBX01nexu_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
I am currently seeing the same issue with a single bge interface on Fr= eeBSD 7.0-STABLE.
 
pciconf -lv | grep -A 3 bge
bge0@pci0:1:8:0:        class=3D0x0= 20000 card=3D0x00d11028 chip=3D0x164414e4 rev=3D0x10 hdr=3D0x00
    vendor     =3D 'Broadcom Corpor= ation'
    device     =3D 'BCM5751F NetXtr= eme Gigabit Ethernet Controller'
    class      =3D network
 
Jul 11 03:50:46 bienvenue kernel: bge0: watchdog timeout -- resetting<= /div>
Jul 11 03:50:46 bienvenue kernel: bge0: link state changed to DOWN
Jul 11 03:50:48 bienvenue kernel: bge0: link state changed to UP
Jul 12 03:55:09 bienvenue kernel: bge0: watchdog timeout -- resetting<= /div>
Jul 12 03:55:09 bienvenue kernel: bge0: link state changed to DOWN
Jul 12 03:55:11 bienvenue kernel: bge0: link state changed to UP
 
FreeBSD bienvenue 7.0-STABLE FreeBSD 7.0-STABLE #1: Mon Jul  7 16= :14:42 WST 2008     root@bienvenue:/usr/obj/usr/src/sys= /BIENVENUE  i386
 
 --_000_3710094049A65F4EA7CA1DC0D39D0E8B01EC139D16EXWAMBX01nexu_-- From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 06:44:53 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E79E31065674 for ; Sat, 12 Jul 2008 06:44:53 +0000 (UTC) (envelope-from brian.mcginty@gmail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.175]) by mx1.freebsd.org (Postfix) with ESMTP id 7651E8FC15 for ; Sat, 12 Jul 2008 06:44:53 +0000 (UTC) (envelope-from brian.mcginty@gmail.com) Received: by wf-out-1314.google.com with SMTP id 24so3543052wfg.7 for ; Fri, 11 Jul 2008 23:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=T2RWck03S08LIGMGPjMHTORN0dayCfA7XNeNRXuG+KY=; b=lhLn4fitwsPZyuJj2uobAXAqOQo6asmU9pfWz4BbwlsWFriID37j1SUIev70e6go2q eBd/mjU6l7PyKUfejbl82RmPWxz2HEqJZLj7XctdrRyqe6PC8Yu9XZVJLc/L+SLusD5t lwRdPX4tAiIBxrd7e0CAyRl1IP/4G68NozNMU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=IyI0Q4GSzA9qhFzlvEI5cfPS4AstIMKkPzj96S5JsPYHdZaHyjBrgKlKQ3J3tGUDhE qPMMcAwpTS/qEMJ8nU5C4ycVS6srd8eWjOD+pv+iJPCsG54tXdxbo7HnPDwxPUKbPh3K eAetd4BZUC572054qAE6k7PFw4JJrkqZPphDY= Received: by 10.142.162.5 with SMTP id k5mr3463344wfe.260.1215845092686; Fri, 11 Jul 2008 23:44:52 -0700 (PDT) Received: by 10.142.199.4 with HTTP; Fri, 11 Jul 2008 23:44:52 -0700 (PDT) Message-ID: <601bffc40807112344n7a683f81y516f540e24d87389@mail.gmail.com> Date: Fri, 11 Jul 2008 23:44:52 -0700 From: "Brian McGinty" To: "Kip Macy" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4867420D.7090406@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <4871E85C.8090907@freebsd.org> <48726422.7050703@gtcomm.net> <200807080107.m6817XxO021966@lava.sentex.ca> <601bffc40807081346q454c1f40td47a0f54806d8a8c@mail.gmail.com> Cc: FreeBSD Net , Paul , Andre Oppermann , Mike Tancsa Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2008 06:44:54 -0000 > Hi Brian > I very much doubt that this is ceteris paribus. This is 384 random IPs > -> 384 random IP addresses with a flow lookup for each packet. Also, > I've read through igb on Linux - it has a lot of optimizations that > the FreeBSD driver lacks and I have yet to implement. Hey Kip, when will you push the optimization into FreeBSD? Cheers, Brian From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 12:45:07 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83F2C106567D for ; Sat, 12 Jul 2008 12:45:07 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id 3C0E98FC19 for ; Sat, 12 Jul 2008 12:45:06 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 5239F41C62F; Sat, 12 Jul 2008 14:45:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id WSZiJHNORV2f; Sat, 12 Jul 2008 14:45:05 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id EC6F041C62D; Sat, 12 Jul 2008 14:45:04 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 6752D44487F; Sat, 12 Jul 2008 12:44:25 +0000 (UTC) Date: Sat, 12 Jul 2008 12:44:25 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Mike Tancsa In-Reply-To: <200807021355.m62Dtpli092002@lava.sentex.ca> Message-ID: <20080712124250.M57089@maildrop.int.zabbadoz.net> References: <4852E23E.2040505@gtcomm.net> <4854EBF1.7020708@FreeBSD.org> <200807010606.m6166jFe084204@lava.sentex.ca> <4869EC1E.8060009@freebsd.org> <20080701084933.W57089@maildrop.int.zabbadoz.net> <20080701092254.T57089@maildrop.int.zabbadoz.net> <486B87DB.3080007@freebsd.org> <200807021355.m62Dtpli092002@lava.sentex.ca> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: Route messages X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2008 12:45:07 -0000 On Wed, 2 Jul 2008, Mike Tancsa wrote: Hi, > It works for me in the lab and on one production machine I patched early this > morning. I just MFCed this to 7-STABLE. So if you update your trees make sure you have rev. 1.332.2.3 of ip_input.c. /bz >>> Index: sys/netinet/ip_input.c >>> =================================================================== >>> RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_input.c,v >>> retrieving revision 1.332.2.2 >>> diff -u -p -r1.332.2.2 ip_input.c >>> --- sys/netinet/ip_input.c 22 Apr 2008 12:02:55 -0000 1.332.2.2 >>> +++ sys/netinet/ip_input.c 1 Jul 2008 09:23:08 -0000 >>> @@ -1363,7 +1363,6 @@ ip_forward(struct mbuf *m, int srcrt) >>> * the ICMP_UNREACH_NEEDFRAG "Next-Hop MTU" field described in >>> RFC1191. >>> */ >>> bzero(&ro, sizeof(ro)); >>> - rtalloc_ign(&ro, RTF_CLONING); >>> error = ip_output(m, NULL, &ro, IP_FORWARDING, NULL, NULL); -- Bjoern A. Zeeb Stop bit received. Insert coin for new game.