From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 01:11:52 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA19516A420 for ; Sun, 5 Feb 2006 01:11:52 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) Received: from mail.npubs.com (mail.wsfamily.com [209.66.100.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 901DD43D45 for ; Sun, 5 Feb 2006 01:11:52 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) From: Nate Nielsen User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051013) X-Accept-Language: en-us, en MIME-Version: 1.0 To: "freebsd-net@FreeBSD.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Message-Id: <20060205011931.E5F08DCA99E@mail.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Sun, 5 Feb 2006 01:19:33 +0000 (GMT) Cc: Subject: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: nielsen@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 01:11:52 -0000 I've been working on polling for the FreeBSD ath wireless driver. On slow CPU's polling helps prevent (by supressing certain interrupts) livelock and increases throughput. This is true of Atheros cards on Soekris and other embedded hardware. Just thought I'd post something here in case anyone is interested in helping with insight, code or testing. I've got a rough (and mostly working) patch. There are some locking issues to contend with... Cheers, Nate From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 06:54:35 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB7CF16A420 for ; Sun, 5 Feb 2006 06:54:35 +0000 (GMT) (envelope-from lukas.muehlethaler@yale.edu) Received: from pantheon-po08.its.yale.edu (pantheon-po08.its.yale.edu [130.132.50.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15E4543D49 for ; Sun, 5 Feb 2006 06:54:34 +0000 (GMT) (envelope-from lukas.muehlethaler@yale.edu) Received: from [10.0.1.1] (69.177.222.28.adsl.snet.net [69.177.222.28] (may be forged)) (authenticated bits=0) by pantheon-po08.its.yale.edu (8.12.11/8.12.11) with ESMTP id k156sXiL026363 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT); Sun, 5 Feb 2006 01:54:34 -0500 Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Lukas Muehlethaler Date: Sun, 5 Feb 2006 01:54:31 -0500 To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.623) X-YaleITSMailFilter: Version 1.2c (attachment(s) not renamed) Subject: Wireless PCI for FreeBSD 6.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 06:54:35 -0000 I want to buy a wireless PCI adapter for a machine running FreeBSD 6.0. After reading the archives, man pages, etc. I realized that his is a tricky business. Some of the adaptors listed as supported have now a different chipset. For example, Netgear WG311T now has the AR5002G chipset and I guess this means it is no longer supported by the ath(4) driver. If you bought recently a wireless PCI adapter in the price range of the Netgear WG311T (less than $50) that was supported by the FreeBSD drivers (not the ndis wrapper), could you give me a shout? Thanks a lot! Lukas From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 08:10:51 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1989F16A459; Sun, 5 Feb 2006 08:10:51 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from mrout2.yahoo.com (mrout2.yahoo.com [216.145.54.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8440443D55; Sun, 5 Feb 2006 08:10:50 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (proxy7.corp.yahoo.com [216.145.48.98]) by mrout2.yahoo.com (8.13.4/8.13.4/y.out) with ESMTP id k1589NIT018674; Sun, 5 Feb 2006 00:09:24 -0800 (PST) Date: Sun, 05 Feb 2006 17:09:28 +0900 Message-ID: From: gnn@FreeBSD.org To: Max Laier In-Reply-To: <200602041616.57224.max@love2party.net> References: <20060201005658.A70005@xorpc.icir.org> <20060202124328.GK29980@comp.chem.msu.su> <200602021437.38385.max@love2party.net> <200602041616.57224.max@love2party.net> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.7 (=?ISO-8859-4?Q?Sanj=F2?=) APEL/10.6 Emacs/22.0.50 (powerpc-apple-darwin8.3.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: freebsd-net@FreeBSD.org, Luigi Rizzo , Hajimu UMEMOTO , Yar Tikhiy Subject: Re: if_bridge.ko requires INET6... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 08:10:51 -0000 At Sat, 4 Feb 2006 16:16:49 +0100, Max Laier wrote: > Here it is. I'd appreciate feedback. pflog_packet() uses a lot of complex > types which makes it necessary to include pfvar.h. This is ugly, but I don't > know how to work around this. I gave this a quick read and it looked OK to me. Later, George From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 17:23:07 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADACA16A420; Sun, 5 Feb 2006 17:23:07 +0000 (GMT) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1562643D46; Sun, 5 Feb 2006 17:23:07 +0000 (GMT) (envelope-from max@love2party.net) Received: from [84.163.254.207] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu7) with ESMTP (Nemesis), id 0ML2Dk-1F5nbH2eSo-00041c; Sun, 05 Feb 2006 18:23:02 +0100 From: Max Laier Organization: FreeBSD To: freebsd-net@freebsd.org Date: Sun, 5 Feb 2006 18:24:20 +0100 User-Agent: KMail/1.9.1 References: <20060201005658.A70005@xorpc.icir.org> <200602021437.38385.max@love2party.net> <200602041616.57224.max@love2party.net> In-Reply-To: <200602041616.57224.max@love2party.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1340944.OCuVqix85F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200602051824.28100.max@love2party.net> X-Provags-ID: kundenserver.de abuse@kundenserver.de login:61c499deaeeba3ba5be80f48ecc83056 Cc: Yar Tikhiy , Hajimu UMEMOTO , Luigi Rizzo Subject: Re: if_bridge.ko requires INET6... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 17:23:07 -0000 --nextPart1340944.OCuVqix85F Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Saturday 04 February 2006 16:16, Max Laier wrote: > On Thursday 02 February 2006 14:37, Max Laier wrote: > > On Thursday 02 February 2006 13:43, Yar Tikhiy wrote: > > > > This needs to be fixed in pf then. > > > > > > Max Laier and I discussed this issue once, and Max had concern > > > over possible performance degradation that might result from > > > calling pflog functions through pointers to be set by a separate > > > pflog module. We can skip touching the pf module in RELENG_6 for > > > now and leave the issue to after 6.1-RELEASE is out. > > > > I have convinced myself that we should really use a function pointer > > here. I will try to commit a sollution to HEAD over the weekend. If you > > are MFC'ing the changes *now*, I'd appreciate if you could spare out pf, > > but I am willing to MFC the changes before 6.1 if testing goes well. > > Here it is. I'd appreciate feedback. pflog_packet() uses a lot of compl= ex > types which makes it necessary to include pfvar.h. This is ugly, but I > don't know how to work around this. =46YI, just committed this to the tree. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1340944.OCuVqix85F Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBD5jTMXyyEoT62BG0RAs7IAJwPR+qbL5XyOndrsfb2eJuCdctUIwCeKSaT ELw0PTx0XE5EQkqTRR4DXAo= =Z9Cd -----END PGP SIGNATURE----- --nextPart1340944.OCuVqix85F-- From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 18:07:09 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E471116A420 for ; Sun, 5 Feb 2006 18:07:09 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5129F43D46 for ; Sun, 5 Feb 2006 18:07:09 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k15I77o7044663 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 5 Feb 2006 10:07:08 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43E63F31.4000109@errno.com> Date: Sun, 05 Feb 2006 10:08:49 -0800 From: Sam Leffler User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051227) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Lukas Muehlethaler References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Wireless PCI for FreeBSD 6.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 18:07:10 -0000 Lukas Muehlethaler wrote: > I want to buy a wireless PCI adapter for a machine running FreeBSD 6.0. > After reading the archives, man pages, etc. I realized that his is a > tricky business. Some of the adaptors listed as supported have now a > different chipset. For example, Netgear WG311T now has the AR5002G > chipset and I guess this means it is no longer supported by the ath(4) > driver. If you bought recently a wireless PCI adapter in the price range > of the Netgear WG311T (less than $50) that was supported by the FreeBSD > drivers (not the ndis wrapper), could you give me a shout? Thanks a lot! I'm not sure why you tihnk the AR5002G is not supported. The hal currently in cvs for freebsd 6.0 and later systems supports all parts except the very latest ones (AR5006 I believe is the tag). Sam From owner-freebsd-net@FreeBSD.ORG Sun Feb 5 18:09:35 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CED1A16A420 for ; Sun, 5 Feb 2006 18:09:35 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8452C43D46 for ; Sun, 5 Feb 2006 18:09:35 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k15I9Yo7044680 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 5 Feb 2006 10:09:35 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43E63FC3.2030409@errno.com> Date: Sun, 05 Feb 2006 10:11:15 -0800 From: Sam Leffler User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051227) X-Accept-Language: en-us, en MIME-Version: 1.0 To: nielsen@memberwebs.com References: <20060205011931.E5F08DCA99E@mail.npubs.com> In-Reply-To: <20060205011931.E5F08DCA99E@mail.npubs.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2006 18:09:35 -0000 Nate Nielsen wrote: > I've been working on polling for the FreeBSD ath wireless driver. > > On slow CPU's polling helps prevent (by supressing certain interrupts) > livelock and increases throughput. This is true of Atheros cards on > Soekris and other embedded hardware. > > Just thought I'd post something here in case anyone is interested in > helping with insight, code or testing. I've got a rough (and mostly > working) patch. There are some locking issues to contend with... You might try explaining why you think polling helps your performance. Unless you've significantly restructured the interrupt handling in the driver most work is deferred to a non-interrupt context. Also the driver in 6.0 and later does tx interrupt mitigation and rx interrupt coalescing so I wouldn't expect interrupt handling to be a performance limitation. There are other issues that can affect performance but you haven't mentioned them... Sam From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 04:55:15 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C4D816A420 for ; Mon, 6 Feb 2006 04:55:15 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) Received: from mail.npubs.com (mail.npubs.com [209.66.100.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED51543D45 for ; Mon, 6 Feb 2006 04:55:14 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) From: Nate Nielsen User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051013) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sam Leffler References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Message-Id: <20060206050301.AC895DCAA3F@mail.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Mon, 6 Feb 2006 05:03:02 +0000 (GMT) Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: nielsen@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 04:55:15 -0000 Sam Leffler wrote: > You might try explaining why you think polling helps your performance. > Unless you've significantly restructured the interrupt handling in the > driver most work is deferred to a non-interrupt context. Yes, I saw that. However the interrupts themselves when they are fired at upwards of a thousand per second do greatly affect performance and scheduling on a slow box. I understand that it may not seem intuitive on faster systems which are completely capable of handling such a small interrupt load. But these embedded systems are slow little puppies, running between 133 and 266 Mhz (586 class chip). Adding polling to this driver does increase performance on embedded systems. With my current patch (on a 233Mhz system), the throughput (in this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to 24Mbits. But that's not the main benefit. The main thing is the scheduling. Without having the thousands of interrupts, the box is better able to balance the RX/TX with the other aspects, such as encapsulation, packet filtering and other activities. When the entire box is polling driven, it's total throughput (ethernet, encapsulation, hardware encryption, wireless) increases greatly and does not exhibit livelock symptoms. Without polling, these slow systems easily exhibit livelock. This is where incoming traffic can cause so many interrupts that outgoing traffic is completely halted and the throughput drops to zero or near zero. Under these conditions userland processes also run barely or not at all. The scheduler has no say in what's going on in the system, as interrupts overwhelm all other activity. > Also the > driver in 6.0 and later does tx interrupt mitigation and rx interrupt > coalescing so I wouldn't expect interrupt handling to be a performance > limitation. Interesting. If there's an option to enable it, I very well may have missed it. However it should be noted, that the default behaviour (in 6.0 release) seems to be that the hardware generates about around 2000 interrupts per second at around 15 - 18 Mbits throughput. > There are other issues that can affect performance but you > haven't mentioned them... Obviously these are not mainstream performance issues for people just trying to connect to an access point. But when the atheros cards are used in backhaul applications and are running in the low power embedded systems you typically see on an antenna mast, polling makes a big difference. Cheers, Nate From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 05:53:24 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E8CA316A420 for ; Mon, 6 Feb 2006 05:53:23 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8555D43D45 for ; Mon, 6 Feb 2006 05:53:23 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k165rLo7047282 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 5 Feb 2006 21:53:23 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43E6E4B6.50100@errno.com> Date: Sun, 05 Feb 2006 21:55:02 -0800 From: Sam Leffler User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051227) X-Accept-Language: en-us, en MIME-Version: 1.0 To: nielsen@memberwebs.com References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> In-Reply-To: <20060206050301.AC895DCAA3F@mail.npubs.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 05:53:24 -0000 Nate Nielsen wrote: > Sam Leffler wrote: > >>You might try explaining why you think polling helps your performance. >>Unless you've significantly restructured the interrupt handling in the >>driver most work is deferred to a non-interrupt context. > > > Yes, I saw that. However the interrupts themselves when they are fired > at upwards of a thousand per second do greatly affect performance and > scheduling on a slow box. Max pps for an ath device are 8-9K/sec. Typical is 3-4K for a single bidirectional tcp stream when the client has a strong signal. > > I understand that it may not seem intuitive on faster systems which are > completely capable of handling such a small interrupt load. But these > embedded systems are slow little puppies, running between 133 and 266 > Mhz (586 class chip). I am familiar with embedded systems. > > Adding polling to this driver does increase performance on embedded > systems. With my current patch (on a 233Mhz system), the throughput (in > this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to 24Mbits. I routinely get >20 Mb/s for a single client running upstream TCP netperf through a soekris 4511. If you are seeing 6Mb/s you have something else wrong. > > But that's not the main benefit. The main thing is the scheduling. > Without having the thousands of interrupts, the box is better able to > balance the RX/TX with the other aspects, such as encapsulation, packet > filtering and other activities. > > When the entire box is polling driven, it's total throughput (ethernet, > encapsulation, hardware encryption, wireless) increases greatly and does > not exhibit livelock symptoms. > > Without polling, these slow systems easily exhibit livelock. This is > where incoming traffic can cause so many interrupts that outgoing > traffic is completely halted and the throughput drops to zero or near > zero. Under these conditions userland processes also run barely or not > at all. The scheduler has no say in what's going on in the system, as > interrupts overwhelm all other activity. I've not seen livelock in any situations though there are some issues with the priority of the taskqueue thread. Polling is not a panacea; you are potentially increasing latency which has ramifications. > > >>Also the >>driver in 6.0 and later does tx interrupt mitigation and rx interrupt >>coalescing so I wouldn't expect interrupt handling to be a performance >>limitation. > > > Interesting. If there's an option to enable it, I very well may have > missed it. It is always used. > > However it should be noted, that the default behaviour (in 6.0 release) > seems to be that the hardware generates about around 2000 interrupts per > second at around 15 - 18 Mbits throughput. You need to identify what kind of interrupts there are and what type of ath hardware you are using. You can trivially reduce the tx interrupt load by turning off interrupts on EOL and just using the periodic interrupts generated every N tx descriptors. But if you profile I suspect you will find the interrupt overhead is not significant relative to other costs. > > >>There are other issues that can affect performance but you >>haven't mentioned them... > > > Obviously these are not mainstream performance issues for people just > trying to connect to an access point. But when the atheros cards are > used in backhaul applications and are running in the low power embedded > systems you typically see on an antenna mast, polling makes a big > difference. I'm not convinced polling is worthwhile w/o a major restructuring of the driver. OTOH this shouldn't stop you from pushing forward... Sam From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 08:22:26 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AD13016A420 for ; Mon, 6 Feb 2006 08:22:26 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) Received: from mail.npubs.com (mail.wsfamily.com [209.66.100.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B30043D48 for ; Mon, 6 Feb 2006 08:22:25 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) From: Nate Nielsen User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051013) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sam Leffler References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> <43E6E4B6.50100@errno.com> Content-Type: multipart/mixed; boundary="------------010604040400040500050206" Message-Id: <20060206083013.99D32DCAA14@mail.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Mon, 6 Feb 2006 08:30:16 +0000 (GMT) Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: nielsen@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 08:22:26 -0000 This is a multi-part message in MIME format. --------------010604040400040500050206 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sam Leffler wrote: > Nate Nielsen wrote: >> Adding polling to this driver does increase performance on embedded >> systems. With my current patch (on a 233Mhz system), the throughput (in >> this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to >> 24Mbits. > > I routinely get >20 Mb/s for a single client running upstream TCP > netperf through a soekris 4511. If you are seeing 6Mb/s you have > something else wrong. Note I was talking about an *increase* of 6Mb/s. In addition my TCP stream ends on the box in question, which obviously increases the load on the system, which brings the numbers we are both seeing roughly in the same ballpark. > I've not seen livelock in any situations though there are some issues > with the priority of the taskqueue thread. With a small number of simple self regulating packet streams (such as TCP) livelock is not really an issue, as the the streams will slow their transmit rate when the box gets near livelock and packets start dropping. However on more complex links where traffic is not (or not completely) self regulating (ie: VOIP, other datagram streams, a high number of TCP streams, asymetric routing) livelock because of interrupt performance is a common occurance. > Polling is not a panacea; > you are potentially increasing latency which has ramifications. Correct, whenever polling is in use (on ethernet as well) we see the latency go up. In addition the system is under higher load when there is no traffic. These are tradeoffs that one accepts when using polling. Obviously DEVICE_POLLING is not configured by default. >> However it should be noted, that the default behaviour (in 6.0 release) >> seems to be that the hardware generates about around 2000 interrupts per >> second at around 15 - 18 Mbits throughput. > > You need to identify what kind of interrupts there are and what type of > ath hardware you are using. The interrupts are the RX and TX interrupts. My polling additions don't mask out any other interrupts. > You can trivially reduce the tx interrupt > load by turning off interrupts on EOL and just using the periodic > interrupts generated every N tx descriptors. Thanks for the tip. I'm sure that would help. I'll look into that. > But if you profile I > suspect you will find the interrupt overhead is not significant relative > to other costs. The very act of the CPU servicing the interrupt (ie: saving registers, switching stacks, etc...) causes overhead. In addition the interrupt handling does not fall under the domain of the scheduler. This isn't just theory it has a tested real life impact. As I noted earlier for a simple TCP stream over a wireless link throughput went up by 6Mb/s. In my real world case: On a 233Mhz net4826 running GIF encapsulation, IPSEC encryption, and wireless backhaul, throughput went from being livelocked at 3.5Mb/s (and userland barely functioning) to over 10Mb/s (with userland scheduled properly). This is with polling used in the sis ethernet driver, the hifn crypto card driver, and the ath driver. Instead of each of these devices generating interrupts, polling (in my case at 256 Hz) allows the system to function smoothly. Yes, there is latency of up to 20ms, but that's a small tradeoff for the throughput and stability increases. > I'm not convinced polling is worthwhile w/o a major restructuring of the > driver. OTOH this shouldn't stop you from pushing forward... As you noted there are other performance enhancements that could be made, and while I'd love to see them implemented, I fear they may be out of my scope and available time. This polling addition is a simple performance enhancement that helps my clients, and perhaps others would also be interested. I'll attach the rough patch, to give an idea of the direction I'm working in. Note that this patch is incomplete. It locks after a while, which is probably due to the way I call the taskqueue callbacks. I'll continue to work on this. Please understand that by posting this patch I'm not pressuring you to help. Thanks again for your advice so far. Cheers, Nate --------------010604040400040500050206 Content-Type: text/x-patch; name="ath-polling-will-lock-your-system.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="ath-polling-will-lock-your-system.patch" Index: if_ath.c =================================================================== RCS file: /home/ncvs/src/sys/dev/ath/if_ath.c,v retrieving revision 1.94.2.8 diff -u -3 -p -r1.94.2.8 if_ath.c --- if_ath.c 29 Jan 2006 07:33:27 -0000 1.94.2.8 +++ if_ath.c 4 Feb 2006 22:42:08 -0000 @@ -44,6 +44,7 @@ __FBSDID("$FreeBSD: src/sys/dev/ath/if_a * is greatly appreciated. */ +#include "opt_device_polling.h" #include "opt_inet.h" #include @@ -174,6 +175,12 @@ static void ath_setcurmode(struct ath_so static void ath_sysctlattach(struct ath_softc *); static void ath_bpfattach(struct ath_softc *); static void ath_announce(struct ath_softc *); +static void ath_update_intr(struct ath_softc *); +static void ath_mask_intr(struct ath_softc *, HAL_INT mask); + +#ifdef DEVICE_POLLING +static poll_handler_t ath_poll; +#endif SYSCTL_DECL(_hw_ath); @@ -476,6 +483,10 @@ ath_attach(u_int16_t devid, struct ath_s ifp->if_snd.ifq_drv_maxlen = IFQ_MAXLEN; IFQ_SET_READY(&ifp->if_snd); +#ifdef DEVICE_POLLING + ifp->if_capabilities |= IFCAP_POLLING; +#endif + ic->ic_ifp = ifp; ic->ic_reset = ath_reset; ic->ic_newassoc = ath_newassoc; @@ -609,6 +620,11 @@ ath_detach(struct ath_softc *sc) DPRINTF(sc, ATH_DEBUG_ANY, "%s: if_flags %x\n", __func__, ifp->if_flags); +#ifdef DEVICE_POLLING + if (ifp->if_capenable & IFCAP_POLLING) + ether_poll_deregister(ifp); +#endif + ath_stop(ifp); bpfdetach(ifp); /* @@ -674,6 +690,22 @@ ath_shutdown(struct ath_softc *sc) ath_stop(ifp); } +static void +ath_update_intr(struct ath_softc *sc) +{ + /* When polling suppress certain interrupts */ + ath_hal_intrset(sc->sc_ah, + sc->sc_imask & ~sc->sc_ipolling); +} + +static void +ath_mask_intr(struct ath_softc *sc, HAL_INT mask) +{ + /* When polling suppress certain interrupts */ + ath_hal_intrset(sc->sc_ah, + (sc->sc_imask & ~sc->sc_ipolling) & mask); +} + /* * Interrupt handler. Most of the actual processing is deferred. */ @@ -700,7 +732,7 @@ ath_intr(void *arg) DPRINTF(sc, ATH_DEBUG_ANY, "%s: if_flags 0x%x\n", __func__, ifp->if_flags); ath_hal_getisr(ah, &status); /* clear ISR */ - ath_hal_intrset(ah, 0); /* disable further intr's */ + ath_mask_intr(sc, 0); /* disable further intr's */ return; } /* @@ -720,11 +752,11 @@ ath_intr(void *arg) * by the hal. */ sc->sc_stats.ast_hardware++; - ath_hal_intrset(ah, 0); /* disable intr's until reset */ + ath_mask_intr(sc, 0); /* disable intr's until reset */ taskqueue_enqueue(taskqueue_swi, &sc->sc_fataltask); } else if (status & HAL_INT_RXORN) { sc->sc_stats.ast_rxorn++; - ath_hal_intrset(ah, 0); /* disable intr's until reset */ + ath_mask_intr(sc, 0); /* disable intr's until reset */ taskqueue_enqueue(taskqueue_swi, &sc->sc_rxorntask); } else { if (status & HAL_INT_SWBA) { @@ -764,18 +796,39 @@ ath_intr(void *arg) * Disable interrupts until we service the MIB * interrupt; otherwise it will continue to fire. */ - ath_hal_intrset(ah, 0); + ath_mask_intr(sc, 0); /* * Let the hal handle the event. We assume it will * clear whatever condition caused the interrupt. */ ath_hal_mibevent(ah, &ATH_NODE(sc->sc_ic.ic_bss)->an_halstats); - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); } } } +#ifdef DEVICE_POLLING + +static void +ath_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + struct ath_softc *sc = ifp->if_softc; + + if (sc->sc_invalid) + return; + + if (!((ifp->if_flags & IFF_UP) && (ifp->if_drv_flags & + IFF_DRV_RUNNING))) + return; + + /* XXX Locks. Call the transmit and receive functions */ + (sc->sc_rxtask.ta_func) (sc, -1); /* taskqueue_enqueue(taskqueue_swi, &sc->sc_rxtask); */ + (sc->sc_txtask.ta_func) (sc, -1); /* taskqueue_enqueue(taskqueue_swi, &sc->sc_txtask); */ +} + +#endif /* DEVICE_POLLING */ + static void ath_fatal_proc(void *arg, int pending) { @@ -898,6 +951,10 @@ ath_init(void *arg) /* * Enable interrupts. + * + * sc_imask holds the current interrupts we're interested in. + * This can be masked by sc_ipolling which suppresses certain + * interrupts when in polling mode. */ sc->sc_imask = HAL_INT_RX | HAL_INT_TX | HAL_INT_RXEOL | HAL_INT_RXORN @@ -908,7 +965,7 @@ ath_init(void *arg) */ if (sc->sc_needmib && ic->ic_opmode == IEEE80211_M_STA) sc->sc_imask |= HAL_INT_MIB; - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); ifp->if_drv_flags |= IFF_DRV_RUNNING; ic->ic_state = IEEE80211_S_INIT; @@ -965,7 +1022,7 @@ ath_stop_locked(struct ifnet *ifp) !sc->sc_ledon); sc->sc_blinking = 0; } - ath_hal_intrset(ah, 0); + ath_mask_intr(sc, 0); } ath_draintxq(sc); if (!sc->sc_invalid) { @@ -1024,7 +1081,7 @@ ath_reset(struct ifnet *ifp) sc->sc_curchan.channel = c->ic_freq; sc->sc_curchan.channelFlags = ath_chan2flags(ic, c); - ath_hal_intrset(ah, 0); /* disable interrupts */ + ath_mask_intr(sc, 0); /* disable interrupts */ ath_draintxq(sc); /* stop xmit side */ ath_stoprecv(sc); /* stop recv side */ /* NB: indicate channel change so we do a full reset */ @@ -1043,7 +1100,7 @@ ath_reset(struct ifnet *ifp) ath_chan_change(sc, c); if (ic->ic_state == IEEE80211_S_RUN) ath_beacon_config(sc); /* restart beacons */ - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); ath_start(ifp); /* restart xmit */ return 0; @@ -2178,12 +2235,12 @@ ath_beacon_config(struct ath_softc *sc) , bs.bs_cfpnext , bs.bs_timoffset ); - ath_hal_intrset(ah, 0); + ath_mask_intr(sc, 0); ath_hal_beacontimers(ah, &bs); sc->sc_imask |= HAL_INT_BMISS; - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); } else { - ath_hal_intrset(ah, 0); + ath_mask_intr(sc, 0); if (nexttbtt == intval) intval |= HAL_BEACON_RESET_TSF; if (ic->ic_opmode == IEEE80211_M_IBSS) { @@ -2209,7 +2266,7 @@ ath_beacon_config(struct ath_softc *sc) } ath_hal_beaconinit(ah, nexttbtt, intval); sc->sc_bmisscount = 0; - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); /* * When using a self-linked beacon descriptor in * ibss mode load it once here. @@ -3975,7 +4032,7 @@ ath_chan_set(struct ath_softc *sc, struc * hardware at the new frequency, and then re-enable * the relevant bits of the h/w. */ - ath_hal_intrset(ah, 0); /* disable interrupts */ + ath_mask_intr(sc, 0); /* disable interrupts */ ath_draintxq(sc); /* clear pending tx frames */ ath_stoprecv(sc); /* turn off frame recv */ if (!ath_hal_reset(ah, ic->ic_opmode, &hchan, AH_TRUE, &status)) { @@ -4007,7 +4064,7 @@ ath_chan_set(struct ath_softc *sc, struc /* * Re-enable interrupts. */ - ath_hal_intrset(ah, sc->sc_imask); + ath_update_intr(sc); } return 0; } @@ -4089,7 +4146,7 @@ ath_newstate(struct ieee80211com *ic, en /* * NB: disable interrupts so we don't rx frames. */ - ath_hal_intrset(ah, sc->sc_imask &~ HAL_INT_GLOBAL); + ath_mask_intr(sc, ~HAL_INT_GLOBAL); /* * Notify the rate control algorithm. */ @@ -4179,9 +4236,8 @@ ath_newstate(struct ieee80211com *ic, en */ ath_beacon_config(sc); } else { - ath_hal_intrset(ah, - sc->sc_imask &~ (HAL_INT_SWBA | HAL_INT_BMISS)); sc->sc_imask &= ~(HAL_INT_SWBA | HAL_INT_BMISS); + ath_update_intr(sc); } done: /* @@ -4627,7 +4683,7 @@ ath_ioctl(struct ifnet *ifp, u_long cmd, struct ath_softc *sc = ifp->if_softc; struct ieee80211com *ic = &sc->sc_ic; struct ifreq *ifr = (struct ifreq *)data; - int error = 0; + int mask, error = 0; ATH_LOCK(sc); switch (cmd) { @@ -4664,6 +4720,33 @@ ath_ioctl(struct ifnet *ifp, u_long cmd, if (ifp->if_drv_flags & IFF_DRV_RUNNING) ath_mode_init(sc); break; + case SIOCSIFCAP: + mask = ifr->ifr_reqcap ^ ifp->if_capenable; +#ifdef DEVICE_POLLING + if (mask & IFCAP_POLLING) { + if (ifr->ifr_reqcap & IFCAP_POLLING) { + error = ether_poll_register(ath_poll, ifp); + if (error) + return(error); + ATH_LOCK(sc); + /* Interrupts to suppress while polling */ + sc->sc_ipolling = HAL_INT_RX | HAL_INT_TX; + ath_update_intr(sc); + ifp->if_capenable |= IFCAP_POLLING; + ATH_UNLOCK(sc); + } else { + error = ether_poll_deregister(ifp); + /* Enable interrupt even in error case */ + ATH_LOCK(sc); + /* Don't suppress any interrupts */ + sc->sc_ipolling = 0; + ath_update_intr(sc); + ifp->if_capenable &= ~IFCAP_POLLING; + ATH_UNLOCK(sc); + } + } +#endif + break; case SIOCGATHSTATS: /* NB: embed these numbers to get a consistent view */ sc->sc_stats.ast_tx_packets = ifp->if_opackets; Index: if_athvar.h =================================================================== RCS file: /home/ncvs/src/sys/dev/ath/if_athvar.h,v retrieving revision 1.27.2.4 diff -u -3 -p -r1.27.2.4 if_athvar.h --- if_athvar.h 29 Jan 2006 07:17:02 -0000 1.27.2.4 +++ if_athvar.h 4 Feb 2006 22:42:08 -0000 @@ -209,6 +209,7 @@ struct ath_softc { u_int8_t sc_protrix; /* protection rate index */ u_int sc_txantenna; /* tx antenna (fixed or auto) */ HAL_INT sc_imask; /* interrupt mask copy */ + HAL_INT sc_ipolling; /* interrupts to mask out when polling */ u_int sc_keymax; /* size of key cache */ u_int8_t sc_keymap[ATH_KEYBYTES];/* key use bit map */ --------------010604040400040500050206-- From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 09:24:46 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8961116A420; Mon, 6 Feb 2006 09:24:46 +0000 (GMT) (envelope-from anders@FreeBSD.org) Received: from totem.fix.no (totem.fix.no [80.91.36.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F44143D55; Mon, 6 Feb 2006 09:24:46 +0000 (GMT) (envelope-from anders@FreeBSD.org) Received: by totem.fix.no (Postfix, from userid 1000) id 20AF78DB146; Mon, 6 Feb 2006 10:24:44 +0100 (CET) Date: Mon, 6 Feb 2006 10:24:44 +0100 From: Anders Nordby To: freebsd-net@freebsd.org Message-ID: <20060206092443.GA61116@totem.fix.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-PGP-Key: http://anders.fix.no/pgp/ X-PGP-Key-FingerPrint: 1E0F C53C D8DF 6A8F EAAD 19C5 D12A BC9F 0083 5956 User-Agent: Mutt/1.5.11 Cc: demon@FreeBSD.org, harti@freebsd.org, kuriyama@FreeBSD.org Subject: 64-bit SNMP counters for FreeBSD && graphing bandwidth usage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 09:24:46 -0000 Hi, Is there any way to have 64-bit SNMP counters in FreeBSD? Especially for ifInOctets/ifOutOctets. It seems the built-in bsnmpd only has Counter32, and net-snmpd the same (--enable-mfd-rewrites, which is supposed to help, seems to only work in Linux). My systems are pushing more than 100 mbps these days, and graphing bandwidth usage with 32-bit counters gives funny-looking graphs. Cheers, -- Anders. From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 11:02:19 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4DD016A420 for ; Mon, 6 Feb 2006 11:02:19 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6353C43D45 for ; Mon, 6 Feb 2006 11:02:19 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k16B2JoP081790 for ; Mon, 6 Feb 2006 11:02:19 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k16B2ICa081784 for freebsd-net@freebsd.org; Mon, 6 Feb 2006 11:02:18 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 6 Feb 2006 11:02:18 GMT Message-Id: <200602061102.k16B2ICa081784@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 11:02:19 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2006/01/30] kern/92552 net A serious bug in most network drivers fro 1 problem total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/07/11] kern/54383 net [nfs] [patch] NFS root configurations wit 1 problem total. From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 11:33:03 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA63A16A420; Mon, 6 Feb 2006 11:33:03 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (comp.chem.msu.su [158.250.32.97]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D90543D49; Mon, 6 Feb 2006 11:32:48 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.13.3/8.13.3) with ESMTP id k16BWfSN026823; Mon, 6 Feb 2006 14:32:41 +0300 (MSK) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.13.3/8.13.3/Submit) id k16BWe2A026815; Mon, 6 Feb 2006 14:32:40 +0300 (MSK) (envelope-from yar) Date: Mon, 6 Feb 2006 14:32:39 +0300 From: Yar Tikhiy To: Max Laier Message-ID: <20060206113239.GD59508@comp.chem.msu.su> References: <20060201005658.A70005@xorpc.icir.org> <20060202124328.GK29980@comp.chem.msu.su> <200602021437.38385.max@love2party.net> <200602041616.57224.max@love2party.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200602041616.57224.max@love2party.net> User-Agent: Mutt/1.5.9i Cc: freebsd-net@freebsd.org, Hajimu UMEMOTO , Luigi Rizzo Subject: Re: if_bridge.ko requires INET6... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 11:33:03 -0000 On Sat, Feb 04, 2006 at 04:16:49PM +0100, Max Laier wrote: > On Thursday 02 February 2006 14:37, Max Laier wrote: > > On Thursday 02 February 2006 13:43, Yar Tikhiy wrote: > > > > This needs to be fixed in pf then. > > > > > > Max Laier and I discussed this issue once, and Max had concern > > > over possible performance degradation that might result from > > > calling pflog functions through pointers to be set by a separate > > > pflog module. We can skip touching the pf module in RELENG_6 for > > > now and leave the issue to after 6.1-RELEASE is out. > > > > I have convinced myself that we should really use a function pointer here. > > I will try to commit a sollution to HEAD over the weekend. If you are > > MFC'ing the changes *now*, I'd appreciate if you could spare out pf, but I > > am willing to MFC the changes before 6.1 if testing goes well. > > Here it is. I'd appreciate feedback. pflog_packet() uses a lot of complex > types which makes it necessary to include pfvar.h. This is ugly, but I don't > know how to work around this. pflog_packet() takes pointers to the types, which are structs, so it should be possible to declare the structs opaquely. AFAIK this trick is legal C and used here and there in our code. E.g.: +#ifdef __FreeBSD__ +struct pfi_kif; +struct pf_rule; +struct pf_rulese; + +typedef int pflog_packet_t(struct pfi_kif *, struct mbuf *, sa_family_t, + u_int8_t, u_int8_t, struct pf_rule *, struct pf_rule *, + struct pf_ruleset *); +extern pflog_packet_t *pflog_packet_ptr; +#define PFLOG_PACKET(i,x,a,b,c,d,e,f,g) do { \ + if (pflog_packet_ptr != NULL) \ + pflog_packet_ptr(i,a,b,c,d,e,f,g); \ +} while (0) +#else Please see another small remark below. > -- > /"\ Best regards, | mlaier@freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier@EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News > Index: contrib/pf/net/if_pflog.c > =================================================================== > RCS file: /usr/store/mlaier/fcvs/src/sys/contrib/pf/net/if_pflog.c,v > retrieving revision 1.18 > diff -u -r1.18 if_pflog.c > --- contrib/pf/net/if_pflog.c 5 Dec 2005 11:58:31 -0000 1.18 > +++ contrib/pf/net/if_pflog.c 4 Feb 2006 15:09:11 -0000 > @@ -376,9 +376,15 @@ > case MOD_LOAD: > LIST_INIT(&pflog_list); > if_clone_attach(&pflog_cloner); > + PF_LOCK(); > + pflog_packet_ptr = pflog_packet; > + PF_UNLOCK(); > break; > > case MOD_UNLOAD: > + PF_LOCK(); > + pflog_packet_ptr = NULL; > + PF_UNLOCK(); > if_clone_detach(&pflog_cloner); > break; > > @@ -400,4 +406,5 @@ > > DECLARE_MODULE(pflog, pflog_mod, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_ANY); > MODULE_VERSION(pflog, PFLOG_MODVER); > +MODULE_DEPEND(pflog, pf, PF_MODVER, PF_MODVER, PF_MODVER); > #endif /* __FreeBSD__ */ > Index: contrib/pf/net/if_pflog.h > =================================================================== > RCS file: /usr/store/mlaier/fcvs/src/sys/contrib/pf/net/if_pflog.h,v > retrieving revision 1.6 > diff -u -r1.6 if_pflog.h > --- contrib/pf/net/if_pflog.h 10 Jun 2005 16:49:03 -0000 1.6 > +++ contrib/pf/net/if_pflog.h 4 Feb 2006 15:08:59 -0000 > @@ -70,10 +70,24 @@ > > #ifdef _KERNEL > > +#ifdef __FreeBSD__ > +/* XXX */ > +#include > + > +typedef int pflog_packet_t(struct pfi_kif *, struct mbuf *, sa_family_t, > + u_int8_t, u_int8_t, struct pf_rule *, struct pf_rule *, > + struct pf_ruleset *); > +extern pflog_packet_t *pflog_packet_ptr; > +#define PFLOG_PACKET(i,x,a,b,c,d,e,f,g) do { \ > + if (pflog_packet_ptr != NULL) \ > + pflog_packet_ptr(i,a,b,c,d,e,f,g); \ > +} while (0) > +#else > #if NPFLOG > 0 > #define PFLOG_PACKET(i,x,a,b,c,d,e,f,g) pflog_packet(i,a,b,c,d,e,f,g) > #else > #define PFLOG_PACKET(i,x,a,b,c,d,e,f,g) ((void)0) > #endif /* NPFLOG > 0 */ > +#endif /* __FreeBSD__ */ > #endif /* _KERNEL */ > #endif /* _NET_IF_PFLOG_H_ */ > Index: contrib/pf/net/pf_ioctl.c > =================================================================== > RCS file: /usr/store/mlaier/fcvs/src/sys/contrib/pf/net/pf_ioctl.c,v > retrieving revision 1.22 > diff -u -r1.22 pf_ioctl.c > --- contrib/pf/net/pf_ioctl.c 5 Dec 2005 11:58:31 -0000 1.22 > +++ contrib/pf/net/pf_ioctl.c 4 Feb 2006 15:09:30 -0000 > @@ -108,6 +108,10 @@ > #include > #endif /* NPFSYNC > 0 */ > > +#ifdef __FreeBSD__ > +#include > +#endif > + > #ifdef INET6 > #include > #include > @@ -230,6 +234,7 @@ > > static volatile int pf_pfil_hooked = 0; > struct mtx pf_task_mtx; > +pflog_packet_t *pflog_packet_ptr = NULL; > > void > init_pf_mutex(void) > Index: modules/Makefile > =================================================================== > RCS file: /usr/store/mlaier/fcvs/src/sys/modules/Makefile,v > retrieving revision 1.472 > diff -u -r1.472 Makefile > --- modules/Makefile 31 Jan 2006 23:11:35 -0000 1.472 > +++ modules/Makefile 3 Feb 2006 22:57:36 -0000 > @@ -180,6 +180,7 @@ > pcn \ > ${_pecoff} \ > ${_pf} \ > + ${_pflog} \ > plip \ > ${_pmc} \ > portalfs \ > @@ -307,6 +308,7 @@ > > .if !defined(NO_PF) || defined(ALL_MODULES) > _pf= pf > +_pflog= pflog > .endif > > .if ${MACHINE_ARCH} == "i386" > Index: modules/pf/Makefile > =================================================================== > RCS file: /usr/store/mlaier/fcvs/src/sys/modules/pf/Makefile,v > retrieving revision 1.8 > diff -u -r1.8 Makefile > --- modules/pf/Makefile 14 Oct 2005 23:30:14 -0000 1.8 > +++ modules/pf/Makefile 3 Feb 2006 22:46:23 -0000 > @@ -6,7 +6,6 @@ > > KMOD= pf > SRCS = pf.c pf_if.c pf_subr.c pf_osfp.c pf_ioctl.c pf_norm.c pf_table.c \ > - if_pflog.c \ > in4_cksum.c \ > opt_pf.h opt_inet.h opt_inet6.h opt_bpf.h > > @@ -15,7 +14,6 @@ > .if !defined(KERNBUILDDIR) > opt_pf.h: > echo "#define DEV_PF 1" > opt_pf.h > - echo "#define DEV_PFLOG 1" >> opt_pf.h If all is right, defining DEV_PF here shouldn't be needed. > opt_inet.h: > echo "#define INET 1" > opt_inet.h > Index: modules/pflog/Makefile > =================================================================== > RCS file: modules/pflog/Makefile > diff -N modules/pflog/Makefile > --- /dev/null 1 Jan 1970 00:00:00 -0000 > +++ modules/pflog/Makefile 3 Feb 2006 22:48:31 -0000 > @@ -0,0 +1,29 @@ > +# $FreeBSD: src/sys/modules/pf/Makefile,v 1.8 2005/10/14 23:30:14 yar Exp $ > + > +.PATH: ${.CURDIR}/../../contrib/pf/net > +.PATH: ${.CURDIR}/../../contrib/pf/netinet > +.PATH: ${.CURDIR}/../../netinet > + > +KMOD= pflog > +SRCS = if_pflog.c \ > + opt_pf.h opt_inet.h opt_inet6.h opt_bpf.h > + > +CFLAGS+= -I${.CURDIR}/../../contrib/pf > + > +.if !defined(KERNBUILDDIR) > +opt_pf.h: > + echo "#define DEV_PFLOG 1" > opt_pf.h Ditto for DEV_PFLOG. > +opt_inet.h: > + echo "#define INET 1" > opt_inet.h > + > +.if !defined(NO_INET6) > +opt_inet6.h: > + echo "#define INET6 1" > opt_inet6.h > +.endif > + > +opt_bpf.h: > + echo "#define DEV_BPF 1" > opt_bpf.h > +.endif > + > +.include Thanks for your work! -- Yar From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 11:36:37 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C4C416A420; Mon, 6 Feb 2006 11:36:37 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (comp.chem.msu.su [158.250.32.97]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E07943D6B; Mon, 6 Feb 2006 11:36:34 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.13.3/8.13.3) with ESMTP id k16BaTa9027274; Mon, 6 Feb 2006 14:36:30 +0300 (MSK) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.13.3/8.13.3/Submit) id k16BaTb5027273; Mon, 6 Feb 2006 14:36:29 +0300 (MSK) (envelope-from yar) Date: Mon, 6 Feb 2006 14:36:29 +0300 From: Yar Tikhiy To: Max Laier Message-ID: <20060206113629.GF59508@comp.chem.msu.su> References: <20060201005658.A70005@xorpc.icir.org> <200602021437.38385.max@love2party.net> <200602041616.57224.max@love2party.net> <200602051824.28100.max@love2party.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200602051824.28100.max@love2party.net> User-Agent: Mutt/1.5.9i Cc: freebsd-net@freebsd.org, Hajimu UMEMOTO , Luigi Rizzo Subject: Re: if_bridge.ko requires INET6... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 11:36:37 -0000 On Sun, Feb 05, 2006 at 06:24:20PM +0100, Max Laier wrote: > On Saturday 04 February 2006 16:16, Max Laier wrote: > > On Thursday 02 February 2006 14:37, Max Laier wrote: > > > On Thursday 02 February 2006 13:43, Yar Tikhiy wrote: > > > > > This needs to be fixed in pf then. > > > > > > > > Max Laier and I discussed this issue once, and Max had concern > > > > over possible performance degradation that might result from > > > > calling pflog functions through pointers to be set by a separate > > > > pflog module. We can skip touching the pf module in RELENG_6 for > > > > now and leave the issue to after 6.1-RELEASE is out. > > > > > > I have convinced myself that we should really use a function pointer > > > here. I will try to commit a sollution to HEAD over the weekend. If you > > > are MFC'ing the changes *now*, I'd appreciate if you could spare out pf, > > > but I am willing to MFC the changes before 6.1 if testing goes well. > > > > Here it is. I'd appreciate feedback. pflog_packet() uses a lot of complex > > types which makes it necessary to include pfvar.h. This is ugly, but I > > don't know how to work around this. > > FYI, just committed this to the tree. I'm sorry I was a bit too late with my remarks... -- Yar From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 17:15:12 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F417916A420 for ; Mon, 6 Feb 2006 17:15:11 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8635843D49 for ; Mon, 6 Feb 2006 17:15:11 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k16HFAo7050492 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Feb 2006 09:15:11 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43E78483.3030106@errno.com> Date: Mon, 06 Feb 2006 09:16:51 -0800 From: Sam Leffler User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051227) X-Accept-Language: en-us, en MIME-Version: 1.0 To: nielsen@memberwebs.com References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> <43E6E4B6.50100@errno.com> <20060206083013.99D32DCAA14@mail.npubs.com> In-Reply-To: <20060206083013.99D32DCAA14@mail.npubs.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 17:15:12 -0000 Nate Nielsen wrote: > Sam Leffler wrote: > >>Nate Nielsen wrote: >> >>>Adding polling to this driver does increase performance on embedded >>>systems. With my current patch (on a 233Mhz system), the throughput (in >>>this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to >>>24Mbits. >> >>I routinely get >20 Mb/s for a single client running upstream TCP >>netperf through a soekris 4511. If you are seeing 6Mb/s you have >>something else wrong. > > > Note I was talking about an *increase* of 6Mb/s. Sorry I missed that. OTOH I'm already routinely getting close to your cited figure on a platform that's ~half the cpu power so I still wonder what's going on. > > In addition my TCP stream ends on the box in question, which obviously > increases the load on the system, which brings the numbers we are both > seeing roughly in the same ballpark. Yes; the only interesting test in my opinion is when the box is acting as an ap or otherwise forwarding the packets. This has major impact on the performance characteristics but more clearly reflects real life use. > > >>I've not seen livelock in any situations though there are some issues >>with the priority of the taskqueue thread. > > > With a small number of simple self regulating packet streams (such as > TCP) livelock is not really an issue, as the the streams will slow their > transmit rate when the box gets near livelock and packets start dropping. > > However on more complex links where traffic is not (or not completely) > self regulating (ie: VOIP, other datagram streams, a high number of TCP > streams, asymetric routing) livelock because of interrupt performance is > a common occurance. Not in my experience but as I said there ARE priority issues with the taskqueue setup in cvs. > > >>Polling is not a panacea; >>you are potentially increasing latency which has ramifications. > > > Correct, whenever polling is in use (on ethernet as well) we see the > latency go up. In addition the system is under higher load when there is > no traffic. These are tradeoffs that one accepts when using polling. > Obviously DEVICE_POLLING is not configured by default. > > >>>However it should be noted, that the default behaviour (in 6.0 release) >>>seems to be that the hardware generates about around 2000 interrupts per >>>second at around 15 - 18 Mbits throughput. >> >>You need to identify what kind of interrupts there are and what type of >>ath hardware you are using. > > > The interrupts are the RX and TX interrupts. My polling additions don't > mask out any other interrupts. I see no statistics; are you sure you are not being pounded by phy errors. You haven't even answered my question about what ath devices you're using. > > >>You can trivially reduce the tx interrupt >>load by turning off interrupts on EOL and just using the periodic >>interrupts generated every N tx descriptors. > > > Thanks for the tip. I'm sure that would help. I'll look into that. If you are polling then doing that is irrelevant. > > >>But if you profile I >>suspect you will find the interrupt overhead is not significant relative >>to other costs. > > > The very act of the CPU servicing the interrupt (ie: saving registers, > switching stacks, etc...) causes overhead. In addition the interrupt > handling does not fall under the domain of the scheduler. > > This isn't just theory it has a tested real life impact. As I noted > earlier for a simple TCP stream over a wireless link throughput went up > by 6Mb/s. > > In my real world case: On a 233Mhz net4826 running GIF encapsulation, > IPSEC encryption, and wireless backhaul, throughput went from being > livelocked at 3.5Mb/s (and userland barely functioning) to over 10Mb/s > (with userland scheduled properly). This is with polling used in the sis > ethernet driver, the hifn crypto card driver, and the ath driver. > > Instead of each of these devices generating interrupts, polling (in my > case at 256 Hz) allows the system to function smoothly. Yes, there is > latency of up to 20ms, but that's a small tradeoff for the throughput > and stability increases. You've changed a bunch of stuff at once and can make no specific claims about the polling change to the ath driver. For all you know changing the way the crypto driver works is what bought you all your performance. Your config is so cpu-bound and i/o bound that buying back any cycles is going to be a big win. > > >>I'm not convinced polling is worthwhile w/o a major restructuring of the >>driver. OTOH this shouldn't stop you from pushing forward... > > > As you noted there are other performance enhancements that could be > made, and while I'd love to see them implemented, I fear they may be out > of my scope and available time. This polling addition is a simple > performance enhancement that helps my clients, and perhaps others would > also be interested. > > I'll attach the rough patch, to give an idea of the direction I'm > working in. Note that this patch is incomplete. It locks after a while, > which is probably due to the way I call the taskqueue callbacks. I'll > continue to work on this. Your patch changes the way interrupts are handled by calling the tx+rx processing directly from the polling routine w/o any locks. This breaks locking assumptions in the driver (as you noted in your comments). To make this work correctly you probably need to restructure the driver along the lines of all the other drivers to use a single lock. I've considered doing this because future additions for radar and apsd support will probably require immediate processing of the rx descriptors in the interrupt handler. The existing setup was an experiment to see if we could make the tx+rx paths run in parallel to get increased concurrency. It's worked out ok but has noticeable overhead on some platforms due to the additional locking overhead. > > Please understand that by posting this patch I'm not pressuring you to > help. Thanks again for your advice so far. I appreciate your working on this stuff. I'm mostly trying to prod you into measuring effects directly rather than inferring stuff based on indirect measurements. Your system starts off overloaded so reducing load can result in noticeable improvements but possibly not for the reasons you think. FWIW my polling changes are in the sam_wifi p4 branch; you can view it at http://perforce.freebsd.org. Sam From owner-freebsd-net@FreeBSD.ORG Mon Feb 6 21:11:23 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27EB116A420 for ; Mon, 6 Feb 2006 21:11:23 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) Received: from mail.npubs.com (mail.wsfamily.com [209.66.100.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FD7043D69 for ; Mon, 6 Feb 2006 21:11:12 +0000 (GMT) (envelope-from nielsen-list@memberwebs.com) From: Nate Nielsen User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051013) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sam Leffler References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> <43E6E4B6.50100@errno.com> <20060206083013.99D32DCAA14@mail.npubs.com> <43E78483.3030106@errno.com> Content-Type: multipart/mixed; boundary="------------030804060208050203030707" Message-Id: <20060206211904.56054DCA99E@mail.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Mon, 6 Feb 2006 21:19:05 +0000 (GMT) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: nielsen@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 21:11:23 -0000 This is a multi-part message in MIME format. --------------030804060208050203030707 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sam Leffler wrote: > I see no statistics; are you sure you are not being pounded by phy > errors. I've put together a small patch to the ath driver which exposes some interupt statistic sysctls, and a small shell script to read these and list them once per second. Both are attached. Here are the results from the hostap box, configured like so: ath0: flags=8843 mtu 1500 inet 10.9.6.1 netmask 0xffffff00 broadcast 10.9.6.255 ether 00:15:6d:10:17:03 media: IEEE 802.11 Wireless Ethernet autoselect status: associated ssid blah channel 165 bssid 00:15:6d:10:17:03 authmode OPEN privacy OFF txpowmax 57 dtimperiod 1 bintval 100 When not active, we see (rx interrupts, tx interrupts, and all other interrupts, respectively, one line per second): RX:0 TX:0 OTH:10 RX:0 TX:0 OTH:11 RX:0 TX:0 OTH:10 RX:0 TX:0 OTH:11 RX:0 TX:0 OTH:10 RX:0 TX:0 OTH:11 RX:0 TX:0 OTH:10 RX:0 TX:0 OTH:11 RX:0 TX:0 OTH:10 RX:0 TX:0 OTH:10 I'm guessing the 10 or 11 interrupts per second are softawre beacon interrupts. I did not spend time verifying this. Once we put load through box (around 20+ Mb/s, two TCP streams, both directions simultaneously): RX:2377 TX:506 OTH:12 RX:2209 TX:509 OTH:10 RX:2118 TX:465 OTH:11 RX:2143 TX:482 OTH:10 RX:2145 TX:496 OTH:11 RX:2154 TX:493 OTH:10 RX:2123 TX:507 OTH:10 RX:2089 TX:471 OTH:11 RX:2021 TX:451 OTH:9 RX:2148 TX:496 OTH:10 As you noted the TX interrupts are mitigated. During this time we have a normal amount of phy errors: # athstats ath0 | cut -c 62-70 phyerr 2170 0 64 71 99 77 102 87 68 72 65 > You haven't even answered my question about what ath devices > you're using. Sorry bout that. These are 802.11a Mini-PCI radios, the chipset is actually marked 'AR5213A' but I guess it's treated as a 5212: mem 0xe1000000-0xe100ffff irq 10 at device 10.0 on pci0 mac 5.9 phy 4.3 radio 3.6 Cheers, Nate --------------030804060208050203030707 Content-Type: text/x-patch; name="ath-intr-table.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="ath-intr-table.patch" Index: sys/dev/ath/if_ath.c =================================================================== RCS file: /home/ncvs/src/sys/dev/ath/if_ath.c,v retrieving revision 1.94.2.8 diff -p -U3 -r1.94.2.8 if_ath.c --- sys/dev/ath/if_ath.c 29 Jan 2006 07:33:27 -0000 1.94.2.8 +++ sys/dev/ath/if_ath.c 6 Feb 2006 21:08:53 -0000 @@ -200,6 +200,13 @@ static int ath_regdomain = 0; /* regul SYSCTL_INT(_hw_ath, OID_AUTO, regdomain, CTLFLAG_RD, &ath_regdomain, 0, "regulatory domain"); +/* XXXX TEMP: for testing number of interrupts */ +struct { int rx_intr; int tx_intr; int oth_intr; } intr_table; +SYSCTL_INT(_hw_ath, OID_AUTO, rx_intr, CTLFLAG_RD, &(intr_table.rx_intr), 0, "XXX TEMP: rx interrupts generated"); +SYSCTL_INT(_hw_ath, OID_AUTO, tx_intr, CTLFLAG_RD, &(intr_table.tx_intr), 0, "XXX TEMP: tx interrupts generated"); +SYSCTL_INT(_hw_ath, OID_AUTO, oth_intr, CTLFLAG_RD, &(intr_table.oth_intr), 0, "XXX TEMP: other interrupts generated"); +/* XXXXXX */ + #ifdef AR_DEBUG static int ath_debug = 0; SYSCTL_INT(_hw_ath, OID_AUTO, debug, CTLFLAG_RW, &ath_debug, @@ -258,6 +265,9 @@ ath_attach(u_int16_t devid, struct ath_s HAL_STATUS status; int error = 0, i; + /* XXX TEMP */ + memset(&intr_table, 0, sizeof(intr_table)); + DPRINTF(sc, ATH_DEBUG_ANY, "%s: devid 0x%x\n", __func__, devid); ifp = sc->sc_ifp = if_alloc(IFT_ETHER); @@ -712,6 +722,15 @@ ath_intr(void *arg) ath_hal_getisr(ah, &status); /* NB: clears ISR too */ DPRINTF(sc, ATH_DEBUG_INTR, "%s: status 0x%x\n", __func__, status); status &= sc->sc_imask; /* discard unasked for bits */ + + /* XXX TEMP: */ + if (status & HAL_INT_RX) + ++intr_table.rx_intr; + else if(status & HAL_INT_TX) + ++intr_table.tx_intr; + else + ++intr_table.oth_intr; + if (status & HAL_INT_FATAL) { /* * Fatal errors are unrecoverable. Typically --------------030804060208050203030707-- From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 05:45:04 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5CCFF16A431 for ; Tue, 7 Feb 2006 05:45:04 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8FAA143D4C for ; Tue, 7 Feb 2006 05:45:03 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 721681A3C20 for ; Mon, 6 Feb 2006 21:45:03 -0800 (PST) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id C121052053; Tue, 7 Feb 2006 00:45:02 -0500 (EST) Date: Tue, 7 Feb 2006 00:45:02 -0500 From: Kris Kennaway To: Kris Kennaway Message-ID: <20060207054502.GA18560@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="azLHFNyN32YCQGCU" Content-Disposition: inline In-Reply-To: <20060116004438.GA27901@xor.obsecurity.org> User-Agent: Mutt/1.4.2.1i Cc: net@FreeBSD.org Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 05:45:04 -0000 --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jan 15, 2006 at 07:44:38PM -0500, Kris Kennaway wrote: > I ran ntpdate on an amd64 system with ipv6 enabled and a skewed clock > (ntpdate stepped it back by about an hour), and immediately got a > use-after-free panic in ifaddr. When I rebooted with memguard enabled > on this malloc type and retried, I got this panic upon changing the > date forward, then back, then forward again (also note the garbage > return data from ntpdate): Has anyone looked at this? This is on the TODO list for 6.1, so the sooner it can be resolved the better. Kris > # date 200606011200 > Thu Jun 1 12:00:00 UTC 2006 > # ntpdate ntp.apple.com > 16 Jan 00:40:18 ntpdate[612]: step time server 17.254.0.28 offset -~9000p= m6}9426375508.195959 sec > # date 200606011200 > Thu Jun 1 12:00:00 UTC 2006 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0xffffffff91bd2198 > fault code =3D supervisor write, protection violation > instruction pointer =3D 0x8:0xffffffff80321346 > stack pointer =3D 0x10:0xffffffffbcfa1b60 > frame pointer =3D 0x10:0xffffffffbcfa1b90 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 14 (swi4: clock sio) > [thread pid 14 tid 100010 ] > Stopped at nd6_timer+0x106: movl %eax,0x198(%rbx) > db> wh > Tracing pid 14 tid 100010 td 0xffffff03e15d6c30 > nd6_timer() at nd6_timer+0x106 > softclock() at softclock+0x279 > ithread_execute_handlers() at ithread_execute_handlers+0x12f > ithread_loop() at ithread_loop+0x99 > fork_exit() at fork_exit+0xdf > fork_trampoline() at fork_trampoline+0xe > --- trap 0, rip =3D 0, rsp =3D 0xffffffffbcfa1d40, rbp =3D 0 --- >=20 > Unfortunately I can't dump on this system, but: >=20 > (kgdb) list *(nd6_timer+0x106) > 0xffffffff80321346 is in nd6_timer (../../../netinet6/nd6.c:585). > 580 goto addrloop; /* XXX: see below = */ > 581 } > 582 if (IFA6_IS_DEPRECATED(ia6)) { > 583 int oldflags =3D ia6->ia6_flags; > 584 > 585 ia6->ia6_flags |=3D IN6_IFF_DEPRECATED; > 586 > 587 /* > 588 * If a temporary address has just become= deprecated, > 589 * regenerate a new one if possible. >=20 > Kris >=20 >=20 --azLHFNyN32YCQGCU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFD6DPeWry0BWjoQKURAv8GAJ9ec5iw0ibNl5iqLtgUBLv0RWhiFwCgh3M+ zoPesXQhYIWn11rhlkEV050= =H1zj -----END PGP SIGNATURE----- --azLHFNyN32YCQGCU-- From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 10:16:18 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 19C4616A420 for ; Tue, 7 Feb 2006 10:16:18 +0000 (GMT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from shuttle.wide.toshiba.co.jp (shuttle.wide.toshiba.co.jp [202.249.10.124]) by mx1.FreeBSD.org (Postfix) with ESMTP id B763643D49 for ; Tue, 7 Feb 2006 10:16:17 +0000 (GMT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from impact.jinmei.org (unknown [2001:200:0:8002:c0fe:1358:5484:d6fe]) by shuttle.wide.toshiba.co.jp (Postfix) with ESMTP id D02391521A; Tue, 7 Feb 2006 19:16:16 +0900 (JST) Date: Tue, 07 Feb 2006 19:16:09 +0900 Message-ID: From: JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: Kris Kennaway In-Reply-To: <20060207054502.GA18560@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> User-Agent: Wanderlust/2.14.0 (Africa) Emacs/21.3 Mule/5.0 (SAKAKI) Organization: Research & Development Center, Toshiba Corp., Kawasaki, Japan. MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: net@FreeBSD.org Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 10:16:18 -0000 >>>>> On Tue, 7 Feb 2006 00:45:02 -0500, >>>>> Kris Kennaway said: >> I ran ntpdate on an amd64 system with ipv6 enabled and a skewed clock >> (ntpdate stepped it back by about an hour), and immediately got a >> use-after-free panic in ifaddr. When I rebooted with memguard enabled >> on this malloc type and retried, I got this panic upon changing the >> date forward, then back, then forward again (also note the garbage >> return data from ntpdate): > Has anyone looked at this? This is on the TODO list for 6.1, so the > sooner it can be resolved the better. Sorry, not really (we've not got a test environment to reproduce it). But from a quick review of nd6.c, there seems to be one thing that is obviously wrong. The possible bug has been there since rev. 1.19 committed in April 2002. We've been probably just lucky so far... Could you try the patch attached below? We'll probably also need to apply this fix to 4.X and 5.X. (a note about the patch: the first chunk is actually not related to the bug, but I could not miss the trivial typo) JINMEI, Tatuya Communication Platform Lab. Corporate R&D Center, Toshiba Corp. jinmei@isl.rdc.toshiba.co.jp --- nd6.c.orig Tue Feb 7 18:58:04 2006 +++ nd6.c Tue Feb 7 18:59:33 2006 @@ -547,7 +547,7 @@ /* * expire interface addresses. * in the past the loop was inside prefix expiry processing. - * However, from a stricter speci-confrmance standpoint, we should + * However, from a stricter spec-confrmance standpoint, we should * rather separate address lifetimes and prefix lifetimes. */ addrloop: @@ -578,8 +578,7 @@ if (regen) goto addrloop; /* XXX: see below */ - } - if (IFA6_IS_DEPRECATED(ia6)) { + } else if (IFA6_IS_DEPRECATED(ia6)) { int oldflags = ia6->ia6_flags; ia6->ia6_flags |= IN6_IFF_DEPRECATED; From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 14:11:36 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 763AC16A420; Tue, 7 Feb 2006 14:11:36 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (cell.sick.ru [217.72.144.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 589DA43D46; Tue, 7 Feb 2006 14:11:34 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (glebius@localhost [127.0.0.1]) by cell.sick.ru (8.13.3/8.13.3) with ESMTP id k17EBWP2030416 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Feb 2006 17:11:32 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.sick.ru (8.13.3/8.13.1/Submit) id k17EBWwq030415; Tue, 7 Feb 2006 17:11:32 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.sick.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 7 Feb 2006 17:11:31 +0300 From: Gleb Smirnoff To: Anders Nordby Message-ID: <20060207141131.GU877@FreeBSD.org> Mail-Followup-To: Gleb Smirnoff , Anders Nordby , freebsd-net@freebsd.org, demon@freebsd.org, harti@freebsd.org, kuriyama@freebsd.org References: <20060206092443.GA61116@totem.fix.no> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20060206092443.GA61116@totem.fix.no> User-Agent: Mutt/1.5.6i Cc: freebsd-net@FreeBSD.org, harti@FreeBSD.org, kuriyama@FreeBSD.org, demon@FreeBSD.org Subject: Re: 64-bit SNMP counters for FreeBSD && graphing bandwidth usage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 14:11:36 -0000 On Mon, Feb 06, 2006 at 10:24:44AM +0100, Anders Nordby wrote: A> Is there any way to have 64-bit SNMP counters in FreeBSD? Especially for A> ifInOctets/ifOutOctets. It seems the built-in bsnmpd only has A> Counter32, and net-snmpd the same (--enable-mfd-rewrites, which is A> supposed to help, seems to only work in Linux). Extended counters live in a separate subtree and bsnmpd supports them: > snmpwalk -v 2c host community ifMIB.ifMIBObjects.ifXTable.ifXEntry A> My systems are pushing more than 100 mbps these days, and graphing A> bandwidth usage with 32-bit counters gives funny-looking graphs. -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 15:38:20 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 316A216A420 for ; Tue, 7 Feb 2006 15:38:20 +0000 (GMT) (envelope-from saundersconsult@hotmail.com) Received: from hotmail.com (bay115-dav18.bay115.hotmail.com [65.54.250.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id E02BE43D45 for ; Tue, 7 Feb 2006 15:38:19 +0000 (GMT) (envelope-from saundersconsult@hotmail.com) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 7 Feb 2006 07:38:19 -0800 Message-ID: Received: from 24.205.176.247 by BAY115-DAV18.phx.gbl with DAV; Tue, 07 Feb 2006 15:38:18 +0000 X-Originating-IP: [24.205.176.247] X-Originating-Email: [saundersconsult@hotmail.com] X-Sender: saundersconsult@hotmail.com From: "Shawn Saunders" To: References: Date: Tue, 7 Feb 2006 07:38:21 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-OriginalArrivalTime: 07 Feb 2006 15:38:19.0345 (UTC) FILETIME=[85384C10:01C62BFC] Subject: Trying to make a Host into a gigabit hub for testing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 15:38:20 -0000 Hello, Based on the below configuration, does anyone have an idea of what I might be doing wrong? The following is a layout of the type of configuration, and I have tried one2many and hub, but was unable to obtain the desired results. Whenever I try to set multiple hooks to the same interface it fails, and trying to bring a group of interfaces to a virtual interface, and then take that virtual interface out multiple different interfaces, seems to fail (I can't even make the connection to the virtual interface to go to multiple outs.) I will have 11 interfaces inbound from 11 different networks. I will have 4 outbound to 4 different IDS's and sensors. I would like to have 11 interfaces come into a single virtual interface. This way I could run tcpdump and/or snort on the box itself and obtain quick info, and do validation of packet data, between the capture system and the IDS and sensors. I would like to redirect all the traffic from that one virtual interface to a group of 4 physical interfaces so the sensors can work on it. Here is a configuration I tried. Any comments or suggestions are appreciated. This initial script, only takes 3 input interfaces into a single virtual interface. This works. Then when I try to take that virtual interface and echo it out multiple interfaces, it fails. #!/bin/sh # Initialize and bring up all interfaces for i in 0 1 2 3 4 5 6 7 8 9 10 11 do /sbin/ifconfig em$i up done for g in 0 1 do /sbin/ifconfig bge$g up done /sbin/ifconfig fxp0 up # Load needed kernel modules /sbin/kldload /boot/kernel/ng_ether.ko /sbin/kldload /boot/kernel/ng_one2many.ko /sbin/kldload /boot/kernel/ng_fec.ko # Create Virtual Interface /usr/sbin/ngctl mkpeer fec dummy fec # Bind physical input interfaces to virtual interface /usr/sbin/ngctl msg fec0: add_iface '"em0"' /usr/sbin/ngctl msg fec0: add_iface '"em1"' /usr/sbin/ngctl msg fec0: add_iface '"em2"' # Set forwarding mode to mac address layer. /usr/sbin/ngctl msg fec0: set_mode_mac # Configure the virtual interface to deliver packets out the others ngctl mkpeer fec0: one2many upper one ngctl name fec0:upper secur ngctl connect bge0: secur: upper many0 ngctl connect bge1: secur: upper many1 ngctl msg secur: setconfig "{ xmitAlg=2 failAlg=1 enabledLinks=[ 1 1 ] }" # Set all interfaces Promisc mode and turn off autosrc routing for s in 0 1 2 3 4 5 6 7 8 9 10 11 do /usr/sbin/ngctl msg em$s: setpromisc 1 /usr/sbin/ngctl msg em$s: setautosrc 0 done for t in 0 1 do /usr/sbin/ngctl msg bge$t: setpromisc 1 /usr/sbin/ngctl msg bge$t: setautosrc 0 done #EOF Before sending 3 pings accross em2: gigihub# netstat -I bge0 ; netstat -I bge1 ; netstat -I fec0 ; netstat -I em2 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge0 1500 00:e0:81:32:f4:52 0 0 5 0 0 bge0 1500 fe80:b::2e0:8 fe80:b::2e0:81ff: 0 - 4 - - Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge1 1500 00:e0:81:32:f4:53 0 0 0 0 0 bge1 1500 fe80:c::2e0:8 fe80:c::2e0:81ff: 0 - 4 - - Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll fec0* 1500 00:04:23:c1:0e:50 11 0 0 0 0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em2 1500 00:04:23:c1:0e:50 41 0 5 0 0 em2 1500 fe80:5::204:2 fe80:5::204:23ff: 0 - 4 - - After sending 3 pings accross em2: gigihub# netstat -I bge0 ; netstat -I bge1 ; netstat -I fec0 ; netstat -I em2 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge0 1500 00:e0:81:32:f4:52 0 0 5 0 0 bge0 1500 fe80:b::2e0:8 fe80:b::2e0:81ff: 0 - 4 - - Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge1 1500 00:e0:81:32:f4:53 0 0 0 0 0 bge1 1500 fe80:c::2e0:8 fe80:c::2e0:81ff: 0 - 4 - - Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll fec0* 1500 00:04:23:c1:0e:50 15 0 0 0 0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em2 1500 00:04:23:c1:0e:50 45 0 5 0 0 em2 1500 fe80:5::204:2 fe80:5::204:23ff: 0 - 4 - - Shawn Saunders From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 16:18:35 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5895316A420 for ; Tue, 7 Feb 2006 16:18:35 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id E74F243D55 for ; Tue, 7 Feb 2006 16:18:34 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 677F55CC1; Tue, 7 Feb 2006 11:18:33 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 05669-05; Tue, 7 Feb 2006 11:18:30 -0500 (EST) Received: from [192.168.1.3] (pool-68-161-67-226.ny325.east.verizon.net [68.161.67.226]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 983675CB3; Tue, 7 Feb 2006 11:18:30 -0500 (EST) Message-ID: <43E8C861.5070209@mac.com> Date: Tue, 07 Feb 2006 11:18:41 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Shawn Saunders References: In-Reply-To: X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-net@freebsd.org Subject: Re: Trying to make a Host into a gigabit hub for testing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 16:18:35 -0000 Shawn Saunders wrote: > The following is a layout of the type of configuration, and I have tried > one2many and hub, but was unable to obtain the desired results. Whenever I > try to set multiple hooks to the same interface it fails, and trying to > bring a group of interfaces to a virtual interface, and then take that > virtual interface out multiple different interfaces, seems to fail (I can't > even make the connection to the virtual interface to go to multiple outs.) > > I will have 11 interfaces inbound from 11 different networks. > I will have 4 outbound to 4 different IDS's and sensors. > I would like to have 11 interfaces come into a single virtual interface. > This way I could run tcpdump and/or snort on the box itself and obtain quick > info, and do validation of packet data, between the capture system and the > IDS and sensors. I would like to redirect all the traffic from that one > virtual interface to a group of 4 physical interfaces so the sensors can work > on it. What you're trying to do doesn't seem to make a lot of sense. While you can use ng_fec to channel-bond two NICs on the same subnet/collision domain for redundancy, or you could implement bridging instead, but you can't use either when the NICs are on different interfaces. Tools like tcpdump prefer to work on a per-interface basis for a reason, this is how the underlying BPF mechanism looks at things, although I believe there is a flag to listen on all interfaces (might be Linux-only?). Unless you've got remarkable hardware, you're not going to be able to fastforward eleven GB NIC's worth of traffic, much less "do validation of packet data". About the closest thing that would do what you've asked for is a Cisco PIX 535, and even that can only take nine GB interfaces (at a $40,000 pricetag, give or take). I guess you could buy two of them, though. Maybe you could get a 12-port GB managed switch with a "roving analysis" port, and configure each interface to a different VLAN, and connect your existing machine to that. Of course, if the other interfaces are busy enough, you're not going to fit all of that traffic into the analysis port. -- -Chuck From owner-freebsd-net@FreeBSD.ORG Tue Feb 7 16:38:33 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC43016A420 for ; Tue, 7 Feb 2006 16:38:33 +0000 (GMT) (envelope-from james.juran@baesystems.com) Received: from smtp4.na.baesystems.com (smtp4.na.baesystems.com [63.164.202.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37BCC43D45 for ; Tue, 7 Feb 2006 16:38:31 +0000 (GMT) (envelope-from james.juran@baesystems.com) Received: from BLUMS0022.bluelnk.net (blums0022.na.baesystems.com [10.40.96.145]) by smtp4.na.baesystems.com (8.12.10/8.12.10) with ESMTP id k17GcUfW008060 for ; Tue, 7 Feb 2006 11:38:30 -0500 (EST) Received: from vahqex3.digitalnet.com (vahqex3.gfgsi.com [159.94.37.43]) by smtp1.na.baesystems.com (8.12.10/8.12.10) with ESMTP id k17GcHNr002115 for ; Tue, 7 Feb 2006 11:38:17 -0500 (EST) Received: from juran.digitalnet.com ([159.94.6.27]) by vahqex3.digitalnet.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 7 Feb 2006 11:38:10 -0500 Received: from localhost (localhost [127.0.0.1]) by juran.digitalnet.com (8.13.4/8.13.4) with ESMTP id k17GcAOv018541 for ; Tue, 7 Feb 2006 11:38:10 -0500 From: James Juran To: freebsd-net@freebsd.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-Y6lvKWoTEX8WbUpefxAQ" Organization: BAE Systems Information Technology, LLC Date: Tue, 07 Feb 2006 11:38:09 -0500 Message-Id: <1139330290.13750.11.camel@juran.digitalnet.com> Mime-Version: 1.0 X-Mailer: Evolution 2.2.3 (2.2.3-2.fc4) X-OriginalArrivalTime: 07 Feb 2006 16:38:10.0656 (UTC) FILETIME=[E1CEDE00:01C62C04] Subject: Zeroing wrong union member in in6_control() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2006 16:38:33 -0000 --=-Y6lvKWoTEX8WbUpefxAQ Content-Type: text/plain Content-Transfer-Encoding: quoted-printable In what looks like a copy&paste remnant from the preceding case, the wrong union member is used as the first argument to bzero in in6_control(). This doesn't cause an actual bug, but making this change would improve code clarity and robustness to change and also avoids a warning from a certain static analysis tool. I'm not a regular FreeBSD contributor, so if this patch is worthwhile can someone please apply it? If I should send things like this to a different mailing list in the future, please let me know. Index: in6.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/netinet6/in6.c,v retrieving revision 1.59 diff -u -p -f -u -p -r1.59 in6.c --- in6.c 31 Oct 2005 23:06:04 -0000 1.59 +++ in6.c 7 Feb 2006 16:30:21 -0000 @@ -568,7 +568,7 @@ in6_control(so, cmd, data, ifp, td) case SIOCGIFSTAT_ICMP6: if (ifp =3D=3D NULL) return EINVAL; - bzero(&ifr->ifr_ifru.ifru_stat, + bzero(&ifr->ifr_ifru.ifru_icmp6stat, sizeof(ifr->ifr_ifru.ifru_icmp6stat)); ifr->ifr_ifru.ifru_icmp6stat =3D *((struct in6_ifextra *)ifp->if_afdata[AF_INET6])->icmp6_ifstat; --=20 James Juran Senior Secure Systems Analyst XTS-400 Operating Systems BAE Systems Information Technology James.Juran@baesystems.com (703) 563-8081 --=-Y6lvKWoTEX8WbUpefxAQ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQBD6Mzx5VGI7rYqFQsRAtljAJ0ajT3zXUkcDFly4U0DII+IFHz4kACfUh0n tGTDpxYlMMj14mMrj2qE7fM= =/YCe -----END PGP SIGNATURE----- --=-Y6lvKWoTEX8WbUpefxAQ-- From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 01:06:21 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59C7016A420 for ; Wed, 8 Feb 2006 01:06:21 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from smtp.openaccess.org (smtp.openaccess.org [66.165.52.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBC8143D45 for ; Wed, 8 Feb 2006 01:06:20 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from [216.57.214.90] (unknown [216.57.214.90]) by smtp.openaccess.org (Postfix) with ESMTP id 81FAC6D4456; Tue, 7 Feb 2006 17:06:19 -0800 (PST) In-Reply-To: <20060206050301.AC895DCAA3F@mail.npubs.com> References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> Mime-Version: 1.0 (Apple Message framework v746.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <8D39FF7E-646E-4770-9ABF-337F69344EA7@staff.openaccess.org> Content-Transfer-Encoding: 7bit From: Michael DeMan Date: Tue, 7 Feb 2006 17:06:29 -0800 To: nielsen@memberwebs.com X-Mailer: Apple Mail (2.746.2) Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 01:06:21 -0000 Hi, Just my 2-cents, but we've found polling to be extremely valuable on low-end hardware as described here. We use it only on fxp drivers, but it moved throughput on 133Mhz hardware from something around 8Mbps to 20Mbps on regular layer-3 packet forwarding and also bumped VPN performance up significantly when used with hardware VPN accelerators. Also, since you can force the system to reserve a certain percent for non-polling tasks, you still have SSH access even when it is running hard, although I suppose at the risk of perhaps packet loss if your userland applications use a lot of CPU. I have no idea on FBSD 5.4 or 6.0 though and would be curious. We're running 5.4 on our normal servers now, but I've been nervous moving the low end equipment off of 4.x because of performance issues. - mike Michael F. DeMan Director of Technology OpenAccess Network Services Bellingham, WA 98225 michael@staff.openaccess.org 360-647-0785 On Feb 5, 2006, at 9:03 PM, Nate Nielsen wrote: > Sam Leffler wrote: >> You might try explaining why you think polling helps your >> performance. >> Unless you've significantly restructured the interrupt handling in >> the >> driver most work is deferred to a non-interrupt context. > > Yes, I saw that. However the interrupts themselves when they are fired > at upwards of a thousand per second do greatly affect performance and > scheduling on a slow box. > > I understand that it may not seem intuitive on faster systems which > are > completely capable of handling such a small interrupt load. But these > embedded systems are slow little puppies, running between 133 and 266 > Mhz (586 class chip). > > Adding polling to this driver does increase performance on embedded > systems. With my current patch (on a 233Mhz system), the throughput > (in > this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to > 24Mbits. > > But that's not the main benefit. The main thing is the scheduling. > Without having the thousands of interrupts, the box is better able to > balance the RX/TX with the other aspects, such as encapsulation, > packet > filtering and other activities. > > When the entire box is polling driven, it's total throughput > (ethernet, > encapsulation, hardware encryption, wireless) increases greatly and > does > not exhibit livelock symptoms. > > Without polling, these slow systems easily exhibit livelock. This is > where incoming traffic can cause so many interrupts that outgoing > traffic is completely halted and the throughput drops to zero or near > zero. Under these conditions userland processes also run barely or not > at all. The scheduler has no say in what's going on in the system, as > interrupts overwhelm all other activity. > >> Also the >> driver in 6.0 and later does tx interrupt mitigation and rx interrupt >> coalescing so I wouldn't expect interrupt handling to be a >> performance >> limitation. > > Interesting. If there's an option to enable it, I very well may have > missed it. > > However it should be noted, that the default behaviour (in 6.0 > release) > seems to be that the hardware generates about around 2000 > interrupts per > second at around 15 - 18 Mbits throughput. > >> There are other issues that can affect performance but you >> haven't mentioned them... > > Obviously these are not mainstream performance issues for people just > trying to connect to an access point. But when the atheros cards are > used in backhaul applications and are running in the low power > embedded > systems you typically see on an antenna mast, polling makes a big > difference. > > Cheers, > Nate > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 02:46:15 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA6D816A420 for ; Wed, 8 Feb 2006 02:46:15 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E00543D46 for ; Wed, 8 Feb 2006 02:46:15 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k182kCo7060034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Feb 2006 18:46:14 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43E95C1C.2030509@errno.com> Date: Tue, 07 Feb 2006 18:49:00 -0800 From: Sam Leffler User-Agent: Thunderbird 1.5 (X11/20060117) MIME-Version: 1.0 To: Michael DeMan References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> <8D39FF7E-646E-4770-9ABF-337F69344EA7@staff.openaccess.org> In-Reply-To: <8D39FF7E-646E-4770-9ABF-337F69344EA7@staff.openaccess.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@FreeBSD.org" , nielsen@memberwebs.com Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 02:46:15 -0000 Michael DeMan wrote: > Hi, > > Just my 2-cents, but we've found polling to be extremely valuable on > low-end hardware as described here. > > We use it only on fxp drivers, but it moved throughput on 133Mhz > hardware from something around 8Mbps to 20Mbps on regular layer-3 packet > forwarding and also bumped VPN performance up significantly when used > with hardware VPN accelerators. > > Also, since you can force the system to reserve a certain percent for > non-polling tasks, you still have SSH access even when it is running > hard, although I suppose at the risk of perhaps packet loss if your > userland applications use a lot of CPU. > > I have no idea on FBSD 5.4 or 6.0 though and would be curious. We're > running 5.4 on our normal servers now, but I've been nervous moving the > low end equipment off of 4.x because of performance issues. I'm not arguing against polling. I started by saying that adding polling to ath wasn't worthwhile w/o restructuring the driver. I then asked Nate to split out various effects so the polling changes could be evaluated independent of other changes (e.g. polling in the hifn driver). I've done NAPI code for the linux version of the ath driver for embedded applications. If I were building ap's on underpowered platforms like the soekris I'd probably do similar changes. But using polling would depend on the application because it introduces latency and some applications cannot tolerate that latency. I know this from direct experience--try implementing HCF for example. Sam From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 05:56:04 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EDE116A420 for ; Wed, 8 Feb 2006 05:56:04 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from mrout3.yahoo.com (mrout3.yahoo.com [216.145.54.173]) by mx1.FreeBSD.org (Postfix) with ESMTP id 16A6E43D45 for ; Wed, 8 Feb 2006 05:56:03 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (proxy8.corp.yahoo.com [216.145.48.13]) by mrout3.yahoo.com (8.13.4/8.13.4/y.out) with ESMTP id k185qLd3082535; Tue, 7 Feb 2006 21:52:21 -0800 (PST) Date: Wed, 08 Feb 2006 14:52:27 +0900 Message-ID: From: gnn@FreeBSD.org To: James Juran In-Reply-To: <1139330290.13750.11.camel@juran.digitalnet.com> References: <1139330290.13750.11.camel@juran.digitalnet.com> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.7 (=?ISO-8859-4?Q?Sanj=F2?=) APEL/10.6 Emacs/22.0.50 (powerpc-apple-darwin8.3.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: freebsd-net@FreeBSD.org Subject: Re: Zeroing wrong union member in in6_control() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 05:56:04 -0000 At Tue, 07 Feb 2006 11:38:09 -0500, James Juran wrote: > > [1 ] > In what looks like a copy&paste remnant from the preceding case, the > wrong union member is used as the first argument to bzero in > in6_control(). This doesn't cause an actual bug, but making this change > would improve code clarity and robustness to change and also avoids a > warning from a certain static analysis tool. > > I'm not a regular FreeBSD contributor, so if this patch is worthwhile > can someone please apply it? If I should send things like this to a > different mailing list in the future, please let me know. The Kame list is still the best one for this (kame ) but I've forwarded it for you. I'll take care of getting this into FreeBSD though. Thanks, George From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 17:10:32 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C1D2516A420 for ; Wed, 8 Feb 2006 17:10:32 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from maritaca.epm.br (disrouter.epm.br [200.17.25.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B58843D45 for ; Wed, 8 Feb 2006 17:10:32 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from localhost (localhost.localdomain [127.0.0.1]) by maritaca.epm.br (Postfix) with ESMTP id 09D333A74 for ; Wed, 8 Feb 2006 15:10:31 -0200 (BRDT) Received: from [172.22.1.166] (ricardo.epm.br [172.22.1.166]) by maritaca.epm.br (Postfix) with ESMTP id B39C03A71 for ; Wed, 8 Feb 2006 15:10:26 -0200 (BRDT) Message-ID: <43EA25EF.3020105@yahoo.com.br> Date: Wed, 08 Feb 2006 15:10:07 -0200 From: "Ricardo A. Reis" User-Agent: Thunderbird 1.5 (X11/20060126) MIME-Version: 1.0 To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit UNIFESP-Virus-Scanned: by amavisd-new at dis.epm.br Cc: Subject: RELENG_6 No buffer space available [udp stream] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 17:10:32 -0000 Hi, I have a problem with iperf and single udp stream, [freebsd - RELENG_6] - >>> [ linux-2.6.X] I in use server mode and no problem occurred, but in client mode this no complete a udp stream iperf -c 172.22.1.142 -u -b 100M -t 30 ------------------------------------------------------------ Client connecting to 172.22.1.142, UDP port 5001 Sending 1470 byte datagrams UDP buffer size: 41.1 KByte (default) ------------------------------------------------------------ [ 3] local 172.22.1.166 port 51517 connected with 172.22.1.142 port 5001 write2 failed: No buffer space available [ 3] 0.0- 0.1 sec 1.45 MBytes 100 Mbits/sec [ 3] Sent 1037 datagrams [ 3] Server Report: # netstat -mb 1/839/840 mbufs in use (current/cache/total) 0/438/438/16832 mbuf clusters in use (current/cache/total/max) 0/4/4464 sfbufs in use (current/peak/max) 0K/1085K/1086K bytes allocated to network (current/cache/total) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 11 calls to protocol drain routines Thanks Ricardo A. Reis UNIFESP Unix and Network Admin From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 18:24:31 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F33E116A420 for ; Wed, 8 Feb 2006 18:24:30 +0000 (GMT) (envelope-from piston@otel.net) Received: from mail.otel.net (gw3.OTEL.net [212.36.8.151]) by mx1.FreeBSD.org (Postfix) with ESMTP id 653F843D49 for ; Wed, 8 Feb 2006 18:24:29 +0000 (GMT) (envelope-from piston@otel.net) Received: from devilspot.otel.net ([212.36.8.194]) by mail.otel.net with smtp (Exim 4.60 (FreeBSD)) (envelope-from ) id 1F6tzN-0009E3-0Q for freebsd-net@FreeBSD.org; Wed, 08 Feb 2006 20:24:26 +0200 Date: Wed, 8 Feb 2006 20:24:57 +0200 From: "S.I" To: freebsd-net@FreeBSD.org Message-Id: <20060208202457.76637c92.piston@otel.net> Organization: OTEL.net X-Mailer: Sylpheed version 2.0.4 (GTK+ 2.8.8; i386-portbld-freebsd5.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: wifi client ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 18:24:31 -0000 Hi, how can i do this on the wireless client PC egg. iwconfig eth0 ap mac under FreeBSD.And is there any software egg. http://www.janmorgenstern.de/projects-software.html for wireless monitoring. Thanks, S.I From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 20:35:51 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B10816A420; Wed, 8 Feb 2006 20:35:51 +0000 (GMT) (envelope-from qingli@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E11643D49; Wed, 8 Feb 2006 20:35:51 +0000 (GMT) (envelope-from qingli@FreeBSD.org) Received: from freefall.freebsd.org (qingli@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k18KZoT9015563; Wed, 8 Feb 2006 20:35:50 GMT (envelope-from qingli@freefall.freebsd.org) Received: (from qingli@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k18KZot0015562; Wed, 8 Feb 2006 20:35:50 GMT (envelope-from qingli) Date: Wed, 8 Feb 2006 20:35:50 GMT From: Qing Li Message-Id: <200602082035.k18KZot0015562@freefall.freebsd.org> To: freebsd-questions@freebsd.org Cc: freebsd-net@freebsd.org Subject: Re: Multiple routes to same destination X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 20:35:51 -0000 I have a private patch that's based on radix_mpath for FreeBSD 5.4. I believe andre@freebsd.org is working on a solution. -- Qing -----Original Message----- From: owner-freebsd-questions@freebsd.org [mailto:owner-freebsd-questions@freebsd.org] On Behalf Of Webster, Andrew Sent: Wednesday, February 08, 2006 10:51 AM To: danial_thom@yahoo.com; Ian Lord; freebsd-questions@freebsd.org Subject: RE: Multiple routes to same destination? Well, in that case, an ISP wouldn't want to use FreeBSD in their core routers :( :( In this particular case, I have redundant links (L1 and L2) between two locations (Loc 1 and Loc 2) with two FreeBSD routers at each location (R1/R2, and R3/R4) which are running OSPF to redistribute routing information between locations. Since FreeBSD limits the entries for a particular network to only one active entry, the all the traffic for would either go on R1->L1->R4 or R2->L2->R3, but not both. Loc 1___ /---R1--L1--R4---\___ Loc 2 \---R2--L2--R3---/ Andrew From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 21:23:05 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 256B716A420; Wed, 8 Feb 2006 21:23:05 +0000 (GMT) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 38B3443D53; Wed, 8 Feb 2006 21:23:04 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [10.251.23.146]) ([10.251.23.146]) by a50.ironport.com with ESMTP; 08 Feb 2006 13:23:04 -0800 Message-ID: <43EA6137.9050606@elischer.org> Date: Wed, 08 Feb 2006 13:23:03 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.11) Gecko/20050727 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Qing Li References: <200602082035.k18KZot0015562@freefall.freebsd.org> In-Reply-To: <200602082035.k18KZot0015562@freefall.freebsd.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Multiple routes to same destination X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 21:23:05 -0000 Qing Li wrote: I use mpd to greate one VPN between the sites, using Multilink PPP, so that data is sent across both links (eitehr round-robon or split packet). I use MPD's udp transport mode to open two UDP sockets and send packets from R1 to R4 and from R2 to R3 (in the diagram below). MPD will automatically detect if on e link is down and redirect everything through the remaining link. > I have a private patch that's based on radix_mpath for FreeBSD 5.4. > I believe andre@freebsd.org is working on a solution. > > -- Qing > > >-----Original Message----- >From: owner-freebsd-questions@freebsd.org [mailto:owner-freebsd-questions@freebsd.org] On Behalf Of Webster, Andrew >Sent: Wednesday, February 08, 2006 10:51 AM >To: danial_thom@yahoo.com; Ian Lord; freebsd-questions@freebsd.org >Subject: RE: Multiple routes to same destination? > >Well, in that case, an ISP wouldn't want to use FreeBSD in their core routers :( :( > >In this particular case, I have redundant links (L1 and L2) between two locations >(Loc 1 and Loc 2) with two FreeBSD routers at each location (R1/R2, and R3/R4) >which are running OSPF to redistribute routing information between locations. >Since FreeBSD limits the entries for a particular network to only one active entry, >the all the traffic for would either go on R1->L1->R4 or R2->L2->R3, but not both. > >Loc 1___ /---R1--L1--R4---\___ Loc 2 > \---R2--L2--R3---/ > > >Andrew > >_______________________________________________ >freebsd-net@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 22:40:06 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D531616A420 for ; Wed, 8 Feb 2006 22:40:06 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5C5BF43D49 for ; Wed, 8 Feb 2006 22:40:06 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 102A746C4E; Wed, 8 Feb 2006 17:39:54 -0500 (EST) Date: Wed, 8 Feb 2006 22:42:40 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: "Ricardo A. Reis" In-Reply-To: <43EA25EF.3020105@yahoo.com.br> Message-ID: <20060208223956.X57882@fledge.watson.org> References: <43EA25EF.3020105@yahoo.com.br> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: net@freebsd.org Subject: Re: RELENG_6 No buffer space available [udp stream] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 22:40:06 -0000 On Wed, 8 Feb 2006, Ricardo A. Reis wrote: > I have a problem with iperf and single udp stream, > > [freebsd - RELENG_6] - >>> [ linux-2.6.X] > > I in use server mode and no problem occurred, but in client mode this no > complete a udp stream > > iperf -c 172.22.1.142 -u -b 100M -t 30 > ------------------------------------------------------------ > Client connecting to 172.22.1.142, UDP port 5001 > Sending 1470 byte datagrams > UDP buffer size: 41.1 KByte (default) > ------------------------------------------------------------ > [ 3] local 172.22.1.166 port 51517 connected with 172.22.1.142 port 5001 > write2 failed: No buffer space available ENOBUFS is the error returned when a process is sending packets faster than the transmitting interface can actually transmit them. It occurs when the interface send queue fills, and the packet is dropped. As UDP is a lossy datagram protocol without flow control, your process doesn't block, but it does get back an error telling it that the packet could not be transmitted. I'm not sure how Linux behaves under the same conditions -- it could be there is silent drop (send returns 0, but the packet is dropped). If the application has been written without knowing that send() can fail for UDP, it might not know how to take that into account, and might exit/error out improperly. Robert N M Watson > [ 3] 0.0- 0.1 sec 1.45 MBytes 100 Mbits/sec > [ 3] Sent 1037 datagrams > [ 3] Server Report: > > > # netstat -mb > 1/839/840 mbufs in use (current/cache/total) > 0/438/438/16832 mbuf clusters in use (current/cache/total/max) > 0/4/4464 sfbufs in use (current/peak/max) > 0K/1085K/1086K bytes allocated to network (current/cache/total) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 11 calls to protocol drain routines > > > Thanks > > Ricardo A. Reis > UNIFESP > Unix and Network Admin > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 10:38:40 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A399516A420 for ; Thu, 9 Feb 2006 10:38:40 +0000 (GMT) (envelope-from V.Ovsyannikov@kr.ru) Received: from ns.kr.ru (ns.kr.ru [84.22.128.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 38BCA43D55 for ; Thu, 9 Feb 2006 10:38:40 +0000 (GMT) (envelope-from V.Ovsyannikov@kr.ru) Received: from gravis.ncux.ru (gravis.ncux.ru [84.22.128.254]) by ns.kr.ru (Postfix) with ESMTP id 5225422E92 for ; Thu, 9 Feb 2006 17:38:38 +0700 (KRAT) Date: Thu, 9 Feb 2006 17:39:23 +0700 From: Vitaliy Ovsyannikov X-Priority: 3 (Normal) Message-ID: <176144277.20060209173923@kr.ru> To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: incomplete+permanent arp entries X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Vitaliy Ovsyannikov List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 10:38:40 -0000 Hello, freebsd-net. I've noticed what it is possible to make incomplete+permanent entries in kernel arp table. xx# arp -s 1.2.3.4 pub arp: invalid Ethernet address 'pub' # arp 1.2.3.4 1.2.3.4 (1.2.3.4) at (incomplete) on vlan1 permanent [vlan] In result of it's permanent entry, kernel doesn't listening for arp-replies for this host. My question is simple: is it bug or feature? -- Sincerely, Vitaliy Ovsyannikov JSC Skala, Krasnoyarsk, Russia From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 12:27:15 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C892716A420; Thu, 9 Feb 2006 12:27:15 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from maritaca.epm.br (disrouter.epm.br [200.17.25.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 237F143D81; Thu, 9 Feb 2006 12:27:09 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from localhost (localhost.localdomain [127.0.0.1]) by maritaca.epm.br (Postfix) with ESMTP id B72863B4D; Thu, 9 Feb 2006 10:27:07 -0200 (BRDT) Received: from [172.22.1.166] (ricardo.epm.br [172.22.1.166]) by maritaca.epm.br (Postfix) with ESMTP id 589253B40; Thu, 9 Feb 2006 10:27:03 -0200 (BRDT) Message-ID: <43EB3502.8060701@yahoo.com.br> Date: Thu, 09 Feb 2006 10:26:42 -0200 From: "Ricardo A. Reis" User-Agent: Thunderbird 1.5 (X11/20060126) MIME-Version: 1.0 To: Robert Watson References: <43EA25EF.3020105@yahoo.com.br> <20060208223956.X57882@fledge.watson.org> In-Reply-To: <20060208223956.X57882@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit UNIFESP-Virus-Scanned: by amavisd-new at dis.epm.br Cc: net@freebsd.org Subject: Re: RELENG_6 No buffer space available [udp stream] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 12:27:15 -0000 Hi Robert, Thanks for your explanation, i don't have ideas about with send() work on freebsd and linux, reading the man in linux and search per ENOBUFS, FYI --------- ENOBUFS The output queue for a network interface was full. This generally indicates that the interface has stopped sending, but may be caused by transient congestion. (Normally, this does not occur in Linux. Packets are just silently dropped when a device queue overflows.) --------- For measure network throughput you recommend netperf or iperf ? Or any other option ? Thanks Ricardo A. Reis UNIFESP Unix and Network Admin >> >> write2 failed: No buffer space available > > ENOBUFS is the error returned when a process is sending packets faster > than the transmitting interface can actually transmit them. It occurs > when the interface send queue fills, and the packet is dropped. As > UDP is a lossy datagram protocol without flow control, your process > doesn't block, but it does get back an error telling it that the > packet could not be transmitted. I'm not sure how Linux behaves under > the same conditions -- it could be there is silent drop (send returns > 0, but the packet is dropped). If the application has been written > without knowing that send() can fail for UDP, it might not know how to > take that into account, and might exit/error out improperly. > > Robert N M Watson From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 12:38:53 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0334016A420 for ; Thu, 9 Feb 2006 12:38:53 +0000 (GMT) (envelope-from geir.egeland@gmail.com) Received: from uproxy.gmail.com (uproxy.gmail.com [66.249.92.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1E58C43D46 for ; Thu, 9 Feb 2006 12:38:51 +0000 (GMT) (envelope-from geir.egeland@gmail.com) Received: by uproxy.gmail.com with SMTP id m2so66346uge for ; Thu, 09 Feb 2006 04:38:47 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:user-agent:mime-version:to:subject:x-enigmail-version:content-type:content-transfer-encoding; b=i9peV+FjgJc2PLd3i4SmNESAGxBw9BX2XZNw4bOtch7TAGwZYZflzABiN4m0IQMfWWF49x/EAPw389fCMOVdrnuRNkufMaJL4iGcPB2lnOED+tgEl/Fy2SljHKRKTDP+MYfwbpGj33T4qGSwXnuKqQ8w/EENmJxoFDfLcBZ+TIA= Received: by 10.67.26.6 with SMTP id d6mr4049849ugj; Thu, 09 Feb 2006 04:32:13 -0800 (PST) Received: from ?128.39.21.12? ( [128.39.21.12]) by mx.gmail.com with ESMTP id k30sm387000ugc.2006.02.09.04.32.13; Thu, 09 Feb 2006 04:32:13 -0800 (PST) Message-ID: <43EB3640.8090906@gmail.com> Date: Thu, 09 Feb 2006 13:32:00 +0100 From: Geir Egeland User-Agent: Thunderbird 1.5 (X11/20060112) MIME-Version: 1.0 To: freebsd-net@freebsd.org, freebsd-questions@freebsd.org X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: IEEE 802.11 Wireless Multimedia Extension (WME) and raw sockets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 12:38:53 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I've been playing around with WME to test various network performance, and come across a problem that I can't quite understand. I have an application that generates traffic with various TOS (BACKGROUND, BEST EFFORT, VOICE, VIDEO). It uses raw sockets to transmit the IP packets. This all works well if ip->ip_len is less than 192 bytes. If ip_>ip_len is larger than 192, the call to ieee80211_classify (/usr/src/sys/net80211/ieee80211_output.c) will classify the packet as "BEST EFFORT" no matter what value my application set the TOS field as. Debugging ieee80211_classify, I see that both ip->ip_tos and ip->ip_len are set to zero when a I send a packet with ip->ip_len larger than 192 bytes. Sniffing the network, I can see my packets have the correct TOS and length, but they don't get the correct WME classification. - -------------ieee80211_output.c(iee80211_classify)------------ if (eh->ether_type == htons(ETHERTYPE_IP)) { const struct ip *ip = (struct ip *) (mtod(m, u_int8_t *) + sizeof (*eh)); /* * IP frame, map the TOS field. */ //added by myself printf("IP_TOS: %d, IP_LEN: %d\n",ip->ip_tos,ntohl(ip->ip_len)); //end switch (ip->ip_tos) { case 0x08: case 0x20: d_wme_ac = WME_AC_BK; /* background */ break; case 0x28: case 0xa0: d_wme_ac = WME_AC_VI; /* video */ break; case 0x30: /* voice */ case 0xe0: case 0x88: /* XXX UPSD */ case 0xb8: d_wme_ac = WME_AC_VO; break; default: d_wme_ac = WME_AC_BE; break; } - ----------------------------------------------------- When I use SOCK_DGRAM socket instead of raw, everything works fine. I use FreeBSD 6.0-STABLE and my wireless NIC uses an atheros chipset. Has anyone got an idea what is going on ? regards, Geir Egeland -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD6zZAAsOHgqjtXwERAqO6AKDVrEBmrlBvIu5qEx/1WSsYryQTGQCgidwv 6U4vVby9nDjEabmtsPzZoeE= =r/wF -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 14:13:14 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4574916A420 for ; Thu, 9 Feb 2006 14:13:14 +0000 (GMT) (envelope-from bv@bilver.wjv.com) Received: from wjv.com (fl-65-40-24-38.sta.sprint-hsd.net [65.40.24.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4E0E43D45 for ; Thu, 9 Feb 2006 14:13:11 +0000 (GMT) (envelope-from bv@bilver.wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by wjv.com (8.13.5/8.13.1) with ESMTP id k19ED8Ne017975; Thu, 9 Feb 2006 09:13:09 -0500 (EST) (envelope-from bv@bilver.wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.13.5/8.13.1/Submit) id k19ECxDR017967; Thu, 9 Feb 2006 09:12:59 -0500 (EST) (envelope-from bv) Date: Thu, 9 Feb 2006 09:12:59 -0500 From: Bill Vermillion To: Vitaliy Ovsyannikov Message-ID: <20060209141259.GA17914@wjv.com> References: <176144277.20060209173923@kr.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <176144277.20060209173923@kr.ru> Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on bilver.wjv.com Cc: freebsd-net@freebsd.org Subject: Re: incomplete+permanent arp entries X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bv@wjv.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 14:13:14 -0000 At Thu, Feb 09, 2006 at 17:39 , our malformed and occasionally flatulent friend Vitaliy Ovsyannikov spewed forth this fount of brain juice: > Hello, freebsd-net. > > I've noticed what it is possible to make incomplete+permanent > entries in kernel arp table. > xx# arp -s 1.2.3.4 pub > arp: invalid Ethernet address 'pub' And 1.2.3.4 is not valid syntax. xx:xx:xx:xx:xx:xx - the MAC address - should preceded and add the IP address with a hostname into /etc/hosts You probably just overlooked that line in the man page. > # arp 1.2.3.4 > 1.2.3.4 (1.2.3.4) at (incomplete) on vlan1 permanent [vlan] > > In result of it's permanent entry, kernel doesn't listening for > arp-replies for this host. > > My question is simple: is it bug or feature? You left out 'user error'. Bill -- Bill Vermillion - bv @ wjv . com From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 15:58:55 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B72816A422; Thu, 9 Feb 2006 15:58:55 +0000 (GMT) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8076943D48; Thu, 9 Feb 2006 15:58:54 +0000 (GMT) (envelope-from sam@errno.com) Received: from [10.0.0.248] (trouble.errno.com [10.0.0.248]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.6) with ESMTP id k19Fwqo7069547 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2006 07:58:53 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <43EB6764.7040900@errno.com> Date: Thu, 09 Feb 2006 08:01:40 -0800 From: Sam Leffler User-Agent: Thunderbird 1.5 (X11/20060117) MIME-Version: 1.0 To: Geir Egeland References: <43EB3640.8090906@gmail.com> In-Reply-To: <43EB3640.8090906@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-questions@freebsd.org Subject: Re: IEEE 802.11 Wireless Multimedia Extension (WME) and raw sockets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 15:58:55 -0000 Geir Egeland wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > I've been playing around with WME to test various network performance, > and come across a problem that I can't quite understand. > I have an application that generates traffic with various TOS > (BACKGROUND, BEST EFFORT, VOICE, VIDEO). It uses raw sockets to transmit > the IP packets. This all works well if ip->ip_len is less than 192 > bytes. If ip_>ip_len is larger than 192, the call to ieee80211_classify > (/usr/src/sys/net80211/ieee80211_output.c) will classify the packet as > "BEST EFFORT" no matter what value my application set the TOS field as. > > Debugging ieee80211_classify, I see that both ip->ip_tos and ip->ip_len > are set to zero when a I send a packet with ip->ip_len larger than 192 > bytes. > Sniffing the network, I can see my packets have the correct TOS and > length, but they don't get the correct WME classification. > > > - -------------ieee80211_output.c(iee80211_classify)------------ > if (eh->ether_type == htons(ETHERTYPE_IP)) { > const struct ip *ip = (struct ip *) > (mtod(m, u_int8_t *) + sizeof (*eh)); > /* > * IP frame, map the TOS field. > */ > //added by myself > printf("IP_TOS: %d, IP_LEN: %d\n",ip->ip_tos,ntohl(ip->ip_len)); > //end > switch (ip->ip_tos) { > case 0x08: > case 0x20: > d_wme_ac = WME_AC_BK; /* background */ > break; > case 0x28: > case 0xa0: > d_wme_ac = WME_AC_VI; /* video */ > break; > case 0x30: /* voice */ > case 0xe0: > case 0x88: /* XXX UPSD */ > case 0xb8: > d_wme_ac = WME_AC_VO; > break; > default: > d_wme_ac = WME_AC_BE; > break; > } > > - ----------------------------------------------------- > > When I use SOCK_DGRAM socket instead of raw, everything works fine. > > I use FreeBSD 6.0-STABLE and my wireless NIC uses an atheros chipset. > > Has anyone got an idea what is going on ? I'll check but the raw socket path must not have the ip header in the expected spot in the mbuf. Most of my testing has been done with a modified version of netperf that slaps a TOS on the socket based on a command line argument so only UDP and TCP (not raw) traffic. Ideally the 802.11 layer should not be doing classification; packets should be tagged and the 802.11 layer then does the mapping according to the standard. Groveling around inside packets to extract stuff like this is evil. Sam From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 16:03:16 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC1E616A420; Thu, 9 Feb 2006 16:03:16 +0000 (GMT) (envelope-from awebster@connectalk.com) Received: from esafe.connectalk.com (esafe.connectalk.com [204.19.165.17]) by mx1.FreeBSD.org (Postfix) with SMTP id 1332643D53; Thu, 9 Feb 2006 16:03:15 +0000 (GMT) (envelope-from awebster@connectalk.com) Received: from connectalk.com ([10.125.204.14]) by eSafe SMTP Relay 1139494893; Thu Feb 9 10:59:24 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5 Date: Thu, 9 Feb 2006 11:03:11 -0500 Message-ID: <9D61D69E3C1F7F459C5513AD830EE2192BC2AA@mtlex01.connectalk.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Multiple routes to same destination Thread-Index: AcYs9iqbWGP4g1v5TrqiF8LiGR8OIAAmyYgQ From: "Webster, Andrew" To: "Julian Elischer" ,"Qing Li" X-ESAFE-STATUS: Mail clean X-ESAFE-DETAILS: Clean Cc: freebsd-net@freebsd.org, freebsd-questions@freebsd.org Subject: RE: Multiple routes to same destination X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 16:03:17 -0000 >=20 > Qing Li wrote: >=20 > I use mpd to greate one VPN between the sites, using Multilink PPP, so > that > data is sent across both links (eitehr round-robon or split packet). > I use MPD's udp transport mode to open two UDP sockets > and send packets from R1 to R4 and from R2 to R3 (in the diagram below). > MPD will automatically detect if on e link is down and redirect > everything through the remaining link. Sounds like a good idea, but would that not cause the MTU to get smaller due to the overhead of a MPPP link? =20 Windoze hosts have a horrible time with MTU detection! >=20 > > I have a private patch that's based on radix_mpath for FreeBSD 5.4. > > I believe andre@freebsd.org is working on a solution. > > > > -- Qing > > > > > >-----Original Message----- > >From: owner-freebsd-questions@freebsd.org [mailto:owner-freebsd- > questions@freebsd.org] On Behalf Of Webster, Andrew > >Sent: Wednesday, February 08, 2006 10:51 AM > >To: danial_thom@yahoo.com; Ian Lord; freebsd-questions@freebsd.org > >Subject: RE: Multiple routes to same destination? > > > >Well, in that case, an ISP wouldn't want to use FreeBSD in their core > routers :( :( > > > >In this particular case, I have redundant links (L1 and L2) between two > locations > >(Loc 1 and Loc 2) with two FreeBSD routers at each location (R1/R2, and > R3/R4) > >which are running OSPF to redistribute routing information between > locations. > >Since FreeBSD limits the entries for a particular network to only one > active entry, > >the all the traffic for would either go on R1->L1->R4 or R2->L2->R3, but > not both. > > > >Loc 1___ /---R1--L1--R4---\___ Loc 2 > > \---R2--L2--R3---/ > > > > > >Andrew > > > >_______________________________________________ > >freebsd-net@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-net > >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions- > unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 16:26:07 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4716F16A420 for ; Thu, 9 Feb 2006 16:26:07 +0000 (GMT) (envelope-from lists@wm-access.no) Received: from lakepoint.domeneshop.no (lakepoint.domeneshop.no [194.63.248.54]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3B4143D45 for ; Thu, 9 Feb 2006 16:26:05 +0000 (GMT) (envelope-from lists@wm-access.no) Received: from [192.168.9.8] (gw1.arcticwireless.no [80.203.184.14]) (authenticated bits=0) by lakepoint.domeneshop.no (8.13.4/8.13.4) with ESMTP id k19GQ4TN020639 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 9 Feb 2006 17:26:04 +0100 Message-ID: <43EB6D14.2070107@wm-access.no> Date: Thu, 09 Feb 2006 17:25:56 +0100 From: =?ISO-8859-1?Q?Sten_Daniel_S=F8rsdal?= User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Vitaliy Ovsyannikov References: <176144277.20060209173923@kr.ru> In-Reply-To: <176144277.20060209173923@kr.ru> X-Enigmail-Version: 0.94.0.0 OpenPGP: id=D6F56A9B Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig493086D4A8DE570B46823556" Cc: freebsd-net@freebsd.org Subject: Re: incomplete+permanent arp entries X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 16:26:07 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig493086D4A8DE570B46823556 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Vitaliy Ovsyannikov wrote: > Hello, freebsd-net. >=20 > I've noticed what it is possible to make incomplete+permanent > entries in kernel arp table. >=20 > xx# arp -s 1.2.3.4 pub > arp: invalid Ethernet address 'pub' >=20 > # arp 1.2.3.4 > 1.2.3.4 (1.2.3.4) at (incomplete) on vlan1 permanent [vlan] >=20 > In result of it's permanent entry, kernel doesn't listening for > arp-replies for this host. >=20 > My question is simple: is it bug or feature? >=20 I think i understand why you ask though, the answer could easily go both = ways. But my conclussion was that, to me, the man page and the error message=20 indicates it is a bug. By that i mean that arp reports the ethernet=20 address to be invalid but it still performs the task as if the ethernet=20 address was valid and no synopsis written for this case. I'd go with bug = and say that the same application can be done with 'route -blackhole=20 -host 1.2.3.4 127.0.0.1' or something similar. If it turns out to be the right thing to do then perhaps you would be so = kind as to sendbug(1) it? --=20 Sten Daniel S=F8rsdal --------------enig493086D4A8DE570B46823556 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD620YMvOF8Nb1apsRAlmIAJ0dke4nwT64Hi8MzGIODJxHE/yJiwCfcu1C vlnsFG9rh1XL2hq0g7JBgvk= =JK+P -----END PGP SIGNATURE----- --------------enig493086D4A8DE570B46823556-- From owner-freebsd-net@FreeBSD.ORG Thu Feb 9 19:32:34 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 86BE216A420; Thu, 9 Feb 2006 19:32:34 +0000 (GMT) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 43CFC43D45; Thu, 9 Feb 2006 19:32:34 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [10.251.23.146]) ([10.251.23.146]) by a50.ironport.com with ESMTP; 09 Feb 2006 11:32:33 -0800 Message-ID: <43EB98D1.30702@elischer.org> Date: Thu, 09 Feb 2006 11:32:33 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.11) Gecko/20050727 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Webster, Andrew" References: <9D61D69E3C1F7F459C5513AD830EE2192BC2AA@mtlex01.connectalk.com> In-Reply-To: <9D61D69E3C1F7F459C5513AD830EE2192BC2AA@mtlex01.connectalk.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Qing Li , freebsd-questions@freebsd.org, freebsd-net@freebsd.org Subject: Re: Multiple routes to same destination X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2006 19:32:34 -0000 Webster, Andrew wrote: >>Qing Li wrote: >> >>I use mpd to greate one VPN between the sites, using Multilink PPP, so >>that >>data is sent across both links (eitehr round-robon or split packet). >>I use MPD's udp transport mode to open two UDP sockets >>and send packets from R1 to R4 and from R2 to R3 (in the diagram >> >> >below). > > >>MPD will automatically detect if on e link is down and redirect >>everything through the remaining link. >> >> > >Sounds like a good idea, but would that not cause the MTU to get smaller >due to the overhead of a MPPP link? >Windoze hosts have a horrible time with MTU detection! > > I think you can now do mtu munging in teh newer mpd if not you could use the daemon to do it from ports. > > >>> I have a private patch that's based on radix_mpath for FreeBSD >>> >>> >5.4. > > >>> I believe andre@freebsd.org is working on a solution. >>> >>> -- Qing >>> >>> >>>-----Original Message----- >>>From: owner-freebsd-questions@freebsd.org [mailto:owner-freebsd- >>> >>> >>questions@freebsd.org] On Behalf Of Webster, Andrew >> >> >>>Sent: Wednesday, February 08, 2006 10:51 AM >>>To: danial_thom@yahoo.com; Ian Lord; freebsd-questions@freebsd.org >>>Subject: RE: Multiple routes to same destination? >>> >>>Well, in that case, an ISP wouldn't want to use FreeBSD in their core >>> >>> >>routers :( :( >> >> >>>In this particular case, I have redundant links (L1 and L2) between >>> >>> >two > > >>locations >> >> >>>(Loc 1 and Loc 2) with two FreeBSD routers at each location (R1/R2, >>> >>> >and > > >>R3/R4) >> >> >>>which are running OSPF to redistribute routing information between >>> >>> >>locations. >> >> >>>Since FreeBSD limits the entries for a particular network to only one >>> >>> >>active entry, >> >> >>>the all the traffic for would either go on R1->L1->R4 or R2->L2->R3, >>> >>> >but > > >>not both. >> >> >>>Loc 1___ /---R1--L1--R4---\___ Loc 2 >>> \---R2--L2--R3---/ >>> >>> >>>Andrew >>> >>>_______________________________________________ >>>freebsd-net@freebsd.org mailing list >>>http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>To unsubscribe, send any mail to >>> >>> >"freebsd-net-unsubscribe@freebsd.org" > > >>> >>> >>_______________________________________________ >>freebsd-questions@freebsd.org mailing list >>http://lists.freebsd.org/mailman/listinfo/freebsd-questions >>To unsubscribe, send any mail to "freebsd-questions- >>unsubscribe@freebsd.org" >> >> > >_______________________________________________ >freebsd-net@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 03:53:33 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3A1D16A420 for ; Fri, 10 Feb 2006 03:53:33 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from smtp.openaccess.org (smtp.openaccess.org [66.165.52.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57F2743D45 for ; Fri, 10 Feb 2006 03:53:33 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from [216.57.214.90] (unknown [216.57.214.90]) by smtp.openaccess.org (Postfix) with ESMTP id 178D46D4473; Thu, 9 Feb 2006 19:53:22 -0800 (PST) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v746.2) X-Priority: 3 Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <78B98AB7-53DF-4825-B861-BBB8890E918F@staff.openaccess.org> Content-Transfer-Encoding: 7bit From: Michael DeMan Date: Thu, 9 Feb 2006 19:53:20 -0800 To: Shawn Saunders X-Mailer: Apple Mail (2.746.2) Cc: freebsd-net@freebsd.org Subject: Re: Trying to make a Host into a gigabit hub for testing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 03:53:33 -0000 Buy a cheap managed switch and set one port up as a monitoring port and dump all your IDS traffic there? Michael F. DeMan Director of Technology OpenAccess Network Services Bellingham, WA 98225 michael@staff.openaccess.org 360-647-0785 On Feb 7, 2006, at 7:38 AM, Shawn Saunders wrote: > Hello, > > Based on the below configuration, does anyone have an idea of what > I might be doing wrong? > > The following is a layout of the type of configuration, and I have > tried > one2many and hub, but was unable to obtain the desired results. > Whenever I > try to set multiple hooks to the same interface it fails, and > trying to > bring a group of interfaces to a virtual interface, and then take that > virtual interface out multiple different interfaces, seems to fail > (I can't > even make the connection to the virtual interface to go to multiple > outs.) > > I will have 11 interfaces inbound from 11 different networks. > I will have 4 outbound to 4 different IDS's and sensors. > I would like to have 11 interfaces come into a single virtual > interface. > This way I could run tcpdump and/or snort on the box itself and > obtain quick > info, and do validation of packet data, between the capture system > and the > IDS and sensors. > I would like to redirect all the traffic from that one virtual > interface to > a group of 4 physical interfaces so the sensors can work on it. > > Here is a configuration I tried. Any comments or suggestions are > appreciated. > This initial script, only takes 3 input interfaces into a single > virtual > interface. This works. > Then when I try to take that virtual interface and echo it out > multiple > interfaces, it fails. > > #!/bin/sh > # Initialize and bring up all interfaces > for i in 0 1 2 3 4 5 6 7 8 9 10 11 > do /sbin/ifconfig em$i up > done > for g in 0 1 > do /sbin/ifconfig bge$g up > done > /sbin/ifconfig fxp0 up > # Load needed kernel modules > /sbin/kldload /boot/kernel/ng_ether.ko > /sbin/kldload /boot/kernel/ng_one2many.ko > /sbin/kldload /boot/kernel/ng_fec.ko > # Create Virtual Interface > /usr/sbin/ngctl mkpeer fec dummy fec > # Bind physical input interfaces to virtual interface > /usr/sbin/ngctl msg fec0: add_iface '"em0"' > /usr/sbin/ngctl msg fec0: add_iface '"em1"' > /usr/sbin/ngctl msg fec0: add_iface '"em2"' > # Set forwarding mode to mac address layer. > /usr/sbin/ngctl msg fec0: set_mode_mac > # Configure the virtual interface to deliver packets out the others > ngctl mkpeer fec0: one2many upper one > ngctl name fec0:upper secur > ngctl connect bge0: secur: upper many0 > ngctl connect bge1: secur: upper many1 > ngctl msg secur: setconfig "{ xmitAlg=2 failAlg=1 enabledLinks=[ 1 > 1 ] }" > # Set all interfaces Promisc mode and turn off autosrc routing > for s in 0 1 2 3 4 5 6 7 8 9 10 11 > do /usr/sbin/ngctl msg em$s: setpromisc 1 > /usr/sbin/ngctl msg em$s: setautosrc 0 > done > for t in 0 1 > do /usr/sbin/ngctl msg bge$t: setpromisc 1 > /usr/sbin/ngctl msg bge$t: setautosrc 0 > done > #EOF > > Before sending 3 pings accross em2: > gigihub# netstat -I bge0 ; netstat -I bge1 ; netstat -I fec0 ; > netstat -I > em2 > > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > bge0 1500 00:e0:81:32:f4:52 0 > 0 5 > 0 0 > bge0 1500 fe80:b::2e0:8 fe80:b::2e0:81ff: 0 - > 4 - > - > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > bge1 1500 00:e0:81:32:f4:53 0 > 0 0 > 0 0 > bge1 1500 fe80:c::2e0:8 fe80:c::2e0:81ff: 0 - > 4 - > - > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > fec0* 1500 00:04:23:c1:0e:50 11 > 0 0 > 0 0 > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > em2 1500 00:04:23:c1:0e:50 41 > 0 5 0 > 0 > em2 1500 fe80:5::204:2 fe80:5::204:23ff: 0 - > 4 - > - > > After sending 3 pings accross em2: > gigihub# netstat -I bge0 ; netstat -I bge1 ; netstat -I fec0 ; > netstat -I > em2 > Name Mtu Network Address Ipkts Ierrs Opkts > Oerrs > Coll > bge0 1500 00:e0:81:32:f4:52 0 > 0 5 0 > 0 > bge0 1500 fe80:b::2e0:8 fe80:b::2e0:81ff: 0 - > 4 - > - > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > bge1 1500 00:e0:81:32:f4:53 0 > 0 0 > 0 0 > bge1 1500 fe80:c::2e0:8 fe80:c::2e0:81ff: 0 - > 4 - > - > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > fec0* 1500 00:04:23:c1:0e:50 15 > 0 0 0 > 0 > Name Mtu Network Address Ipkts Ierrs > Opkts Oerrs > Coll > em2 1500 00:04:23:c1:0e:50 45 0 > 5 0 > 0 > em2 1500 fe80:5::204:2 fe80:5::204:23ff: 0 - > 4 - > - > > Shawn Saunders _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 11:29:05 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E75916A420 for ; Fri, 10 Feb 2006 11:29:05 +0000 (GMT) (envelope-from rebehn@ant.uni-bremen.de) Received: from antsrv1.ant.uni-bremen.de (antsrv1.ant.uni-bremen.de [134.102.176.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id E3BA343D46 for ; Fri, 10 Feb 2006 11:29:04 +0000 (GMT) (envelope-from rebehn@ant.uni-bremen.de) Received: from bremerhaven.ant.uni-bremen.de ([134.102.176.10]) by antsrv1.ant.uni-bremen.de with esmtp (Exim 4.60 (FreeBSD)) (envelope-from ) id 1F7WSR-0000ZG-SU; Fri, 10 Feb 2006 12:28:59 +0100 Message-ID: <43EC78FB.3020709@ant.uni-bremen.de> Date: Fri, 10 Feb 2006 12:28:59 +0100 From: Heinrich Rebehn User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.1) Gecko/20060130 SeaMonkey/1.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Startup problems with openldap and nss_ldap X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 11:29:05 -0000 Hi list, Since my last protupgrade i am having severe startup problems. /usr/local/etc/rc.d/slapd.sh had already taken quite some time in the past, but now it has become even worse. I interrupted with ^C after a few minutes, and when i tried to login as root on the console, i had to wait again for some minutes. There seems to be a chicken/egg pproblem here: slapd is by default started with "-u ldap -g ldap", but this requires a ruuning slapd to resolve "ldap", although it is in the local files. What also concerns me, is that even root login is hindered by slapd not running, although the root password is, of course, stored locally. I found out that i can remedy this situation by starting slapd simply with "/usr/local/exec/slapd", i.e. as root, but i think the default startup as "ldap" is there for a reason. Is there any clean solution for this? My configuration: root@antsrv1 [~] # uname -r 5.4-RELEASE-p8 root@antsrv1 [~] # pkg_info -Ix ldap nss_ldap-1.244 RFC 2307 NSS module openldap-client-2.2.30 Open source LDAP client implementation openldap-server-2.2.30 Open source LDAP server implementation pam_ldap-1.8.0 A pam module for authenticating with LDAP root@antsrv1 [~] # root@antsrv1 [~] # cat /etc/nsswitch.conf group: files[success=return] ldap #group_compat: nis hosts: files dns networks: files passwd: files[success=return] ldap #passwd_compat: nis shells: files Thanks for any help, Heinrich Rebehn University of Bremen Physics / Electrical and Electronics Engineering - Department of Telecommunications - Phone : +49/421/218-4664 Fax : -3341 From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 13:46:04 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C733816A420 for ; Fri, 10 Feb 2006 13:46:04 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 2D96743D5C for ; Fri, 10 Feb 2006 13:46:03 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 16360 invoked by uid 31014); 10 Feb 2006 13:46:02 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Fri, 10 Feb 2006 08:46:02 -0500 Mime-Version: 1.0 (Apple Message framework v623) Content-Transfer-Encoding: 7bit Message-Id: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> Content-Type: text/plain; charset=US-ASCII; format=flowed To: freebsd-net@freebsd.org From: Marcos Bedinelli Date: Fri, 10 Feb 2006 08:46:00 -0500 X-Mailer: Apple Mail (2.623) Subject: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 13:46:04 -0000 Hello all, We have a 2.4GHz Intel Xeon machine running FreeBSD 6.0-RELEASE-p2. Due to heavy network traffic, CPU utilization on that machine is 100%: === mull [~]$top -S last pid: 94989; load averages: 3.69, 4.02, 4.36 up 25+07:21:34 14:51:43 105 processes: 2 running, 46 sleeping, 57 waiting CPU states: 0.0% user, 0.0% nice, 0.3% system, 99.4% interrupt, 0.3% idle Mem: 20M Active, 153M Inact, 84M Wired, 4K Cache, 60M Buf, 237M Free Swap: 999M Total, 999M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 60 root 1 -44 -163 0K 8K WAIT 355.6H 72.17% swi1: net 39 root 1 -68 -187 0K 8K WAIT 52.3H 5.22% irq28: bge0 40 root 1 -68 -187 0K 8K WAIT 28.3H 2.25% irq29: bge1 11 root 1 171 52 0K 8K RUN 166.6H 0.00% idle 63 root 1 -16 0 0K 8K - 121:55 0.00% yarrow 61 root 1 -32 -151 0K 8K WAIT 46:21 0.00% swi4: clock sio [...] === Does anyone know whether a dual CPU system can help us improve the situation? I was wondering if the software interrupt threads would be divided between the two processors. Any help/insight is greatly appreciated Thanks! -- Marcos Bedinelli From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 13:56:44 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E57216A420 for ; Fri, 10 Feb 2006 13:56:44 +0000 (GMT) (envelope-from piston@otel.net) Received: from mail.otel.net (gw3.OTEL.net [212.36.8.151]) by mx1.FreeBSD.org (Postfix) with ESMTP id D9EAA43D66 for ; Fri, 10 Feb 2006 13:56:43 +0000 (GMT) (envelope-from piston@otel.net) Received: from devilspot.otel.net ([212.36.8.194]) by mail.otel.net with smtp (Exim 4.60 (FreeBSD)) (envelope-from ) id 1F7YlN-000IhT-CD for freebsd-net@freebsd.org; Fri, 10 Feb 2006 15:56:41 +0200 Date: Fri, 10 Feb 2006 15:57:25 +0200 From: "S.I" To: freebsd-net@freebsd.org Message-Id: <20060210155725.3d3c9f13.piston@otel.net> In-Reply-To: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> Organization: OTEL.net X-Mailer: Sylpheed version 2.0.4 (GTK+ 2.8.8; i386-portbld-freebsd5.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 13:56:44 -0000 You must migrate to AMD Opteron. INTEL very very suxX. On Fri, 10 Feb 2006 08:46:00 -0500 Marcos Bedinelli wrote: > Hello all, > > We have a 2.4GHz Intel Xeon machine running FreeBSD 6.0-RELEASE-p2. Due > to heavy network traffic, CPU utilization on that machine is 100%: > > === > > mull [~]$top -S > last pid: 94989; load averages: 3.69, 4.02, 4.36 up > 25+07:21:34 14:51:43 > 105 processes: 2 running, 46 sleeping, 57 waiting > CPU states: 0.0% user, 0.0% nice, 0.3% system, 99.4% interrupt, > 0.3% idle > Mem: 20M Active, 153M Inact, 84M Wired, 4K Cache, 60M Buf, 237M Free > Swap: 999M Total, 999M Free > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND > 60 root 1 -44 -163 0K 8K WAIT 355.6H 72.17% swi1: > net > 39 root 1 -68 -187 0K 8K WAIT 52.3H 5.22% irq28: > bge0 > 40 root 1 -68 -187 0K 8K WAIT 28.3H 2.25% irq29: > bge1 > 11 root 1 171 52 0K 8K RUN 166.6H 0.00% idle > 63 root 1 -16 0 0K 8K - 121:55 0.00% yarrow > 61 root 1 -32 -151 0K 8K WAIT 46:21 0.00% swi4: > clock sio > [...] > > === > > > Does anyone know whether a dual CPU system can help us improve the > situation? I was wondering if the software interrupt threads would be > divided between the two processors. > > Any help/insight is greatly appreciated > > Thanks! > > -- > Marcos Bedinelli > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 14:16:34 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 00FD216A420 for ; Fri, 10 Feb 2006 14:16:34 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 5B0F743D49 for ; Fri, 10 Feb 2006 14:16:33 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 18215 invoked by uid 31014); 10 Feb 2006 14:16:32 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Fri, 10 Feb 2006 09:16:32 -0500 Mime-Version: 1.0 (Apple Message framework v623) Content-Transfer-Encoding: 7bit Message-Id: <814d0f891d2f3b170ae99c1e4b9caf0d@madhaus.cns.utoronto.ca> Content-Type: text/plain; charset=US-ASCII; format=flowed To: freebsd-net@freebsd.org From: Marcos Bedinelli Date: Fri, 10 Feb 2006 09:16:31 -0500 X-Mailer: Apple Mail (2.623) Subject: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 14:16:34 -0000 Hi S.I, I should've mentioned before that we are trying to save some money here, therefore the idea is to add a second 2.4GHz Intel Xeon CPU to our current box. However, if there is consensus that a second processor will buy us nothing, we'll need to acquire a new system and will consider your suggestion. Thanks, -- Marcos Bedinelli UNIVERSITY OF TORONTO e-mail:bedinelli@madhaus.cns.utoronto.ca Network Develop. Group - CNS voice : +1 416 978 7063 4 BANCROFT AV - RV 101C fax : +1 416 978 6620 TORONTO ON M5S 1C1 - CANADA * 1906 - 2006 ONE HUNDRED YEARS OF AVIATION * * http://individual.utoronto.ca/firstflight/ * From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 14:28:24 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81E4016A420 for ; Fri, 10 Feb 2006 14:28:24 +0000 (GMT) (envelope-from regnauld@starbsd.org) Received: from flow.starbsd.org (flow.catpipe.net [195.249.214.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 30C4E43D66 for ; Fri, 10 Feb 2006 14:28:15 +0000 (GMT) (envelope-from regnauld@starbsd.org) Received: by flow.starbsd.org (Postfix, from userid 1001) id CE1C717005; Fri, 10 Feb 2006 15:28:10 +0100 (CET) Date: Fri, 10 Feb 2006 15:28:10 +0100 From: Phil Regnauld To: Marcos Bedinelli Message-ID: <20060210142810.GK1251@flow.eu.org> References: <814d0f891d2f3b170ae99c1e4b9caf0d@madhaus.cns.utoronto.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <814d0f891d2f3b170ae99c1e4b9caf0d@madhaus.cns.utoronto.ca> X-Operating-System: FreeBSD 6.0-STABLE i386 Organization: catpipe Systems ApS User-Agent: Mutt/1.5.9i Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 14:28:24 -0000 Marcos Bedinelli (bedinelli) writes: > I should've mentioned before that we are trying to save some money > here, therefore the idea is to add a second 2.4GHz Intel Xeon CPU to > our current box. > > However, if there is consensus that a second processor will buy us > nothing, we'll need to acquire a new system and will consider your > suggestion. The problem is a combination of how your system is configured, and what you network load is. If you have very high traffic, then it might help to have different netcards on different PCI busses to start with. Also, run with polling mode, that might help. With regards to using a second CPU, it _should_ help with 6.0, but that really depends on the workload. AMD processors (hypertransport in fact) can make a big different because of memory bandwidth if you are doing a lot of copying between buffers. From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 15:31:56 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C47B16A420 for ; Fri, 10 Feb 2006 15:31:56 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id D85A443D70 for ; Fri, 10 Feb 2006 15:31:45 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 674F95CA3; Fri, 10 Feb 2006 10:31:45 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 14957-09; Fri, 10 Feb 2006 10:31:44 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-209-142.ny325.east.verizon.net [68.160.209.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 8B4625C50; Fri, 10 Feb 2006 10:31:44 -0500 (EST) Message-ID: <43ECB1E7.8010308@mac.com> Date: Fri, 10 Feb 2006 10:31:51 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Marcos Bedinelli References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> In-Reply-To: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 15:31:56 -0000 Marcos Bedinelli wrote: [ ... ] > Does anyone know whether a dual CPU system can help us improve the > situation? I was wondering if the software interrupt threads would be > divided between the two processors. > > Any help/insight is greatly appreciated Adding SMP into the mix makes thing more complicated, and you should consider other alternatives first before spending money on hardware that might not help. vmstat -i output would be useful to know, along with what your machine is doing-- is it routing traffic, running a firewall (if so, which), etc. You might take a look at "man polling" and see whether enabling that improves your situation. -- -Chuck From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 17:23:12 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9DAC116A420 for ; Fri, 10 Feb 2006 17:23:12 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 136B243D48 for ; Fri, 10 Feb 2006 17:23:11 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 30386 invoked by uid 31014); 10 Feb 2006 17:23:11 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Fri, 10 Feb 2006 12:23:11 -0500 In-Reply-To: <43ECB1E7.8010308@mac.com> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org From: Marcos Bedinelli Date: Fri, 10 Feb 2006 12:23:09 -0500 X-Mailer: Apple Mail (2.623) Subject: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 17:23:12 -0000 Hello all, thanks for the replies. Most of you have suggested that I turn on polling and give it a try. The machine is in production, hence I need to schedule downtime for that. The system is mainly being used as a dedicated router. It runs OSPF, BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K packets/s (inbound and outbound combined), each one of them. Some of you have asked for the following information: - As I indicated before, polling is currently disabled. - Hyperthreading (HTT) is disabled. mull [~]$vmstat -i interrupt total rate irq1: atkbd0 3466 0 irq6: fdc0 10 0 irq13: npx0 1 0 irq14: ata0 47 0 irq21: fxp1 20462527 8 irq28: bge0 3511765157 1444 irq29: bge1 3633124373 1494 irq30: aac0 1842472 0 cpu0: timer 566751007 233 Total 7733949060 3181 mull [~]$netstat -m 644/646/1290 mbufs in use (current/cache/total) 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) 0/5/4528 sfbufs in use (current/peak/max) 1447K/975K/2422K bytes allocated to network (current/cache/total) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Thank you, -- Marcos From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 18:06:41 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0B72C16A420 for ; Fri, 10 Feb 2006 18:06:41 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 99F8843D46 for ; Fri, 10 Feb 2006 18:06:40 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id DC0C85CB6; Fri, 10 Feb 2006 13:06:39 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 52580-01; Fri, 10 Feb 2006 13:06:38 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-209-142.ny325.east.verizon.net [68.160.209.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 9B02D5C50; Fri, 10 Feb 2006 13:06:38 -0500 (EST) Message-ID: <43ECD636.3070403@mac.com> Date: Fri, 10 Feb 2006 13:06:46 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Marcos Bedinelli References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> In-Reply-To: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 18:06:41 -0000 Marcos Bedinelli wrote: [ ... ] > mull [~]$vmstat -i > interrupt total rate > irq1: atkbd0 3466 0 > irq6: fdc0 10 0 > irq13: npx0 1 0 > irq14: ata0 47 0 > irq21: fxp1 20462527 8 > irq28: bge0 3511765157 1444 > irq29: bge1 3633124373 1494 > irq30: aac0 1842472 0 > cpu0: timer 566751007 233 > Total 7733949060 3181 Interesting, what do you have HZ ("sysctl kern.clockrate") set to? Does setting it to somewhere around 500, 1000, or 2000 help? You're definitely going to want to increase HZ if you enable polling mode... > mull [~]$netstat -m > 644/646/1290 mbufs in use (current/cache/total) > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > 0/5/4528 sfbufs in use (current/peak/max) > 1447K/975K/2422K bytes allocated to network (current/cache/total) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines You might try increasing the # of kern.ipc.nmbclusters, say by a factor of 2. This may not help much, seems like you're bottlenecking servicing the bge interrupts. I assume you've read "man tuning" and do not have something like WITNESS enabled in your kernel? :-) -- -Chuck From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 19:06:24 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE64D16A420 for ; Fri, 10 Feb 2006 19:06:23 +0000 (GMT) (envelope-from ianchov@gmail.com) Received: from uproxy.gmail.com (uproxy.gmail.com [66.249.92.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2AC9B43D5A for ; Fri, 10 Feb 2006 19:06:22 +0000 (GMT) (envelope-from ianchov@gmail.com) Received: by uproxy.gmail.com with SMTP id k40so71164ugc for ; Fri, 10 Feb 2006 11:06:21 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=WW6Az6KGW/g1kf4gd9it1LMq31r3xRIYWX2L8aW1g1SHF+X+cVGt+LzH/lLm9P+HewKqxT22/3Bq/DzvgLkQxr6bJo9WhhGIIVmz73NI1fGLoYOu4peDiirvAc0I0nSpeNqC5c9gOg/xJ5HX5xtQzZ9xF/WJs6d3/w+PNj/OIVA= Received: by 10.48.235.15 with SMTP id i15mr2985716nfh; Fri, 10 Feb 2006 11:06:21 -0800 (PST) Received: by 10.49.26.12 with HTTP; Fri, 10 Feb 2006 11:06:21 -0800 (PST) Message-ID: <18e02bd30602101106s3781e4a0pd214cbb277181b2a@mail.gmail.com> Date: Fri, 10 Feb 2006 21:06:21 +0200 From: Iantcho Vassilev To: Chuck Swiger In-Reply-To: <43ECD636.3070403@mac.com> MIME-Version: 1.0 References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <43ECD636.3070403@mac.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Marcos Bedinelli , freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 19:06:24 -0000 Can someone make a little bit clear about vmstat -i. What exactly we should look for? On my systems i have also high total column. On 2/10/06, Chuck Swiger wrote: > > Marcos Bedinelli wrote: > [ ... ] > > mull [~]$vmstat -i > > interrupt total rate > > irq1: atkbd0 3466 0 > > irq6: fdc0 10 0 > > irq13: npx0 1 0 > > irq14: ata0 47 0 > > irq21: fxp1 20462527 8 > > irq28: bge0 3511765157 1444 > > irq29: bge1 3633124373 1494 > > irq30: aac0 1842472 0 > > cpu0: timer 566751007 233 > > Total 7733949060 3181 > > Interesting, what do you have HZ ("sysctl kern.clockrate") set to? Does > setting > it to somewhere around 500, 1000, or 2000 help? You're definitely going > to want > to increase HZ if you enable polling mode... > > > mull [~]$netstat -m > > 644/646/1290 mbufs in use (current/cache/total) > > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > > 0/5/4528 sfbufs in use (current/peak/max) > > 1447K/975K/2422K bytes allocated to network (current/cache/total) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > You might try increasing the # of kern.ipc.nmbclusters, say by a factor o= f > 2. > This may not help much, seems like you're bottlenecking servicing the bge > interrupts. > > I assume you've read "man tuning" and do not have something like WITNESS > enabled > in your kernel? :-) > > -- > -Chuck > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 19:54:37 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4338316A420 for ; Fri, 10 Feb 2006 19:54:37 +0000 (GMT) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 06C8143D45 for ; Fri, 10 Feb 2006 19:54:36 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [10.251.23.146]) ([10.251.23.146]) by a50.ironport.com with ESMTP; 10 Feb 2006 11:54:36 -0800 Message-ID: <43ECEF7C.2090101@elischer.org> Date: Fri, 10 Feb 2006 11:54:36 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.11) Gecko/20050727 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Marcos Bedinelli References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> In-Reply-To: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 19:54:37 -0000 Marcos Bedinelli wrote: > Hello all, > > thanks for the replies. Most of you have suggested that I turn on > polling and give it a try. The machine is in production, hence I need > to schedule downtime for that. > > The system is mainly being used as a dedicated router. It runs OSPF, > BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. > The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K > packets/s (inbound and outbound combined), each one of them. I have found that most people can optimise there ipfw rulests considerably. for example: a first rule of: 1 allow ip from any to any in recv {inside interfacfe} 2 allow ip from any to any out xmit {inside interface} will cut your ipfw load by 50% immediatly. (you should only be filterring on one interface usually) use 'skipto' rules to immediatly send incoming and outgoing data to different rules sets. etc. (I you want to privatly send me your ruleset I can probably help you do this) julian > > > Some of you have asked for the following information: > > > - As I indicated before, polling is currently disabled. > > > - Hyperthreading (HTT) is disabled. > > > mull [~]$vmstat -i > interrupt total rate > irq1: atkbd0 3466 0 > irq6: fdc0 10 0 > irq13: npx0 1 0 > irq14: ata0 47 0 > irq21: fxp1 20462527 8 > irq28: bge0 3511765157 1444 > irq29: bge1 3633124373 1494 > irq30: aac0 1842472 0 > cpu0: timer 566751007 233 > Total 7733949060 3181 > > > mull [~]$netstat -m > 644/646/1290 mbufs in use (current/cache/total) > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > 0/5/4528 sfbufs in use (current/peak/max) > 1447K/975K/2422K bytes allocated to network (current/cache/total) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > > > Thank you, > > -- > Marcos > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 19:57:30 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A691616A420 for ; Fri, 10 Feb 2006 19:57:30 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 84B6843D53 for ; Fri, 10 Feb 2006 19:57:29 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 7888 invoked by uid 31014); 10 Feb 2006 19:57:28 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Fri, 10 Feb 2006 14:57:28 -0500 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <63274172a54fb70a88d6cb55b9ae6e23@madhaus.cns.utoronto.ca> Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org From: Marcos Bedinelli Date: Fri, 10 Feb 2006 14:57:26 -0500 X-Mailer: Apple Mail (2.623) Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 19:57:30 -0000 Hi, On 10-Feb-06, at 13:06, Chuck Swiger wrote: > Marcos Bedinelli wrote: > [ ... ] >> mull [~]$vmstat -i >> interrupt total rate >> irq1: atkbd0 3466 0 >> irq6: fdc0 10 0 >> irq13: npx0 1 0 >> irq14: ata0 47 0 >> irq21: fxp1 20462527 8 >> irq28: bge0 3511765157 1444 >> irq29: bge1 3633124373 1494 >> irq30: aac0 1842472 0 >> cpu0: timer 566751007 233 >> Total 7733949060 3181 > > Interesting, what do you have HZ ("sysctl kern.clockrate") set to? > Does setting > it to somewhere around 500, 1000, or 2000 help? You're definitely > going to want > to increase HZ if you enable polling mode... mull [~]$sysctl kern.clockrate kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 } >> mull [~]$netstat -m >> 644/646/1290 mbufs in use (current/cache/total) >> 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) >> 0/5/4528 sfbufs in use (current/peak/max) >> 1447K/975K/2422K bytes allocated to network (current/cache/total) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> 0 calls to protocol drain routines > > You might try increasing the # of kern.ipc.nmbclusters, say by a > factor of 2. > This may not help much, seems like you're bottlenecking servicing the > bge > interrupts. > > I assume you've read "man tuning" and do not have something like > WITNESS enabled > in your kernel? :-) WITNESS is not part of my current KERNCONF file. To be honest, I didn't read the whole "man tuning" document because there are many parts that don't apply to this situation. However, the "CPU, MEMORY, DISK, NETWORK" section is quite clear about the following: "If your system runs out of CPU (idle times are perpetually 0%) then you need to consider upgrading the CPU or moving to an SMP motherboard (multiple CPU's), or perhaps you need to revisit the programs that are causing the load and try to optimize them." That's basically the problem I am experiencing: memory is fine, swap is fine, disk access is fine, CPU utilization is way high... The machine is in production and needs to have its performance improved asap. Consequently, we are fine with the idea of spending some $ with a second processor, provided that someone can tell me whether such matter can be solved using this approach. What we would like to avoid is spending $ with a second CPU that ultimately won't do any good for us. Cheers! -- Marcos From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 20:22:51 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A2AE16A420 for ; Fri, 10 Feb 2006 20:22:51 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 7BF2B43D5C for ; Fri, 10 Feb 2006 20:22:44 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 9401 invoked by uid 31014); 10 Feb 2006 20:22:42 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Fri, 10 Feb 2006 15:22:42 -0500 In-Reply-To: <43ECEF7C.2090101@elischer.org> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <43ECEF7C.2090101@elischer.org> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Marcos Bedinelli Date: Fri, 10 Feb 2006 15:22:41 -0500 To: Julian Elischer X-Mailer: Apple Mail (2.623) Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 20:22:51 -0000 Hi Julian, On 10-Feb-06, at 14:54, Julian Elischer wrote: > I have found that most people can optimise there ipfw rulests > considerably. > > for example: a first rule of: > 1 allow ip from any to any in recv {inside interfacfe} > 2 allow ip from any to any out xmit {inside interface} > will cut your ipfw load by 50% immediatly. > (you should only be filterring on one interface usually) > > use 'skipto' rules to immediatly send incoming and outgoing data to > different rules sets. > > etc. > (I you want to privatly send me your ruleset I can probably help you > do this) > > julian Thank you very much for your input and kind offer. Not long ago I removed the entire ruleset on that machine and the impact was minimal (i.e., CPU utilization was still above 98%). Nevertheless, I am sure my ruleset can benefit from some polishing. I would like to take the liberty of writing to you in the future to exchange some ideas, provided you have no objections. Thanks! -- Marcos From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 20:51:50 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B508016A426 for ; Fri, 10 Feb 2006 20:51:50 +0000 (GMT) (envelope-from regnauld@starbsd.org) Received: from flow.starbsd.org (x0.dk [62.242.165.154]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F6C643D45 for ; Fri, 10 Feb 2006 20:51:49 +0000 (GMT) (envelope-from regnauld@starbsd.org) Received: by flow.starbsd.org (Postfix, from userid 1001) id 1544C17005; Fri, 10 Feb 2006 21:51:49 +0100 (CET) Date: Fri, 10 Feb 2006 21:51:48 +0100 From: Phil Regnauld To: Marcos Bedinelli Message-ID: <20060210205148.GA3557@flow.eu.org> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <63274172a54fb70a88d6cb55b9ae6e23@madhaus.cns.utoronto.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <63274172a54fb70a88d6cb55b9ae6e23@madhaus.cns.utoronto.ca> X-Operating-System: FreeBSD 6.0-STABLE i386 Organization: catpipe Systems ApS User-Agent: Mutt/1.5.9i Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 20:51:52 -0000 Marcos Bedinelli (bedinelli) writes: > > "If your system runs out of CPU (idle times are perpetually 0%) then > you need to consider upgrading the CPU or moving to an SMP motherboard > (multiple CPU's), or perhaps you need to revisit the programs that are > causing the load and try to optimize them." Can't remember if it was in this thread or another, but what does top -S tell you ? Where is the CPU usage going ? Take a look at systat -vmstat 1 as well. From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 20:55:27 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 07D0516A420 for ; Fri, 10 Feb 2006 20:55:27 +0000 (GMT) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA92E43D48 for ; Fri, 10 Feb 2006 20:55:26 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [10.251.23.146]) ([10.251.23.146]) by a50.ironport.com with ESMTP; 10 Feb 2006 12:55:24 -0800 Message-ID: <43ECFDBD.3020606@elischer.org> Date: Fri, 10 Feb 2006 12:55:25 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.11) Gecko/20050727 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Marcos Bedinelli References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <43ECEF7C.2090101@elischer.org> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 20:55:27 -0000 Marcos Bedinelli wrote: > Hi Julian, > > > On 10-Feb-06, at 14:54, Julian Elischer wrote: > >> I have found that most people can optimise there ipfw rulests >> considerably. >> >> for example: a first rule of: >> 1 allow ip from any to any in recv {inside interfacfe} >> 2 allow ip from any to any out xmit {inside interface} >> will cut your ipfw load by 50% immediatly. >> (you should only be filterring on one interface usually) >> >> use 'skipto' rules to immediatly send incoming and outgoing data to >> different rules sets. >> >> etc. >> (I you want to privatly send me your ruleset I can probably help you >> do this) >> >> julian > > > > Thank you very much for your input and kind offer. > > Not long ago I removed the entire ruleset on that machine and the > impact was minimal (i.e., CPU utilization was still above 98%). yes but throughput probably went up ;-) > > > Nevertheless, I am sure my ruleset can benefit from some polishing. I > would like to take the liberty of writing to you in the future to > exchange some ideas, provided you have no objections. whenever you are would like to .. > > Thanks! > > -- > Marcos From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 21:39:45 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1FCDE16A420 for ; Fri, 10 Feb 2006 21:39:45 +0000 (GMT) (envelope-from _pppp@mail.ru) Received: from f16.mail.ru (f16.mail.ru [194.67.57.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC11D43D46 for ; Fri, 10 Feb 2006 21:39:44 +0000 (GMT) (envelope-from _pppp@mail.ru) Received: from mail by f16.mail.ru with local id 1F7fzS-000GUW-00; Sat, 11 Feb 2006 00:39:42 +0300 Received: from [85.140.69.145] by koi.mail.ru with HTTP; Sat, 11 Feb 2006 00:39:42 +0300 From: dima <_pppp@mail.ru> To: Marcos Bedinelli Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [85.140.69.145] Date: Sat, 11 Feb 2006 00:39:42 +0300 In-Reply-To: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dima <_pppp@mail.ru> List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 21:39:45 -0000 > Hello all, > > thanks for the replies. Most of you have suggested that I turn on > polling and give it a try. The machine is in production, hence I need > to schedule downtime for that. > > The system is mainly being used as a dedicated router. It runs OSPF, > BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. > The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K > packets/s (inbound and outbound combined), each one of them. The second CPU wouldn't help you for sure. There's only one [swi1: net] kernel thread which deals with all the kernel traffic. The option of per-CPU [swi: net] threads was discussed on freebsd-arch@ several months ago, but it wouldn't be implemented soon. So, the only hardware option is installing the fastest CPU possible. There are several software (FreeBSD specific) options though: 1. You should surely try polling(4). 50kpps mean 50000 interrupts and the same amount of context switches, which are quite expensive. 2. FastForwarding. It's the most suitable for you. As I know, Quagga inserts its dynamic routes to the system routing table. And FastForwarding is aware of routing table and firewall rules. And the most exciting: you can switch it on/off without reboot: # sysctl net.inet.ip.fastforwarding=1 The only limitation is it applies to IPv4 unicast traffic only. There's no documentation on this feature as i know (am I wrong, or should I report this as a documentation bug?) but you can look at the comments in the beginning of /sys/netinet/ip_fastfwd.c The authors reported up to 1Mpps (see page 10 at http://people.freebsd.org/~andre/FreeBSD-5.3-Networking.pdf) > > > Some of you have asked for the following information: > > > - As I indicated before, polling is currently disabled. > > > - Hyperthreading (HTT) is disabled. > > > mull [~]$vmstat -i > interrupt total rate > irq1: atkbd0 3466 0 > irq6: fdc0 10 0 > irq13: npx0 1 0 > irq14: ata0 47 0 > irq21: fxp1 20462527 8 > irq28: bge0 3511765157 1444 > irq29: bge1 3633124373 1494 > irq30: aac0 1842472 0 > cpu0: timer 566751007 233 > Total 7733949060 3181 > > > mull [~]$netstat -m > 644/646/1290 mbufs in use (current/cache/total) > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > 0/5/4528 sfbufs in use (current/peak/max) > 1447K/975K/2422K bytes allocated to network (current/cache/total) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > > > Thank you, > > -- > Marcos From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 22:19:19 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1496916A420; Fri, 10 Feb 2006 22:19:19 +0000 (GMT) (envelope-from ru@ip.net.ua) Received: from tigra.ip.net.ua (tigra.ip.net.ua [82.193.96.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4759443D45; Fri, 10 Feb 2006 22:19:17 +0000 (GMT) (envelope-from ru@ip.net.ua) Received: from localhost (rocky.ip.net.ua [82.193.96.2]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id k1AMJGte066362; Sat, 11 Feb 2006 00:19:16 +0200 (EET) (envelope-from ru@ip.net.ua) Received: from tigra.ip.net.ua ([82.193.96.10]) by localhost (rocky.ip.net.ua [82.193.96.2]) (amavisd-new, port 10024) with LMTP id 97555-03; Sat, 11 Feb 2006 00:19:11 +0200 (EET) Received: from heffalump.ip.net.ua (heffalump.ip.net.ua [82.193.96.213]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id k1AMEEa7066289 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 11 Feb 2006 00:14:14 +0200 (EET) (envelope-from ru@ip.net.ua) Received: (from ru@localhost) by heffalump.ip.net.ua (8.13.4/8.13.4) id k1AMEFOM041979; Sat, 11 Feb 2006 00:14:15 +0200 (EET) (envelope-from ru) Date: Sat, 11 Feb 2006 00:14:15 +0200 From: Ruslan Ermilov To: Andre Oppermann Message-ID: <20060210221415.GC33588@ip.net.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="St7VIuEGZ6dlpu13" Content-Disposition: inline User-Agent: Mutt/1.5.11 X-Virus-Scanned: amavisd-new at ip.net.ua Cc: net@FreeBSD.org Subject: [FIX] dummynet breaks IP reassembly X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 22:19:19 -0000 --St7VIuEGZ6dlpu13 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Andre, If I'm not mistaken, *you* converted ipfw to use pfil(9). During the conversion, the following bug was introduced. When forwarding fragmented packets through a dummynet pipe (ip_input -> ip_forward -> ip_output -> pipe -> ip_output) the last ip_output() in the chain that does the actual IP delivery sets ip_id of all fragments to different values, making it impossible to reassemble the packet at receive side. Example of forwarding a fragmented IP datagram from em0 to xl0: # tcpdump -nvi em0 net 192.168.2 tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 96 byt= es 00:04:35.170994 IP (tos 0x0, ttl 64, id 2186, offset 0, flags [+], proto: = ICMP (1), length: 1500) 192.168.1.3 > 192.168.2.1: ICMP echo request, id 28= 19, seq 0, length 1480 00:04:35.171242 IP (tos 0x0, ttl 64, id 2186, offset 1480, flags [none], p= roto: ICMP (1), length: 548) 192.168.1.3 > 192.168.2.1: icmp # tcpdump -nvi xl0 net 192.168.2 tcpdump: listening on xl0, link-type EN10MB (Ethernet), capture size 96 byt= es 00:04:35.171028 IP (tos 0x0, ttl 63, id 10560, offset 0, flags [+], proto:= ICMP (1), length: 1500) 192.168.1.3 > 192.168.2.1: ICMP echo request, id 2= 819, seq 0, length 1480 00:04:35.171266 IP (tos 0x0, ttl 63, id 10561, offset 1480, flags [none], = proto: ICMP (1), length: 548) 192.168.1.3 > 192.168.2.1: icmp Note that IDs were rewritten from the original 2186 and worse, they are different. In 4,x, the "flags" field (see the patch below) was used to keep the IP_FORWARDING bit passed to ip_output() by ip_forward(). This bit was kept in the dummynet packet tag (in the "flags" field) and later passed to the second ip_output() call, causing the latter to NOT touch the IP header again. This feature was lost, resulting in a bug. Since dummynet never calls ip_output() with unfilled header, it's safe to always call ip_output() from dummynet with the IP_FORWARDING bit set, to indicate it's forwarded =66rom another ip_output() and so it shouldn't attempt to modify the header. Surprisingly, it "seemed" to work for packets exceeding MTU and originating from the dummynetting host, mainly because the packet wasn't fragmented while in dummynet. Yet, the ip_id field was always incremented in my tests (pipe with no bandwidth limitation), comparing it before and after the dummynet processing. Now, the ip_id is always preserved. # tcpdump -nvi em0 net 192.168.2 tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 96 byt= es 00:12:04.654669 IP (tos 0x0, ttl 64, id 2192, offset 0, flags [+], proto: = ICMP (1), length: 1500) 192.168.1.3 > 192.168.2.1: ICMP echo request, id 79= 39, seq 0, length 1480 00:12:04.654917 IP (tos 0x0, ttl 64, id 2192, offset 1480, flags [none], p= roto: ICMP (1), length: 548) 192.168.1.3 > 192.168.2.1: icmp # tcpdump -nvi xl0 net 192.168.2 tcpdump: listening on xl0, link-type EN10MB (Ethernet), capture size 96 byt= es 00:12:04.654703 IP (tos 0x0, ttl 63, id 2192, offset 0, flags [+], proto: = ICMP (1), length: 1500) 192.168.1.3 > 192.168.2.1: ICMP echo request, id 79= 39, seq 0, length 1480 00:12:04.654939 IP (tos 0x0, ttl 63, id 2192, offset 1480, flags [none], p= roto: ICMP (1), length: 548) 192.168.1.3 > 192.168.2.1: icmp (Note the ip_id is preserved when forwarding.) I'm not sure about the IPv6 portion but it's consistent with the normal forwarding path so I believe it's correct. Comments? %%% Index: ip_dummynet.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/netinet/ip_dummynet.c,v retrieving revision 1.98 diff -u -p -r1.98 ip_dummynet.c --- ip_dummynet.c 3 Feb 2006 11:38:19 -0000 1.98 +++ ip_dummynet.c 10 Feb 2006 21:20:59 -0000 @@ -769,7 +769,7 @@ dummynet_send(struct mbuf *m) pkt =3D dn_tag_get(m); switch (pkt->dn_dir) { case DN_TO_IP_OUT: - ip_output(m, NULL, NULL, pkt->flags, NULL, NULL); + ip_output(m, NULL, NULL, IP_FORWARDING, NULL, NULL); break ; case DN_TO_IP_IN : ip =3D mtod(m, struct ip *); @@ -783,7 +783,7 @@ dummynet_send(struct mbuf *m) break; =20 case DN_TO_IP6_OUT: - ip6_output(m, NULL, NULL, pkt->flags, NULL, NULL, NULL); + ip6_output(m, NULL, NULL, IPV6_FORWARDING, NULL, NULL, NULL); break; #endif case DN_TO_IFB_FWD: @@ -1129,7 +1129,6 @@ locate_pipe(int pipe_nr) * ifp the 'ifp' parameter from the caller. * NULL in ip_input, destination interface in ip_output, * rule matching rule, in case of multiple passes - * flags flags from the caller, only used in ip_output * */ static int @@ -1213,8 +1212,6 @@ dummynet_io(struct mbuf *m, int dir, str pkt->dn_dir =3D dir ; =20 pkt->ifp =3D fwa->oif; - if (dir =3D=3D DN_TO_IP_OUT || dir =3D=3D DN_TO_IP6_OUT) - pkt->flags =3D fwa->flags; =20 if (q->head =3D=3D NULL) q->head =3D m; Index: ip_dummynet.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/netinet/ip_dummynet.h,v retrieving revision 1.38 diff -u -p -r1.38 ip_dummynet.h --- ip_dummynet.h 29 Nov 2005 00:11:01 -0000 1.38 +++ ip_dummynet.h 10 Feb 2006 21:13:45 -0000 @@ -130,7 +130,6 @@ struct dn_pkt_tag { =20 dn_key output_time; /* when the pkt is due for delivery */ struct ifnet *ifp; /* interface, for ip_output */ - int flags ; /* flags, for ip_output (IPv6 ?) */ struct _ip6dn_args ip6opt; /* XXX ipv6 options */ }; #endif /* _KERNEL */ Index: ip_fw.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/netinet/ip_fw.h,v retrieving revision 1.103 diff -u -p -r1.103 ip_fw.h --- ip_fw.h 13 Dec 2005 12:16:02 -0000 1.103 +++ ip_fw.h 10 Feb 2006 21:15:41 -0000 @@ -510,8 +510,6 @@ struct ip_fw_args { struct ip_fw *rule; /* matching rule */ struct ether_header *eh; /* for bridged packets */ =20 - int flags; /* for dummynet */ - struct ipfw_flow_id f_id; /* grabbed from IP header */ u_int32_t cookie; /* a cookie depending on rule action */ struct inpcb *inp; %%% Cheers, --=20 Ruslan Ermilov ru@FreeBSD.org FreeBSD committer --St7VIuEGZ6dlpu13 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFD7RA3qRfpzJluFF4RAixfAJ9hPuJK5tiSZsXGuRzCIi/WsH9jrQCcCkAe cgPUE8BgRuyhmE8MvRF3FXc= =+aWq -----END PGP SIGNATURE----- --St7VIuEGZ6dlpu13-- From owner-freebsd-net@FreeBSD.ORG Fri Feb 10 23:01:08 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 291F116A420 for ; Fri, 10 Feb 2006 23:01:08 +0000 (GMT) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7FB4B43D48 for ; Fri, 10 Feb 2006 23:01:07 +0000 (GMT) (envelope-from max@love2party.net) Received: from [84.163.233.94] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu4) with ESMTP (Nemesis), id 0ML21M-1F7hG80B4Z-00070m; Sat, 11 Feb 2006 00:01:01 +0100 From: Max Laier Organization: FreeBSD To: freebsd-net@freebsd.org Date: Sat, 11 Feb 2006 00:02:12 +0100 User-Agent: KMail/1.9.1 References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <43ECEF7C.2090101@elischer.org> In-Reply-To: <43ECEF7C.2090101@elischer.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart142454074.3HhXPCN98c"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200602110002.21275.max@love2party.net> X-Provags-ID: kundenserver.de abuse@kundenserver.de login:61c499deaeeba3ba5be80f48ecc83056 Cc: Marcos Bedinelli , Julian Elischer Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2006 23:01:08 -0000 --nextPart142454074.3HhXPCN98c Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Friday 10 February 2006 20:54, Julian Elischer wrote: > Marcos Bedinelli wrote: > > Hello all, > > > > thanks for the replies. Most of you have suggested that I turn on > > polling and give it a try. The machine is in production, hence I need > > to schedule downtime for that. > > > > The system is mainly being used as a dedicated router. It runs OSPF, > > BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. > > The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K > > packets/s (inbound and outbound combined), each one of them. > > I have found that most people can optimise there ipfw rulests considerabl= y. > > for example: a first rule of: > 1 allow ip from any to any in recv {inside interfacfe} > 2 allow ip from any to any out xmit {inside interface} > will cut your ipfw load by 50% immediatly. > (you should only be filterring on one interface usually) > > use 'skipto' rules to immediatly send incoming and outgoing data to > different rules sets. =46WIW, pf does some of those optimizations automatically called "skip step= s"=20 and "pfctl -o" restructures the ruleset so that often matching rules are=20 moved to the top. I know that this does not map directly to IPFW, but it=20 might still be interesting to have a look at it. > etc. > (I you want to privatly send me your ruleset I can probably help you do > this) > > julian > > > Some of you have asked for the following information: > > > > > > - As I indicated before, polling is currently disabled. > > > > > > - Hyperthreading (HTT) is disabled. > > > > > > mull [~]$vmstat -i > > interrupt total rate > > irq1: atkbd0 3466 0 > > irq6: fdc0 10 0 > > irq13: npx0 1 0 > > irq14: ata0 47 0 > > irq21: fxp1 20462527 8 > > irq28: bge0 3511765157 1444 > > irq29: bge1 3633124373 1494 > > irq30: aac0 1842472 0 > > cpu0: timer 566751007 233 > > Total 7733949060 3181 > > > > > > mull [~]$netstat -m > > 644/646/1290 mbufs in use (current/cache/total) > > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > > 0/5/4528 sfbufs in use (current/peak/max) > > 1447K/975K/2422K bytes allocated to network (current/cache/total) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > > > > > > Thank you, > > > > -- > > Marcos > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart142454074.3HhXPCN98c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBD7Rt9XyyEoT62BG0RAo8/AJ9BPfGdS7cT+ZOdEaXJvHsUh7gNKQCePW68 4d9JPOO7fqXbS9qdzD6SNec= =kNJT -----END PGP SIGNATURE----- --nextPart142454074.3HhXPCN98c-- From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 02:01:40 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B974916A420 for ; Sat, 11 Feb 2006 02:01:40 +0000 (GMT) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F2E743D45 for ; Sat, 11 Feb 2006 02:01:40 +0000 (GMT) (envelope-from mike@sentex.net) Received: from BLUELAPIS.sentex.ca (cage.simianscience.com [64.7.134.1]) by smarthost2.sentex.ca (8.13.4/8.13.4) with SMTP id k1B21Ypm038574; Fri, 10 Feb 2006 21:01:38 -0500 (EST) (envelope-from mike@sentex.net) From: Mike Tancsa To: Marcos Bedinelli Date: Fri, 10 Feb 2006 21:01:45 -0500 Message-ID: <9qgqu19rt594h5k4t73s3dng3alprcd06f@4ax.com> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <43ECB1E7.8010308@mac.com> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <63274172a54fb70a88d6cb55b9ae6e23@madhaus.cns.utoronto.ca> In-Reply-To: <63274172a54fb70a88d6cb55b9ae6e23@madhaus.cns.utoronto.ca> X-Mailer: Forte Agent 1.93/32.576 English (American) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.51 on 205.211.164.50 Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 02:01:40 -0000 On Fri, 10 Feb 2006 14:57:26 -0500, in sentex.lists.freebsd.net you wrote: > >"If your system runs out of CPU (idle times are perpetually 0%) then=20 >you need to consider upgrading the CPU or moving to an SMP motherboard=20 >(multiple CPU's), or perhaps you need to revisit the programs that are=20 >causing the load and try to optimize them." > >That's basically the problem I am experiencing: memory is fine, swap is=20 >fine, disk access is fine, CPU utilization is way high... > >The machine is in production and needs to have its performance improved=20 >asap. Consequently, we are fine with the idea of spending some $ with a=20 >second processor, provided that someone can tell me whether such matter=20 >can be solved using this approach. What we would like to avoid is=20 >spending $ with a second CPU that ultimately won't do any good for us. If the box is just doing routing etc, adding a second CPU will not really help matters and in some cases can make it worse. Your top output indicates the load is all from servicing interrupts. If your box has a PCI-X slot in it, you might try something like a dual port em NIC. They can be bought in the GTA area for around $150 or so and might perform better than the broadcoms you have. If you want to make a minimal upgrade, try swapping out your 2.4Ghz xeon for a faster CPU (provided the MB can handle it). Otherwise, look at a new box that has fast memory throughput. But stay with a single CPU. Experiment with polling, but be careful as you can start to loose packets if the box cannot keep up. There are some interesting networking enhancements coming soon to FreeBSD with the em driver, but it will be a little while I think before it makes it to RELENG_6 ---Mike -------------------------------------------------------- Mike Tancsa, Sentex communications http://www.sentex.net Providing Internet Access since 1994 mike@sentex.net, (http://www.tancsa.com) From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 02:58:14 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A59F416A420 for ; Sat, 11 Feb 2006 02:58:14 +0000 (GMT) (envelope-from frank@ircnow.org) Received: from scott.blazing.de (scott.blazing.de [80.86.187.157]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C64643D48 for ; Sat, 11 Feb 2006 02:58:13 +0000 (GMT) (envelope-from frank@ircnow.org) Received: by scott.blazing.de (Postfix, from userid 510) id 5C1AD95659; Sat, 11 Feb 2006 03:58:12 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on scott.blazing.de X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.0 Received: from shodan.nognu.de (shodan.nognu.de [85.14.216.230]) by scott.blazing.de (Postfix) with ESMTP id A8ACE95646 for ; Sat, 11 Feb 2006 03:58:10 +0100 (CET) Received: by shodan.nognu.de (nbSMTP-1.00) for uid 1002 frank@ircnow.org; Sat, 11 Feb 2006 03:58:10 +0100 (CET) Date: Sat, 11 Feb 2006 03:58:10 +0100 From: Frank Steinborn To: freebsd-net@freebsd.org Message-ID: <20060211025810.GA11128@scott.blazing.de> Mail-Followup-To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-PGP: 0x41F1741D User-Agent: mutt-ng/devel-r581 (FreeBSD) Subject: Can't delete v6 addy from gif0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 02:58:14 -0000 Hello, i'm trying to delete an IPv6-address from my gif-interface (named blazing): shodan:~# ifconfig blazing inet6 delete 2001:1638:17ad::9 ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address I tried 2001:1638:17ad:0:0:0:0:9 too without success. Adding the address (ifconfig blazing inet6 add) worked fine, though. Any hints? Thanks, Frank From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 03:50:27 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E1BA016A422 for ; Sat, 11 Feb 2006 03:50:26 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9622443D46 for ; Sat, 11 Feb 2006 03:50:26 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 7B0F71A3C26; Fri, 10 Feb 2006 19:50:26 -0800 (PST) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id B762A515A3; Fri, 10 Feb 2006 22:50:25 -0500 (EST) Date: Fri, 10 Feb 2006 22:50:25 -0500 From: Kris Kennaway To: "JINMEI Tatuya / ?$B?@L@C#:H" Message-ID: <20060211035025.GA77114@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="huq684BweRXVnRxX" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i Cc: net@FreeBSD.org, Kris Kennaway Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 03:50:27 -0000 --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 07, 2006 at 07:16:09PM +0900, JINMEI Tatuya / ?$B?@L@C#:H wrote: > >>>>> On Tue, 7 Feb 2006 00:45:02 -0500,=20 > >>>>> Kris Kennaway said: >=20 > >> I ran ntpdate on an amd64 system with ipv6 enabled and a skewed clock > >> (ntpdate stepped it back by about an hour), and immediately got a > >> use-after-free panic in ifaddr. When I rebooted with memguard enabled > >> on this malloc type and retried, I got this panic upon changing the > >> date forward, then back, then forward again (also note the garbage > >> return data from ntpdate): >=20 > > Has anyone looked at this? This is on the TODO list for 6.1, so the > > sooner it can be resolved the better. >=20 > Sorry, not really (we've not got a test environment to reproduce it). > But from a quick review of nd6.c, there seems to be one thing that is > obviously wrong. The possible bug has been there since rev. 1.19 > committed in April 2002. We've been probably just lucky so far... >=20 > Could you try the patch attached below? We'll probably also need to > apply this fix to 4.X and 5.X. The patch did not fix the panic. Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0xffffffff919d5198 fault code =3D supervisor write, protection violation instruction pointer =3D 0x8:0xffffffff8031fa76 stack pointer =3D 0x10:0xffffffffbcda4b60 frame pointer =3D 0x10:0xffffffffbcda4b90 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 14 (swi4: clock sio) [thread pid 14 tid 100010 ] Stopped at nd6_timer+0x106: movl %eax,0x198(%rbx) db> wh Tracing pid 14 tid 100010 td 0xffffff03e15d6c30 nd6_timer() at nd6_timer+0x106 softclock() at softclock+0x279 ithread_execute_handlers() at ithread_execute_handlers+0x12f ithread_loop() at ithread_loop+0x99 fork_exit() at fork_exit+0xdf fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffffffbcda4d40, rbp =3D 0 --- db> > (a note about the patch: the first chunk is actually not related to > the bug, but I could not miss the trivial typo) You missed the other one though :-) > - * However, from a stricter speci-confrmance standpoint, we should > + * However, from a stricter spec-confrmance standpoint, we should ^o Kris --huq684BweRXVnRxX Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFD7V8BWry0BWjoQKURAre3AKD3OcY47WraUlT1cO8IWihXA9Px1gCg2686 pc4yt9EYr/UQYfxSczsPkVI= =72Zw -----END PGP SIGNATURE----- --huq684BweRXVnRxX-- From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 06:39:08 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B2A5D16A420 for ; Sat, 11 Feb 2006 06:39:08 +0000 (GMT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from shuttle.wide.toshiba.co.jp (shuttle.wide.toshiba.co.jp [202.249.10.124]) by mx1.FreeBSD.org (Postfix) with ESMTP id 582B643D48 for ; Sat, 11 Feb 2006 06:39:08 +0000 (GMT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from impact.jinmei.org (unknown [2001:200:0:8002:644a:1300:9f15:d623]) by shuttle.wide.toshiba.co.jp (Postfix) with ESMTP id 8D33F1521E; Sat, 11 Feb 2006 15:39:06 +0900 (JST) Date: Sat, 11 Feb 2006 15:38:56 +0900 Message-ID: From: JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: Kris Kennaway In-Reply-To: <20060211035025.GA77114@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> <20060211035025.GA77114@xor.obsecurity.org> User-Agent: Wanderlust/2.14.0 (Africa) Emacs/21.3 Mule/5.0 (SAKAKI) Organization: Research & Development Center, Toshiba Corp., Kawasaki, Japan. MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: net@FreeBSD.org Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 06:39:08 -0000 >>>>> On Fri, 10 Feb 2006 22:50:25 -0500, >>>>> Kris Kennaway said: >> Sorry, not really (we've not got a test environment to reproduce it). >> But from a quick review of nd6.c, there seems to be one thing that is >> obviously wrong. The possible bug has been there since rev. 1.19 >> committed in April 2002. We've been probably just lucky so far... >> >> Could you try the patch attached below? We'll probably also need to >> apply this fix to 4.X and 5.X. > The patch did not fix the panic. Hmm, but this time the point where the panic happened should be different. Can you identify where it was? >> (a note about the patch: the first chunk is actually not related to >> the bug, but I could not miss the trivial typo) > You missed the other one though :-) That's right:-) I was informed of that one off-list. >> - * However, from a stricter speci-confrmance standpoint, we should >> + * However, from a stricter spec-confrmance standpoint, we should > ^o JINMEI, Tatuya Communication Platform Lab. Corporate R&D Center, Toshiba Corp. jinmei@isl.rdc.toshiba.co.jp From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 07:14:21 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F6C716A420 for ; Sat, 11 Feb 2006 07:14:21 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF73F43D6A for ; Sat, 11 Feb 2006 07:14:13 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 555541A3C1B; Fri, 10 Feb 2006 23:14:13 -0800 (PST) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 1EE5953BBA; Sat, 11 Feb 2006 02:14:12 -0500 (EST) Date: Sat, 11 Feb 2006 02:14:11 -0500 From: Kris Kennaway To: "JINMEI Tatuya / ?$B?@L@C#:H" Message-ID: <20060211071411.GA82302@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> <20060211035025.GA77114@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i Cc: net@FreeBSD.org, Kris Kennaway Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 07:14:21 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 11, 2006 at 03:38:56PM +0900, JINMEI Tatuya / ?$B?@L@C#:H wrote: > >>>>> On Fri, 10 Feb 2006 22:50:25 -0500,=20 > >>>>> Kris Kennaway said: >=20 > >> Sorry, not really (we've not got a test environment to reproduce it). > >> But from a quick review of nd6.c, there seems to be one thing that is > >> obviously wrong. The possible bug has been there since rev. 1.19 > >> committed in April 2002. We've been probably just lucky so far... > >>=20 > >> Could you try the patch attached below? We'll probably also need to > >> apply this fix to 4.X and 5.X. >=20 > > The patch did not fix the panic. >=20 > Hmm, but this time the point where the panic happened should be > different. Can you identify where it was? I reduced the hw.physmem size and was able to get a dump: Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0xffffffff80862198 fault code =3D supervisor write, protection violation instruction pointer =3D 0x8:0xffffffff80333a86 stack pointer =3D 0x10:0xffffffffabc31b50 frame pointer =3D 0x10:0xffffffffabc31b80 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 14 (swi4: clock sio) Dumping 3071 MB (2 chunks) chunk 0: 1MB (151 pages) ... ok chunk 1: 3071MB (786176 pages) 3056 3040 3024 3008 2992 2976 2960 2944 29= 28 2912 2896 2880 2864 2848 2832 2816 2800 2784 2768 2752 2736 2720 2704 26= 88 2672 2656 2640 2624 2608 2592 2576 2560 2544 2528 2512 2496 2480 2464 24= 48 2432 2416 2400 2384 2368 2352 2336 2320 2304 2288 2272 2256 2240 2224 22= 08 2192 2176 2160 2144 2128 2112 2096 2080 2064 2048 2032 2016 2000 1984 19= 68 1952 1936 1920 1904 1888 1872 1856 1840 1824 1808 1792 1776 1760 1744 17= 28 1712 1696 1680 1664 1648 1632 1616 1600 1584 1568 1552 1536 1520 1504 14= 88 1472 1456 1440 1424 1408 1392 1376 1360 1344 1328 1312 1296 1280 1264 12= 48 1232 1216 1200 1184 1168 1152 1136 1120 1104 1088 1072 1056 1040 1024 10= 08 992 976 960 944 928 912 896 880 864 848 832 816 800 784 768 752 736 720 = 704 688 672 656 640 624 608 592 576 560 544 528 512 496 480 464 448 432 416= 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 11= 2 96 80 64 48 32 16 #0 doadump () at pcpu.h:172 172 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:172 #1 0xffffffff80181f31 in db_fncall (dummy1=3D0, dummy2=3D0, dummy3=3D0, du= mmy4=3D0x0) at ../../../ddb/db_command.c:489 #2 0xffffffff80181c80 in db_command (last_cmdp=3D0xffffffff805cfea8, cmd_t= able=3D0x0, aux_cmd_tablep=3D0xffffffff8047ebd0, aux_cmd_tablep_end=3D0xffffffff8047ebd8) at ../../../ddb/db_command.c:4= 04 #3 0xffffffff80181da7 in db_command_loop () at ../../../ddb/db_command.c:4= 55 #4 0xffffffff80183feb in db_trap (type=3D-1413277552, code=3D0) at ../../.= ./ddb/db_main.c:221 #5 0xffffffff80280d0c in kdb_trap (type=3D12, code=3D0, tf=3D0xffffffffabc= 31aa0) at ../../../kern/subr_kdb.c:485 #6 0xffffffff803ea0ab in trap_fatal (frame=3D0xffffffffabc31aa0, eva=3D184= 46744071570858392) at ../../../amd64/amd64/trap.c:687 #7 0xffffffff803e9d1c in trap_pfault (frame=3D0xffffffffabc31aa0, usermode= =3D0) at ../../../amd64/amd64/trap.c:609 #8 0xffffffff803e9915 in trap (frame=3D {tf_rdi =3D -2141607072, tf_rsi =3D -1096395428656, tf_rdx =3D 64, tf= _rcx =3D 4, tf_r8 =3D 0, tf_r9 =3D 4, tf_rax =3D 80, tf_rbx =3D -2138693632= , tf_rbp =3D -1413276800, tf_r10 =3D -1096385087968, tf_r11 =3D 14073748829= 6312, tf_r12 =3D 0, tf_r13 =3D -2138689536, tf_r14 =3D 0, tf_r15 =3D -21441= 26592, tf_trapno =3D 12, tf_addr =3D -2138693224, tf_flags =3D -27503810621= 59859712, tf_err =3D 3, tf_rip =3D -2144126330, tf_cs =3D 8, tf_rflags =3D = 66054, tf_rsp =3D -1413276832, tf_ss =3D 16}) at ../../../amd64/amd64/trap.c:383 #9 0xffffffff803d46ab in calltrap () at ../../../amd64/amd64/exception.S:1= 68 #10 0xffffffff80333a86 in nd6_timer (ignored_arg=3D0xffffffff8059ab60) at .= ./../../netinet6/nd6.c:585 #11 0xffffffff80270bf9 in softclock (dummy=3D0xffffffff8059ab60) at ../../.= ./kern/kern_timeout.c:290 #12 0xffffffff802442a6 in ithread_execute_handlers (p=3D0xffffffff8059ab60,= ie=3D0xffffff000087b800) at ../../../kern/kern_intr.c:662 #13 0xffffffff80244423 in ithread_loop (arg=3D0xffffffff8059ab60) at ../../= ../kern/kern_intr.c:745 #14 0xffffffff80242c4a in fork_exit (callout=3D0xffffffff802443b0 , arg=3D0xffffff00000364e0, frame=3D0xffffffffabc31c90) at ../../../kern/kern_fork.c:802 #15 0xffffffff803d4a0e in fork_trampoline () at ../../../amd64/amd64/except= ion.S:394 #16 0x0000000000000000 in ?? () Previous frame identical to this frame (corrupt stack?) (kgdb) frame 10 #10 0xffffffff80333a86 in nd6_timer (ignored_arg=3D0xffffffff8059ab60) at .= ./../../netinet6/nd6.c:585 585 ia6->ia6_flags |=3D IN6_IFF_DEPRECATED; That's the same place it was before. Note that this is a use-after-free situation: memguard is monitoring the ifaddr malloc type, and caused the panic when this code attempted to write into a malloced structure after it had been freed. Kris --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFD7Y7DWry0BWjoQKURAvFoAKCiiZqOOIDWItF1a6QlxrnF4qgduwCfTHIq pzMIf9M79InkWMFm1iHYykY= =ZJJx -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 07:31:37 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2FE5C16A420 for ; Sat, 11 Feb 2006 07:31:37 +0000 (GMT) (envelope-from jinmei@impact.jinmei.org) Received: from impact.jinmei.org (kame201.kame.net [203.178.141.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id CA89343D48 for ; Sat, 11 Feb 2006 07:31:36 +0000 (GMT) (envelope-from jinmei@impact.jinmei.org) Received: by impact.jinmei.org (Postfix, from userid 2308) id AA7002E35A; Sat, 11 Feb 2006 16:31:23 +0900 (JST) From: JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: Kris Kennaway In-Reply-To: <20060211071411.GA82302@xor.obsecurity.org> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> <20060211035025.GA77114@xor.obsecurity.org> <20060211071411.GA82302@xor.obsecurity.org> User-Agent: Wanderlust/2.14.0 (Africa) Emacs/21.3 Mule/5.0 (SAKAKI) Organization: Research & Development Center, Toshiba Corp., Kawasaki, Japan. MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Message-Id: <20060211073123.AA7002E35A@impact.jinmei.org> Date: Sat, 11 Feb 2006 16:31:23 +0900 (JST) Cc: net@FreeBSD.org Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 07:31:37 -0000 >>>>> On Sat, 11 Feb 2006 02:14:11 -0500, >>>>> Kris Kennaway said: >> >> Sorry, not really (we've not got a test environment to reproduce it). >> >> But from a quick review of nd6.c, there seems to be one thing that is >> >> obviously wrong. The possible bug has been there since rev. 1.19 >> >> committed in April 2002. We've been probably just lucky so far... >> >> >> >> Could you try the patch attached below? We'll probably also need to >> >> apply this fix to 4.X and 5.X. >> >> > The patch did not fix the panic. >> >> Hmm, but this time the point where the panic happened should be >> different. Can you identify where it was? > I reduced the hw.physmem size and was able to get a dump: > (kgdb) frame 10 > #10 0xffffffff80333a86 in nd6_timer (ignored_arg=0xffffffff8059ab60) at ../../../netinet6/nd6.c:585 > 585 ia6->ia6_flags |= IN6_IFF_DEPRECATED; Are you sure you applied the patch? In the 'patched' version of nd6.c, line 585 is blank, so at least it doesn't match the above backtrace. To make it very clear, I've put a copy of 'before' and 'after' the patch to nd6.c at: http://www.jinmei.org/nd6.c.orig and http://www.jinmei.org/nd6.c respectively. It seems you are still using nd6.c.orig, whose line 585 sets the IN6_IFF_DEPRECATED flag. JINMEI, Tatuya Communication Platform Lab. Corporate R&D Center, Toshiba Corp. jinmei@isl.rdc.toshiba.co.jp From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 07:32:49 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B46FC16A420 for ; Sat, 11 Feb 2006 07:32:49 +0000 (GMT) (envelope-from suz@alaxala.net) Received: from pc1.alaxala.net (pos-2-0.hitachi2.fujisawa.wide.ad.jp [203.178.142.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id A963443D68 for ; Sat, 11 Feb 2006 07:32:41 +0000 (GMT) (envelope-from suz@alaxala.net) Received: from localhost (localhost [127.0.0.1]) by pc1.alaxala.net (Postfix) with ESMTP id 1F4A3BB5C; Sat, 11 Feb 2006 16:32:41 +0900 (JST) Received: from pc1.alaxala.net ([127.0.0.1]) by localhost (pc1.alaxala.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 42207-03; Sat, 11 Feb 2006 16:32:37 +0900 (JST) Received: from flora220.uki-uki.net (240.163.192.61.tokyo.global.alpha-net.ne.jp [61.192.163.240]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pc1.alaxala.net (Postfix) with ESMTP id BCFD6BB59; Sat, 11 Feb 2006 16:32:37 +0900 (JST) Date: Sat, 11 Feb 2006 16:32:15 +0900 Message-ID: From: SUZUKI Shinsuke To: frank@ircnow.org X-cite: xcite 1.33 In-Reply-To: <20060211025810.GA11128@scott.blazing.de> References: <20060211025810.GA11128@scott.blazing.de> User-Agent: Wanderlust/2.15.1 (Almost Unreal) Emacs/22.0 Mule/5.0 (SAKAKI) Organization: Technical Marketing Dept., ALAXALA Networks Corporation MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: amavisd-new at alaxala.net Cc: freebsd-net@freebsd.org Subject: Re: Can't delete v6 addy from gif0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 07:32:50 -0000 Hi, >>>>> On Sat, 11 Feb 2006 03:58:10 +0100 >>>>> frank@ircnow.org(Frank Steinborn) said: > i'm trying to delete an IPv6-address from my gif-interface (named > blazing): > > shodan:~# ifconfig blazing inet6 delete 2001:1638:17ad::9 > ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address > > I tried 2001:1638:17ad:0:0:0:0:9 too without success. Adding the > address (ifconfig blazing inet6 add) worked fine, though. Any hints? please try the folllowing command, instead. ifconfig blazing inet6 2001:1638:17ad::9 delete As is written in man ifconfig, "delete" (alias of "-alias") should appear after the address argument. Thanks, ---- SUZUKI, Shinsuke @ KAME Project From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 13:37:16 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAFED16A420 for ; Sat, 11 Feb 2006 13:37:15 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from mrout2.yahoo.com (mrout2.yahoo.com [216.145.54.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DF9843D45 for ; Sat, 11 Feb 2006 13:37:15 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (proxy8.corp.yahoo.com [216.145.48.13]) by mrout2.yahoo.com (8.13.4/8.13.4/y.out) with ESMTP id k1BDb3g0048056 for ; Sat, 11 Feb 2006 05:37:04 -0800 (PST) Date: Sat, 11 Feb 2006 22:37:11 +0900 Message-ID: From: gnn@freebsd.org To: freebsd-net@freebsd.org User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.7 (=?ISO-8859-4?Q?Sanj=F2?=) APEL/10.6 Emacs/22.0.50 (powerpc-apple-darwin8.3.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Subject: Separating the kernel and user land versions of PF_KEY. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 13:37:16 -0000 Hi Folks, The attached patch makes it so that the user land and kernel land versions of the pf_key structures are different and therefore no longer dependent. This is one step in moving us away from the place we're in now where changes to one side require changes in the other. At some point soon a more full overhaul of the code will take place, likely along the lines of the code found in OpenBSD (look at net/pfkey*.[ch] there). Please send feedback etc. on this patch along to me. BTW Although this patch contains a lot of p4 cruft it applies cleanly against HEAD, at least as of a few days ago and passes the CT test suite ipsec4 which is available by installing the ct port. Thanks, George Change 89716 by gnn@gnn_devbox_fast_ipsec on 2006/01/15 02:41:52 First cut at removing PF_KEY data structures from the keydb. This code does not work completely yet but needs to be saved. Affected files ... ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/ipsec.c#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key.c#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key_var.h#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/keydb.h#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_ah.c#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_esp.c#2 edit ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_tcp.c#2 edit Differences ... ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/ipsec.c#2 (text+ko) ==== Index: sys/netipsec/ipsec.c --- sys/netipsec/ipsec.c.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/ipsec.c Sun Feb 5 15:06:16 2006 @@ -92,6 +92,7 @@ #include +#define IPSEC_DEBUG #ifdef IPSEC_DEBUG int ipsec_debug = 1; #else ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key.c#2 (text+ko) ==== Index: sys/netipsec/key.c --- sys/netipsec/key.c.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/key.c Sun Feb 5 15:06:16 2006 @@ -420,7 +420,10 @@ static struct mbuf *key_setsadbxsa2 __P((u_int8_t, u_int32_t, u_int32_t)); static struct mbuf *key_setsadbxpolicy __P((u_int16_t, u_int8_t, u_int32_t)); -static void *key_dup(const void *, u_int, struct malloc_type *); +static struct seckey *key_dup_keymsg(const struct sadb_key *, u_int, + struct malloc_type *); +static struct seclifetime *key_dup_lifemsg(const struct sadb_lifetime *src, + struct malloc_type *type); #ifdef INET6 static int key_ismyaddr6 __P((struct sockaddr_in6 *)); #endif @@ -488,6 +491,10 @@ static int key_senderror __P((struct socket *, struct mbuf *, int)); static int key_validate_ext __P((const struct sadb_ext *, int)); static int key_align __P((struct mbuf *, struct sadb_msghdr *)); +static struct mbuf *key_setlifetime(struct seclifetime *src, + u_int16_t exttype); +static struct mbuf *key_setkey(struct seckey *src, u_int16_t exttype); + #if 0 static const char *key_getfqdn __P((void)); static const char *key_getuserfqdn __P((void)); @@ -909,8 +916,8 @@ /* What the best method is to compare ? */ if (key_preferred_oldsa) { - if (candidate->lft_c->sadb_lifetime_addtime > - sav->lft_c->sadb_lifetime_addtime) { + if (candidate->lft_c->addtime > + sav->lft_c->addtime) { candidate = sav; } continue; @@ -918,8 +925,8 @@ } /* preferred new sa rather than old sa */ - if (candidate->lft_c->sadb_lifetime_addtime < - sav->lft_c->sadb_lifetime_addtime) { + if (candidate->lft_c->addtime < + sav->lft_c->addtime) { d = candidate; candidate = sav; } else @@ -930,7 +937,7 @@ * suitable candidate and the lifetime of the SA is not * permanent. */ - if (d->lft_c->sadb_lifetime_addtime != 0) { + if (d->lft_c->addtime != 0) { struct mbuf *m, *result; u_int8_t satype; @@ -2787,9 +2794,9 @@ } else { KASSERT(sav->iv == NULL, ("iv but no xform")); if (sav->key_auth != NULL) - bzero(_KEYBUF(sav->key_auth), _KEYLEN(sav->key_auth)); + bzero(sav->key_auth->key_data, _KEYLEN(sav->key_auth)); if (sav->key_enc != NULL) - bzero(_KEYBUF(sav->key_enc), _KEYLEN(sav->key_enc)); + bzero(sav->key_enc->key_data, _KEYLEN(sav->key_enc)); } if (sav->key_auth != NULL) { free(sav->key_auth, M_IPSEC_MISC); @@ -3038,9 +3045,11 @@ goto fail; } - sav->key_auth = key_dup(key0, len, M_IPSEC_MISC); - if (sav->key_auth == NULL) { - ipseclog((LOG_DEBUG, "%s: No more memory.\n",__func__)); + sav->key_auth = (struct seckey *)key_dup_keymsg(key0, len, + M_IPSEC_MISC); + if (sav->key_auth == NULL ) { + ipseclog((LOG_DEBUG, "%s: No more memory.\n", + __func__)); error = ENOBUFS; goto fail; } @@ -3061,12 +3070,15 @@ } switch (mhp->msg->sadb_msg_satype) { case SADB_SATYPE_ESP: + /* XXX FIX ME */ if (len == PFKEY_ALIGN8(sizeof(struct sadb_key)) && sav->alg_enc != SADB_EALG_NULL) { error = EINVAL; break; } - sav->key_enc = key_dup(key0, len, M_IPSEC_MISC); + sav->key_enc = (struct seckey *)key_dup_keymsg(key0, + len, + M_IPSEC_MISC); if (sav->key_enc == NULL) { ipseclog((LOG_DEBUG, "%s: No more memory.\n", __func__)); @@ -3126,13 +3138,10 @@ goto fail; } - sav->lft_c->sadb_lifetime_len = - PFKEY_UNIT64(sizeof(struct sadb_lifetime)); - sav->lft_c->sadb_lifetime_exttype = SADB_EXT_LIFETIME_CURRENT; - sav->lft_c->sadb_lifetime_allocations = 0; - sav->lft_c->sadb_lifetime_bytes = 0; - sav->lft_c->sadb_lifetime_addtime = time_second; - sav->lft_c->sadb_lifetime_usetime = 0; + sav->lft_c->allocations = 0; + sav->lft_c->bytes = 0; + sav->lft_c->addtime = time_second; + sav->lft_c->usetime = 0; /* lifetimes for HARD and SOFT */ { @@ -3144,7 +3153,7 @@ error = EINVAL; goto fail; } - sav->lft_h = key_dup(lft0, sizeof(*lft0), M_IPSEC_MISC); + sav->lft_h = key_dup_lifemsg(lft0, M_IPSEC_MISC); if (sav->lft_h == NULL) { ipseclog((LOG_DEBUG, "%s: No more memory.\n",__func__)); error = ENOBUFS; @@ -3159,7 +3168,7 @@ error = EINVAL; goto fail; } - sav->lft_s = key_dup(lft0, sizeof(*lft0), M_IPSEC_MISC); + sav->lft_s = key_dup_lifemsg(lft0, M_IPSEC_MISC); if (sav->lft_s == NULL) { ipseclog((LOG_DEBUG, "%s: No more memory.\n",__func__)); error = ENOBUFS; @@ -3271,9 +3280,7 @@ u_int32_t seq, pid; { struct mbuf *result = NULL, *tres = NULL, *m; - int l = 0; int i; - void *p; int dumporder[] = { SADB_EXT_SA, SADB_X_EXT_SA2, SADB_EXT_LIFETIME_HARD, SADB_EXT_LIFETIME_SOFT, @@ -3290,7 +3297,6 @@ for (i = sizeof(dumporder)/sizeof(dumporder[0]) - 1; i >= 0; i--) { m = NULL; - p = NULL; switch (dumporder[i]) { case SADB_EXT_SA: m = key_setsadbsa(sav); @@ -3325,36 +3331,45 @@ case SADB_EXT_KEY_AUTH: if (!sav->key_auth) continue; - l = PFKEY_UNUNIT64(sav->key_auth->sadb_key_len); - p = sav->key_auth; + m = key_setkey(sav->key_auth, SADB_EXT_KEY_AUTH); + if (!m) + goto fail; break; case SADB_EXT_KEY_ENCRYPT: if (!sav->key_enc) continue; - l = PFKEY_UNUNIT64(sav->key_enc->sadb_key_len); - p = sav->key_enc; + m = key_setkey(sav->key_enc, SADB_EXT_KEY_ENCRYPT); + if (!m) + goto fail; break; case SADB_EXT_LIFETIME_CURRENT: if (!sav->lft_c) continue; - l = PFKEY_UNUNIT64(((struct sadb_ext *)sav->lft_c)->sadb_ext_len); - p = sav->lft_c; + m = key_setlifetime(sav->lft_c, + SADB_EXT_LIFETIME_CURRENT); + if (!m) + goto fail; break; case SADB_EXT_LIFETIME_HARD: if (!sav->lft_h) continue; - l = PFKEY_UNUNIT64(((struct sadb_ext *)sav->lft_h)->sadb_ext_len); - p = sav->lft_h; + m = key_setlifetime(sav->lft_h, + SADB_EXT_LIFETIME_HARD); + if (!m) + goto fail; break; case SADB_EXT_LIFETIME_SOFT: if (!sav->lft_s) continue; - l = PFKEY_UNUNIT64(((struct sadb_ext *)sav->lft_s)->sadb_ext_len); - p = sav->lft_s; + m = key_setlifetime(sav->lft_h, + SADB_EXT_LIFETIME_SOFT); + + if (!m) + goto fail; break; case SADB_EXT_ADDRESS_PROXY: @@ -3366,29 +3381,15 @@ continue; } - if ((!m && !p) || (m && p)) + if (!m) goto fail; - if (p && tres) { - M_PREPEND(tres, l, M_DONTWAIT); - if (!tres) - goto fail; - bcopy(p, mtod(tres, caddr_t), l); - continue; - } - if (p) { - m = key_alloc_mbuf(l); - if (!m) - goto fail; - m_copyback(m, 0, l, p); - } - if (tres) m_cat(m, tres); tres = m; + } m_cat(result, tres); - if (result->m_len < sizeof(struct sadb_msg)) { result = m_pullup(result, sizeof(struct sadb_msg)); if (result == NULL) @@ -3609,21 +3610,67 @@ } /* %%% utilities */ -/* - * copy a buffer into the new buffer allocated. +/* Take a key message (sadb_key) from the socket and turn it into one + * of the kernel's key structures (seckey). + * + * IN: pointer to the src + * OUT: NULL no more memory + */ +struct seckey * +key_dup_keymsg(const struct sadb_key *src, u_int len, + struct malloc_type *type) +{ + struct seckey *dst = NULL; + dst = (struct seckey *)malloc(sizeof(struct seckey), type, M_NOWAIT); + if (dst != NULL) { + dst->bits = src->sadb_key_bits; + dst->key_data = (char *)malloc(len, type, M_NOWAIT); + if (dst->key_data != NULL) { + bcopy(src + sizeof(struct sadb_key), + dst->key_data, len); + ipseclog((LOG_DEBUG, "%s: source bits %p\n", __func__, + src + sizeof(struct sadb_key))); + ipseclog((LOG_DEBUG, "%s: dst bits %p\n", __func__, + dst->key_data)); + } else { + ipseclog((LOG_DEBUG, "%s: No more memory.\n", + __func__)); + free(dst, type); + dst = NULL; + } + } else { + ipseclog((LOG_DEBUG, "%s: No more memory.\n", + __func__)); + + } + return dst; +} + +/* Take a lifetime message (sadb_lifetime) passed in on a socket and + * turn it into one of the kernel's lifetime structures (seclifetime). + * + * IN: pointer to the destination, source and malloc type + * OUT: NULL, no more memory */ -static void * -key_dup(const void *src, u_int len, struct malloc_type *type) + +static struct seclifetime * +key_dup_lifemsg(const struct sadb_lifetime *src, + struct malloc_type *type) { - void *copy; + struct seclifetime *dst = NULL; - copy = malloc(len, type, M_NOWAIT); - if (copy == NULL) { + dst = (struct seclifetime *)malloc(sizeof(struct seclifetime), + type, M_NOWAIT); + if (dst == NULL) { /* XXX counter */ ipseclog((LOG_DEBUG, "%s: No more memory.\n", __func__)); - } else - bcopy(src, copy, len); - return copy; + } else { + dst->allocations = src->sadb_lifetime_allocations; + dst->bytes = src->sadb_lifetime_bytes; + dst->addtime = src->sadb_lifetime_addtime; + dst->usetime = src->sadb_lifetime_usetime; + } + return dst; } /* compare my own address @@ -4071,13 +4118,13 @@ } /* check SOFT lifetime */ - if (sav->lft_s->sadb_lifetime_addtime != 0 && - now - sav->created > sav->lft_s->sadb_lifetime_addtime) { + if (sav->lft_s->addtime != 0 && + now - sav->created > sav->lft_s->addtime) { /* * check SA to be used whether or not. * when SA hasn't been used, delete it. */ - if (sav->lft_c->sadb_lifetime_usetime == 0) { + if (sav->lft_c->usetime == 0) { key_sa_chgstate(sav, SADB_SASTATE_DEAD); KEY_FREESAV(&sav); } else { @@ -4096,8 +4143,8 @@ * when new SA is installed. Caution when it's * installed too big lifetime by time. */ - else if (sav->lft_s->sadb_lifetime_bytes != 0 && - sav->lft_s->sadb_lifetime_bytes < sav->lft_c->sadb_lifetime_bytes) { + else if (sav->lft_s->bytes != 0 && + sav->lft_s->bytes < sav->lft_c->bytes) { key_sa_chgstate(sav, SADB_SASTATE_DYING); /* @@ -4122,15 +4169,15 @@ continue; } - if (sav->lft_h->sadb_lifetime_addtime != 0 && - now - sav->created > sav->lft_h->sadb_lifetime_addtime) { + if (sav->lft_h->addtime != 0 && + now - sav->created > sav->lft_h->addtime) { key_sa_chgstate(sav, SADB_SASTATE_DEAD); KEY_FREESAV(&sav); } #if 0 /* XXX Should we keep to send expire message until HARD lifetime ? */ else if (sav->lft_s != NULL - && sav->lft_s->sadb_lifetime_addtime != 0 - && now - sav->created > sav->lft_s->sadb_lifetime_addtime) { + && sav->lft_s->addtime != 0 + && now - sav->created > sav->lft_s->addtime) { /* * XXX: should be checked to be * installed the valid SA. @@ -4144,8 +4191,8 @@ } #endif /* check HARD lifetime by bytes */ - else if (sav->lft_h->sadb_lifetime_bytes != 0 && - sav->lft_h->sadb_lifetime_bytes < sav->lft_c->sadb_lifetime_bytes) { + else if (sav->lft_h->bytes != 0 && + sav->lft_h->bytes < sav->lft_c->bytes) { key_sa_chgstate(sav, SADB_SASTATE_DEAD); KEY_FREESAV(&sav); } @@ -4977,20 +5024,23 @@ } /* make structure */ - sah->idents = malloc(idsrclen, M_IPSEC_MISC, M_NOWAIT); + sah->idents = malloc(sizeof(struct secident), M_IPSEC_MISC, M_NOWAIT); if (sah->idents == NULL) { ipseclog((LOG_DEBUG, "%s: No more memory.\n", __func__)); return ENOBUFS; } - sah->identd = malloc(iddstlen, M_IPSEC_MISC, M_NOWAIT); + sah->identd = malloc(sizeof(struct secident), M_IPSEC_MISC, M_NOWAIT); if (sah->identd == NULL) { free(sah->idents, M_IPSEC_MISC); sah->idents = NULL; ipseclog((LOG_DEBUG, "%s: No more memory.\n", __func__)); return ENOBUFS; } - bcopy(idsrc, sah->idents, idsrclen); - bcopy(iddst, sah->identd, iddstlen); + sah->idents->type = idsrc->sadb_ident_type; + sah->idents->id = idsrc->sadb_ident_id; + + sah->identd->type = iddst->sadb_ident_type; + sah->identd->id = iddst->sadb_ident_id; return 0; } @@ -6262,10 +6312,10 @@ lt = mtod(m, struct sadb_lifetime *); lt->sadb_lifetime_len = PFKEY_UNIT64(sizeof(struct sadb_lifetime)); lt->sadb_lifetime_exttype = SADB_EXT_LIFETIME_CURRENT; - lt->sadb_lifetime_allocations = sav->lft_c->sadb_lifetime_allocations; - lt->sadb_lifetime_bytes = sav->lft_c->sadb_lifetime_bytes; - lt->sadb_lifetime_addtime = sav->lft_c->sadb_lifetime_addtime; - lt->sadb_lifetime_usetime = sav->lft_c->sadb_lifetime_usetime; + lt->sadb_lifetime_allocations = sav->lft_c->allocations; + lt->sadb_lifetime_bytes = sav->lft_c->bytes; + lt->sadb_lifetime_addtime = sav->lft_c->addtime; + lt->sadb_lifetime_usetime = sav->lft_c->usetime; lt = (struct sadb_lifetime *)(mtod(m, caddr_t) + len / 2); bcopy(sav->lft_s, lt, sizeof(*lt)); m_cat(result, m); @@ -7103,21 +7153,21 @@ * XXX Currently, there is a difference of bytes size * between inbound and outbound processing. */ - sav->lft_c->sadb_lifetime_bytes += m->m_pkthdr.len; + sav->lft_c->bytes += m->m_pkthdr.len; /* to check bytes lifetime is done in key_timehandler(). */ /* * We use the number of packets as the unit of - * sadb_lifetime_allocations. We increment the variable + * allocations. We increment the variable * whenever {esp,ah}_{in,out}put is called. */ - sav->lft_c->sadb_lifetime_allocations++; + sav->lft_c->allocations++; /* XXX check for expires? */ /* - * NOTE: We record CURRENT sadb_lifetime_usetime by using wall clock, + * NOTE: We record CURRENT usetime by using wall clock, * in seconds. HARD and SOFT lifetime are measured by the time - * difference (again in seconds) from sadb_lifetime_usetime. + * difference (again in seconds) from usetime. * * usetime * v expire expire @@ -7125,7 +7175,7 @@ * <--------------> HARD * <-----> SOFT */ - sav->lft_c->sadb_lifetime_usetime = time_second; + sav->lft_c->usetime = time_second; /* XXX check for expires? */ return; @@ -7214,3 +7264,55 @@ return m; } + +static struct mbuf * +key_setkey(struct seckey *src, u_int16_t exttype) +{ + struct mbuf *m; + struct sadb_key *p; + int len = PFKEY_ALIGN8(sizeof(struct sadb_key)); + + if (src == NULL) + return NULL; + + m = key_alloc_mbuf(len); + if (m == NULL) + return NULL; + p = mtod(m, struct sadb_key *); + bzero(p, len); + p->sadb_key_len = PFKEY_UNIT64(len); + p->sadb_key_exttype = exttype; + p->sadb_key_bits = src->bits; + ipseclog((LOG_DEBUG, "%s: setting key data %s\n", + __func__, src->key_data)); + bcopy(src->key_data, _KEYBUF(p), len); + + return m; +} + +static struct mbuf * +key_setlifetime(struct seclifetime *src, u_int16_t exttype) +{ + struct mbuf *m = NULL; + struct sadb_lifetime *p; + int len = PFKEY_ALIGN8(sizeof(struct sadb_lifetime)); + + if (src == NULL) + return NULL; + + m = key_alloc_mbuf(len); + if (m == NULL) + return m; + p = mtod(m, struct sadb_lifetime *); + + bzero(p, len); + p->sadb_lifetime_len = PFKEY_UNIT64(len); + p->sadb_lifetime_exttype = exttype; + p->sadb_lifetime_allocations = src->allocations; + p->sadb_lifetime_bytes = src->bytes; + p->sadb_lifetime_addtime = src->addtime; + p->sadb_lifetime_usetime = src->usetime; + + return m; + +} ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key_var.h#2 (text+ko) ==== Index: sys/netipsec/key_var.h --- sys/netipsec/key_var.h.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/key_var.h Sun Feb 5 15:06:16 2006 @@ -66,8 +66,8 @@ #ifdef _KERNEL #define _ARRAYLEN(p) (sizeof(p)/sizeof(p[0])) -#define _KEYLEN(key) ((u_int)((key)->sadb_key_bits >> 3)) -#define _KEYBITS(key) ((u_int)((key)->sadb_key_bits)) +#define _KEYLEN(key) ((u_int)((key)->bits >> 3)) +#define _KEYBITS(key) ((u_int)((key)->bits)) #define _KEYBUF(key) ((caddr_t)((caddr_t)(key) + sizeof(struct sadb_key))) #endif /*_KERNEL*/ ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/keydb.h#2 (text+ko) ==== Index: sys/netipsec/keydb.h --- sys/netipsec/keydb.h.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/keydb.h Sun Feb 5 15:06:16 2006 @@ -60,14 +60,39 @@ /* see IPSEC_MANUAL_REQID_MAX. */ }; +/* + * In order to split out the keydb implementation from that of the + * PF_KEY sockets we need to define a few structures that while they + * may seem common are likely to diverge over time. + */ + +/* sadb_identity */ +struct secident { + u_int16_t type; + u_int64_t id; +}; + +/* sadb_key */ +struct seckey { + u_int16_t bits; + char *key_data; +}; + +struct seclifetime { + u_int32_t allocations; + u_int64_t bytes; + u_int64_t addtime; + u_int64_t usetime; +}; + /* Security Association Data Base */ struct secashead { LIST_ENTRY(secashead) chain; struct secasindex saidx; - struct sadb_ident *idents; /* source identity */ - struct sadb_ident *identd; /* destination identity */ + struct secident *idents; /* source identity */ + struct secident *identd; /* destination identity */ /* XXX I don't know how to use them. */ u_int8_t state; /* MATURE or DEAD. */ @@ -97,8 +122,8 @@ u_int32_t spi; /* SPI Value, network byte order */ u_int32_t flags; /* holder for SADB_KEY_FLAGS */ - struct sadb_key *key_auth; /* Key for Authentication */ - struct sadb_key *key_enc; /* Key for Encryption */ + struct seckey *key_auth; /* Key for Authentication */ + struct seckey *key_enc; /* Key for Encryption */ caddr_t iv; /* Initilization Vector */ u_int ivlen; /* length of IV */ void *sched; /* intermediate encryption key */ @@ -107,9 +132,9 @@ struct secreplay *replay; /* replay prevention */ time_t created; /* for lifetime */ - struct sadb_lifetime *lft_c; /* CURRENT lifetime, it's constant. */ - struct sadb_lifetime *lft_h; /* HARD lifetime */ - struct sadb_lifetime *lft_s; /* SOFT lifetime */ + struct seclifetime *lft_c; /* CURRENT lifetime, it's constant. */ + struct seclifetime *lft_h; /* HARD lifetime */ + struct seclifetime *lft_s; /* SOFT lifetime */ u_int32_t seq; /* sequence number */ pid_t pid; /* message's pid */ ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_ah.c#2 (text+ko) ==== Index: sys/netipsec/xform_ah.c --- sys/netipsec/xform_ah.c.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/xform_ah.c Sun Feb 5 15:06:16 2006 @@ -201,7 +201,7 @@ bzero(cria, sizeof (*cria)); cria->cri_alg = sav->tdb_authalgxform->type; cria->cri_klen = _KEYBITS(sav->key_auth); - cria->cri_key = _KEYBUF(sav->key_auth); + cria->cri_key = sav->key_auth->key_data; return 0; } @@ -231,7 +231,7 @@ int err; if (sav->key_auth) - bzero(_KEYBUF(sav->key_auth), _KEYLEN(sav->key_auth)); + bzero(sav->key_auth->key_data, _KEYLEN(sav->key_auth)); err = crypto_freesession(sav->tdb_cryptoid); sav->tdb_cryptoid = 0; @@ -622,8 +622,8 @@ /* Authentication operation. */ crda->crd_alg = ahx->type; - crda->crd_key = _KEYBUF(sav->key_auth); crda->crd_klen = _KEYBITS(sav->key_auth); + crda->crd_key = sav->key_auth->key_data; /* Find out if we've already done crypto. */ for (mtag = m_tag_find(m, PACKET_TAG_IPSEC_IN_CRYPTO_DONE, NULL); @@ -1014,7 +1014,7 @@ /* Authentication operation. */ crda->crd_alg = ahx->type; - crda->crd_key = _KEYBUF(sav->key_auth); + crda->crd_key = sav->key_auth->key_data; crda->crd_klen = _KEYBITS(sav->key_auth); /* Allocate IPsec-specific opaque crypto info. */ ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_esp.c#2 (text+ko) ==== Index: sys/netipsec/xform_esp.c --- sys/netipsec/xform_esp.c.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/xform_esp.c Sun Feb 5 15:06:16 2006 @@ -215,7 +215,7 @@ bzero(&crie, sizeof (crie)); crie.cri_alg = sav->tdb_encalgxform->type; crie.cri_klen = _KEYBITS(sav->key_enc); - crie.cri_key = _KEYBUF(sav->key_enc); + crie.cri_key = sav->key_enc->key_data; /* XXX Rounds ? */ if (sav->tdb_authalgxform && sav->tdb_encalgxform) { @@ -248,7 +248,7 @@ int error = ah_zeroize(sav); if (sav->key_enc) - bzero(_KEYBUF(sav->key_enc), _KEYLEN(sav->key_enc)); + bzero(sav->key_enc->key_data, _KEYLEN(sav->key_enc)); if (sav->iv) { free(sav->iv, M_XDATA); sav->iv = NULL; @@ -381,7 +381,7 @@ crda->crd_inject = m->m_pkthdr.len - alen; crda->crd_alg = esph->type; - crda->crd_key = _KEYBUF(sav->key_auth); + crda->crd_key = sav->key_auth->key_data; crda->crd_klen = _KEYBITS(sav->key_auth); /* Copy the authenticator */ @@ -418,7 +418,7 @@ crde->crd_inject = skip + hlen - sav->ivlen; crde->crd_alg = espx->type; - crde->crd_key = _KEYBUF(sav->key_enc); + crde->crd_key = sav->key_enc->key_data; crde->crd_klen = _KEYBITS(sav->key_enc); /* XXX Rounds ? */ } @@ -802,7 +802,7 @@ /* Encryption operation. */ crde->crd_alg = espx->type; - crde->crd_key = _KEYBUF(sav->key_enc); + crde->crd_key = sav->key_enc->key_data; crde->crd_klen = _KEYBITS(sav->key_enc); /* XXX Rounds ? */ } else @@ -841,7 +841,7 @@ /* Authentication operation. */ crda->crd_alg = esph->type; - crda->crd_key = _KEYBUF(sav->key_auth); + crda->crd_key = sav->key_auth->key_data; crda->crd_klen = _KEYBITS(sav->key_auth); } ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/xform_tcp.c#2 (text+ko) ==== Index: sys/netipsec/xform_tcp.c --- sys/netipsec/xform_tcp.c.~1~ Sun Feb 5 15:06:16 2006 +++ sys/netipsec/xform_tcp.c Sun Feb 5 15:06:16 2006 @@ -117,7 +117,7 @@ { if (sav->key_auth) - bzero(_KEYBUF(sav->key_auth), _KEYLEN(sav->key_auth)); + bzero(sav->key_auth->key_data, _KEYLEN(sav->key_auth)); sav->tdb_cryptoid = 0; sav->tdb_authalgxform = NULL; End of Patch. Change 90069 by gnn@gnn_tahi_fast_ipsec on 2006/01/21 13:06:06 Fix pointer arithmetic so that we actually put the key in the database and not random garbage. First working version with new structures. Affected files ... ... //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key.c#3 edit Differences ... ==== //depot/projects/gnn_fast_ipsec/src/sys/netipsec/key.c#3 (text+ko) ==== Index: sys/netipsec/key.c --- sys/netipsec/key.c.~1~ Sun Feb 5 15:06:27 2006 +++ sys/netipsec/key.c Sun Feb 5 15:06:27 2006 @@ -2799,10 +2799,14 @@ bzero(sav->key_enc->key_data, _KEYLEN(sav->key_enc)); } if (sav->key_auth != NULL) { + if (sav->key_auth->key_data != NULL) + free(sav->key_auth->key_data, M_IPSEC_MISC); free(sav->key_auth, M_IPSEC_MISC); sav->key_auth = NULL; } if (sav->key_enc != NULL) { + if (sav->key_enc->key_data != NULL) + free(sav->key_enc->key_data, M_IPSEC_MISC); free(sav->key_enc, M_IPSEC_MISC); sav->key_enc = NULL; } @@ -3070,7 +3074,6 @@ } switch (mhp->msg->sadb_msg_satype) { case SADB_SATYPE_ESP: - /* XXX FIX ME */ if (len == PFKEY_ALIGN8(sizeof(struct sadb_key)) && sav->alg_enc != SADB_EALG_NULL) { error = EINVAL; @@ -3620,18 +3623,14 @@ key_dup_keymsg(const struct sadb_key *src, u_int len, struct malloc_type *type) { - struct seckey *dst = NULL; + struct seckey *dst; dst = (struct seckey *)malloc(sizeof(struct seckey), type, M_NOWAIT); if (dst != NULL) { dst->bits = src->sadb_key_bits; dst->key_data = (char *)malloc(len, type, M_NOWAIT); if (dst->key_data != NULL) { - bcopy(src + sizeof(struct sadb_key), + bcopy((const char *)src + sizeof(struct sadb_key), dst->key_data, len); - ipseclog((LOG_DEBUG, "%s: source bits %p\n", __func__, - src + sizeof(struct sadb_key))); - ipseclog((LOG_DEBUG, "%s: dst bits %p\n", __func__, - dst->key_data)); } else { ipseclog((LOG_DEBUG, "%s: No more memory.\n", __func__)); @@ -7265,12 +7264,25 @@ return m; } +/* + * Take one of the kernel's security keys and convert it into a PF_KEY + * structure within an mbuf, suitable for sending up to a waiting + * application in user land. + * + * IN: + * src: A pointer to a kernel security key. + * exttype: Which type of key this is. Refer to the PF_KEY data structures. + * OUT: + * a valid mbuf or NULL indicating an error + * + */ + static struct mbuf * key_setkey(struct seckey *src, u_int16_t exttype) { struct mbuf *m; struct sadb_key *p; - int len = PFKEY_ALIGN8(sizeof(struct sadb_key)); + int len = PFKEY_ALIGN8(sizeof(struct sadb_key) + _KEYLEN(src)); if (src == NULL) return NULL; @@ -7285,11 +7297,25 @@ p->sadb_key_bits = src->bits; ipseclog((LOG_DEBUG, "%s: setting key data %s\n", __func__, src->key_data)); - bcopy(src->key_data, _KEYBUF(p), len); + bcopy(src->key_data, _KEYBUF(p), _KEYLEN(src)); return m; } +/* + * Take one of the kernel's lifetime data structures and convert it + * into a PF_KEY structure within an mbuf, suitable for sending up to + * a waiting application in user land. + * + * IN: + * src: A pointer to a kernel lifetime structure. + * exttype: Which type of lifetime this is. Refer to the PF_KEY + * data structures for more information. + * OUT: + * a valid mbuf or NULL indicating an error + * + */ + static struct mbuf * key_setlifetime(struct seclifetime *src, u_int16_t exttype) { End of Patch. From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 16:35:38 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DFDA416A420 for ; Sat, 11 Feb 2006 16:35:38 +0000 (GMT) (envelope-from gizmen@blurp.t2.ds.pwr.wroc.pl) Received: from blurp.t2.ds.pwr.wroc.pl (blurp.t2.ds.pwr.wroc.pl [156.17.224.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 47A5243D48 for ; Sat, 11 Feb 2006 16:35:38 +0000 (GMT) (envelope-from gizmen@blurp.t2.ds.pwr.wroc.pl) Received: from localhost (localhost [127.0.0.1]) by blurp.t2.ds.pwr.wroc.pl (Postfix) with ESMTP id 53910738 for ; Sat, 11 Feb 2006 17:35:34 +0100 (CET) Received: from blurp.t2.ds.pwr.wroc.pl ([127.0.0.1]) by localhost (blurp.pl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 78617-03 for ; Sat, 11 Feb 2006 17:35:33 +0100 (CET) Received: by blurp.t2.ds.pwr.wroc.pl (Postfix, from userid 1001) id 79316722; Sat, 11 Feb 2006 17:35:33 +0100 (CET) Date: Sat, 11 Feb 2006 17:35:33 +0100 From: GiZmen To: net@freebsd.org Message-ID: <20060211163533.GA87353@blurp.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.11 X-Virus-Scanned: amavisd-new at blurp.pl Cc: Subject: fastforward problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 16:35:39 -0000 Hi, I would like to use fastforward option on my freebsd 6.0-stable box. But i have strange problem with it. My box is a router for LAN with IP visible to internet. I am managinc C class network. When i turn on fastforward option any of the computers in my network can't open google.com page. This option is quite good because i have quite a lot interrupts and this option lower this about half. Could anyone tell me why i can't open google.com page ? I have tested different pages and i don't have such problem only with google. THANKS for any comments -- Best Regards: GiZmen UNIX is user-friendly; it's just picky about its friends UNIX is simple; it just takes a genius to understand its simplicity From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 17:45:15 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E90BF16A420 for ; Sat, 11 Feb 2006 17:45:14 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 91E5143D45 for ; Sat, 11 Feb 2006 17:45:14 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 1385746BB4; Sat, 11 Feb 2006 12:45:02 -0500 (EST) Date: Sat, 11 Feb 2006 17:48:04 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: dima <_pppp@mail.ru> In-Reply-To: Message-ID: <20060211174648.X90460@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marcos Bedinelli , freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 17:45:15 -0000 On Sat, 11 Feb 2006, dima wrote: >> The system is mainly being used as a dedicated router. It runs OSPF, BGP >> and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. The box >> has 2 gigabit interfaces that handle on average 200Mbp/s - 50K packets/s >> (inbound and outbound combined), each one of them. > > The second CPU wouldn't help you for sure. There's only one [swi1: net] kernel thread which deals with all the kernel traffic. The option of per-CPU [swi: net] threads was discussed on freebsd-arch@ several months ago, but it wouldn't be implemented soon. So, the only hardware option is installing the fastest CPU possible. > There are several software (FreeBSD specific) options though: If you set net.isr.direct=1, the netisr workload is moved from the netisr thread to the thread performing the dispatch -- typically, the ithread. If you have multiple interfaces and they are assigned different ithreads, then the work can occur in parallel. However, there are some other properties of this setting that are important, so it affects different workloads in different ways. Robert N M Watson > 1. You should surely try polling(4). 50kpps mean 50000 interrupts and the same amount of context switches, which are quite expensive. > 2. FastForwarding. It's the most suitable for you. As I know, Quagga inserts its dynamic routes to the system routing table. And FastForwarding is aware of routing table and firewall rules. And the most exciting: you can switch it on/off without reboot: > # sysctl net.inet.ip.fastforwarding=1 > The only limitation is it applies to IPv4 unicast traffic only. There's no documentation on this feature as i know (am I wrong, or should I report this as a documentation bug?) but you can look at the comments in the beginning of /sys/netinet/ip_fastfwd.c > The authors reported up to 1Mpps (see page 10 at http://people.freebsd.org/~andre/FreeBSD-5.3-Networking.pdf) > >> >> >> Some of you have asked for the following information: >> >> >> - As I indicated before, polling is currently disabled. >> >> >> - Hyperthreading (HTT) is disabled. >> >> >> mull [~]$vmstat -i >> interrupt total rate >> irq1: atkbd0 3466 0 >> irq6: fdc0 10 0 >> irq13: npx0 1 0 >> irq14: ata0 47 0 >> irq21: fxp1 20462527 8 >> irq28: bge0 3511765157 1444 >> irq29: bge1 3633124373 1494 >> irq30: aac0 1842472 0 >> cpu0: timer 566751007 233 >> Total 7733949060 3181 >> >> >> mull [~]$netstat -m >> 644/646/1290 mbufs in use (current/cache/total) >> 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) >> 0/5/4528 sfbufs in use (current/peak/max) >> 1447K/975K/2422K bytes allocated to network (current/cache/total) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> 0 calls to protocol drain routines >> >> >> >> Thank you, >> >> -- >> Marcos > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Feb 11 23:21:13 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3B6C16A420 for ; Sat, 11 Feb 2006 23:21:13 +0000 (GMT) (envelope-from dwmalone@maths.tcd.ie) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by mx1.FreeBSD.org (Postfix) with SMTP id 8519043D72 for ; Sat, 11 Feb 2006 23:21:11 +0000 (GMT) (envelope-from dwmalone@maths.tcd.ie) Received: from walton.maths.tcd.ie ([134.226.81.10] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 11 Feb 2006 23:21:11 +0000 (GMT) Date: Sat, 11 Feb 2006 23:21:11 +0000 From: David Malone To: Kris Kennaway Message-ID: <20060211232111.GA48677@walton.maths.tcd.ie> References: <20060116004438.GA27901@xor.obsecurity.org> <20060207054502.GA18560@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060207054502.GA18560@xor.obsecurity.org> User-Agent: Mutt/1.5.6i Sender: dwmalone@maths.tcd.ie Cc: net@FreeBSD.org Subject: Re: Changing time causes ipv6 panics X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2006 23:21:13 -0000 On Tue, Feb 07, 2006 at 12:45:02AM -0500, Kris Kennaway wrote: > Has anyone looked at this? This is on the TODO list for 6.1, so the > sooner it can be resolved the better. I wonder if a lot of these panics are related to races on the in6_ifaddr variable? Access to it is unsynchronised currently, and I think it has been causing various problems. See my followup to PR 93170. I'm trying to sort out some locking patches for it, I'll test them against your clock-setting test. David.