From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 13 11:52:06 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36C1C106564A for ; Sun, 13 Apr 2008 11:52:06 +0000 (UTC) (envelope-from barner@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 953CE8FC1F for ; Sun, 13 Apr 2008 11:52:05 +0000 (UTC) (envelope-from barner@gmx.de) Received: (qmail invoked by alias); 13 Apr 2008 11:25:22 -0000 Received: from ppp-88-217-30-40.dynamic.mnet-online.de (EHLO dose.local.invalid) [88.217.30.40] by mail.gmx.net (mp055) with SMTP; 13 Apr 2008 13:25:22 +0200 X-Authenticated: #147403 X-Provags-ID: V01U2FsdGVkX1/+elhIaSP+AiyZyG7ZR9ajRI5yIIZNJPnj60Cc+l 7oyyLQTHQ51jqJ Received: by dose.local.invalid (Postfix, from userid 1000) id 453C2C1DD; Sun, 13 Apr 2008 12:33:52 +0200 (CEST) Date: Sun, 13 Apr 2008 12:33:52 +0200 From: Simon Barner To: Mike Silbersack Message-ID: <20080413103351.GA1382@dose.local.invalid> References: <200803172156.37407.modelnine@modelnine.org> <20080317214510.G89676@odysseus.silby.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZPt4rx8FFjLCG7dd" Content-Disposition: inline In-Reply-To: <20080317214510.G89676@odysseus.silby.com> User-Agent: Mutt/1.5.16 (2007-06-09) X-Y-GMX-Trusted: 0 Cc: freebsd-hackers@freebsd.org, Heiko Wundram Subject: Re: valgrind on FreeBSD 7 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2008 11:52:06 -0000 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Mike Silbersack wrote: >=20 > On Mon, 17 Mar 2008, Heiko Wundram wrote: >=20 >> Hey all! >>=20 >> It's been some time now, and I just wanted to ping quietly whether there= 's=20 >> any >> chance to get (a preview of) the valgrind adapted for FreeBSD 7. >>=20 >> out is a futile task, I guess, as I didn't manage to find it in the=20 >> FreeBSD >> perforce repository the last time I looked (and was pointed there), but >> that's probably due to my personal stupidity in using the web-frontend f= or >> Perforce (which I find absolutely horrible). >>=20 >> Thanks for any pointer/hint/message! >>=20 >> --=20 >> Heiko Wundram >=20 > Here's a tarball of what's in perforce right now. I tried it a little bi= t,=20 > and it seemed to work for me. Make sure to install the kernel module! >=20 > http://www.silby.com/valgrind_freebsd_3.tar.gz >=20 > But don't send me questions about it - I'm not an expert on it, I'm just= =20 > the guy who grabbed it from perforce and found that it seems to work. :) Could you please provide me with the details, so I can update my (horribly outdated :( valgrind ports? --=20 Best regards / Viele Gr=FC=DFe, barner@FreeBSD.= org Simon Barner barner@gmx.de --ZPt4rx8FFjLCG7dd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFIAeGPCkn+/eutqCoRAkvPAJ976z00pxvOUJcZ3QOf9qtGuwMDwACgjYkF iCwgzsX5+H+BEDpjMItTevU= =h/yT -----END PGP SIGNATURE----- --ZPt4rx8FFjLCG7dd-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 14 19:42:21 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00ACB106566C for ; Mon, 14 Apr 2008 19:42:21 +0000 (UTC) (envelope-from volker@vwsoft.com) Received: from frontmail.ipactive.de (frontmail.maindns.de [85.214.95.103]) by mx1.freebsd.org (Postfix) with ESMTP id B459B8FC15 for ; Mon, 14 Apr 2008 19:42:20 +0000 (UTC) (envelope-from volker@vwsoft.com) Received: from mail.vtec.ipme.de (F72a1.f.ppp-pool.de [195.4.114.161]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by frontmail.ipactive.de (Postfix) with ESMTP id AB70E128844 for ; Mon, 14 Apr 2008 21:18:50 +0200 (CEST) Received: from cesar.sz.vwsoft.com (cesar.sz.vwsoft.com [192.168.16.3]) (Authenticated sender: relay@sz.vwsoft.com) by mail.vtec.ipme.de (Postfix) with ESMTP id F024E3F439 for ; Mon, 14 Apr 2008 21:19:40 +0200 (CEST) Message-ID: <4803AE07.9010106@vwsoft.com> Date: Mon, 14 Apr 2008 21:18:31 +0200 From: Volker User-Agent: Thunderbird 2.0.0.12 (X11/20080316) MIME-Version: 1.0 To: hackers@freebsd.org X-Enigmail-Version: 0.95.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit MailScanner-NULL-Check: 1208805594.01463@VJhq2YDwb9BxKA2CS2WCWw X-MailScanner-ID: F024E3F439.8440B X-VWSoft-MailScanner: Found to be clean X-MailScanner-From: volker@vwsoft.com X-ipactive-MailScanner-Information: Please contact the ISP for more information X-ipactive-MailScanner: Found to be clean X-ipactive-MailScanner-From: volker@vwsoft.com Cc: Subject: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2008 19:42:21 -0000 Hi hackers! I need to hunt kernel memory allocations done by a kld. Unfortunately the kld in question is a blob, no access to source code, so I need to check for kmem_alloc() + kmem_malloc() + kmem_free() and print information from there to the console screen. >From within these functions, I need to get the name (or an ID) of the module trying to allocate memory (and display that or - better - filter some allocations out). Thx, Volker From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 14 20:51:11 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12FBE1065679; Mon, 14 Apr 2008 20:51:11 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id AD2D68FC0A; Mon, 14 Apr 2008 20:51:10 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [202.108.54.204]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id CE42828449; Tue, 15 Apr 2008 04:51:09 +0800 (CST) Received: from localhost (tarsier.geekcn.org [202.108.54.204]) by tarsier.geekcn.org (Postfix) with ESMTP id 7AB65EB09CA; Tue, 15 Apr 2008 04:51:09 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([202.108.54.204]) by localhost (mail.geekcn.org [202.108.54.204]) (amavisd-new, port 10024) with ESMTP id N+lXXMgPrfVc; Tue, 15 Apr 2008 04:51:04 +0800 (CST) Received: from charlie.delphij.net (71.5.7.139.ptr.us.xo.net [71.5.7.139]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id E8B70EB47BF; Tue, 15 Apr 2008 04:51:02 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=NZOEpu2C+v6ncJzEVKpXbrTjzdjwaSAthwlxgbn8FGeRTEkVjwAtQBQkWTV01FfdH Q93DQdZWBBpJTOqH+5W+A== Message-ID: <4803C3B4.4000405@delphij.net> Date: Mon, 14 Apr 2008 13:51:00 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.12 (X11/20080312) MIME-Version: 1.0 To: Simon Barner References: <200803172156.37407.modelnine@modelnine.org> <20080317214510.G89676@odysseus.silby.com> <20080413103351.GA1382@dose.local.invalid> In-Reply-To: <20080413103351.GA1382@dose.local.invalid> X-Enigmail-Version: 0.95.6 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Heiko Wundram Subject: Re: valgrind on FreeBSD 7 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2008 20:51:11 -0000 Simon Barner wrote: > Mike Silbersack wrote: >> On Mon, 17 Mar 2008, Heiko Wundram wrote: >> >>> Hey all! >>> >>> It's been some time now, and I just wanted to ping quietly whether there's >>> any >>> chance to get (a preview of) the valgrind adapted for FreeBSD 7. >>> >>> out is a futile task, I guess, as I didn't manage to find it in the >>> FreeBSD >>> perforce repository the last time I looked (and was pointed there), but >>> that's probably due to my personal stupidity in using the web-frontend for >>> Perforce (which I find absolutely horrible). >>> >>> Thanks for any pointer/hint/message! >>> >>> -- >>> Heiko Wundram >> Here's a tarball of what's in perforce right now. I tried it a little bit, >> and it seemed to work for me. Make sure to install the kernel module! >> >> http://www.silby.com/valgrind_freebsd_3.tar.gz >> >> But don't send me questions about it - I'm not an expert on it, I'm just >> the guy who grabbed it from perforce and found that it seems to work. :) > > Could you please provide me with the details, so I can update my > (horribly outdated :( valgrind ports? It was available from p4 at: //depot/projects/valgrind/... Note that this version does not work on architectures other than i386. Cheers, -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 10:06:19 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 433AB106566C for ; Tue, 15 Apr 2008 10:06:19 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 19B318FC1F for ; Tue, 15 Apr 2008 10:06:19 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 99B2846B92; Tue, 15 Apr 2008 06:06:18 -0400 (EDT) Date: Tue, 15 Apr 2008 11:06:18 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Volker In-Reply-To: <4803AE07.9010106@vwsoft.com> Message-ID: <20080415105245.X14695@fledge.watson.org> References: <4803AE07.9010106@vwsoft.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: hackers@freebsd.org Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 10:06:19 -0000 On Mon, 14 Apr 2008, Volker wrote: > I need to hunt kernel memory allocations done by a kld. Unfortunately the > kld in question is a blob, no access to source code, so I need to check for > kmem_alloc() + kmem_malloc() + kmem_free() and print information from there > to the console screen. > >> From within these functions, I need to get the name (or an ID) of the > module trying to allocate memory (and display that or - better - filter some > allocations out). Once DTrace is in the tree, it will be possible to do this sort of thing automatically, but in the mean time, you might consider using KTR to trace those kernel allocator calls along with stack(9) to generate traces. I believe there's a variant of the CTR trace call that automatically collects stack traces. You can then filter it down to ones that pass through the kld of interest. BTW, does the module directly call those allocator functions? You might want to run nm on the module to look at what kernel functions it explictly depends on. While I'm aware of modules that do directly invoke the low-level allocators, most will prefer to use our higher level malloc(9) and uma(9) interfaces (for good reason). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 10:06:34 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9ACCF1065674 for ; Tue, 15 Apr 2008 10:06:34 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail18.syd.optusnet.com.au (mail18.syd.optusnet.com.au [211.29.132.199]) by mx1.freebsd.org (Postfix) with ESMTP id 1573C8FC28 for ; Tue, 15 Apr 2008 10:06:33 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c220-239-20-82.belrs4.nsw.optusnet.com.au [220.239.20.82]) by mail18.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m3FA6UrT019330 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 15 Apr 2008 20:06:31 +1000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.2/8.14.1) with ESMTP id m3FA6TZE032665; Tue, 15 Apr 2008 20:06:29 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.2/8.14.2/Submit) id m3FA6Sdi032664; Tue, 15 Apr 2008 20:06:28 +1000 (EST) (envelope-from peter) Date: Tue, 15 Apr 2008 20:06:28 +1000 From: Peter Jeremy To: Volker Message-ID: <20080415100628.GS73016@server.vk2pj.dyndns.org> References: <4803AE07.9010106@vwsoft.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4f28nU6agdXSinmL" Content-Disposition: inline In-Reply-To: <4803AE07.9010106@vwsoft.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.17 (2007-11-01) Cc: hackers@freebsd.org Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 10:06:34 -0000 --4f28nU6agdXSinmL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 14, 2008 at 09:18:31PM +0200, Volker wrote: >From within these functions, I need to get the name (or an ID) of the >module trying to allocate memory (and display that or - better - filter >some allocations out). About all I can suggest is working out the return address and finding which module that exists within. That doesn't sound particularly nice. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --4f28nU6agdXSinmL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkgEfiQACgkQ/opHv/APuIcYigCfRfdZtkh1oZw09JTiKFdx5D1U m0UAnRkSmz2vK4Vn2R24/7GSKeZPqvJu =0g8m -----END PGP SIGNATURE----- --4f28nU6agdXSinmL-- From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 11:43:11 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E885D106564A for ; Tue, 15 Apr 2008 11:43:11 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 94EB38FC16 for ; Tue, 15 Apr 2008 11:43:11 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id 50F59E4195B; Tue, 15 Apr 2008 11:43:08 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id 760E516FC4; Tue, 15 Apr 2008 13:41:23 +0200 (CEST) Date: Tue, 15 Apr 2008 13:41:23 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, hackers@freebsd.org Message-ID: <20080415114123.GG343@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, hackers@freebsd.org References: <4803AE07.9010106@vwsoft.com> <20080415100628.GS73016@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080415100628.GS73016@server.vk2pj.dyndns.org> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 11:43:12 -0000 On Tue, Apr 15, 2008 at 08:06:28PM +1000, Peter Jeremy wrote: > About all I can suggest is working out the return address and finding > which module that exists within. That doesn't sound particularly nice. Search gcc's info file for __builtin_return_address for the first part. Joerg From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 12:14:04 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D95C5106564A for ; Tue, 15 Apr 2008 12:14:04 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 95CF78FC0C for ; Tue, 15 Apr 2008 12:14:04 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id 50F59E4195B; Tue, 15 Apr 2008 11:43:08 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id 760E516FC4; Tue, 15 Apr 2008 13:41:23 +0200 (CEST) Date: Tue, 15 Apr 2008 13:41:23 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, hackers@freebsd.org Message-ID: <20080415114123.GG343@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, hackers@freebsd.org References: <4803AE07.9010106@vwsoft.com> <20080415100628.GS73016@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080415100628.GS73016@server.vk2pj.dyndns.org> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 12:14:04 -0000 On Tue, Apr 15, 2008 at 08:06:28PM +1000, Peter Jeremy wrote: > About all I can suggest is working out the return address and finding > which module that exists within. That doesn't sound particularly nice. Search gcc's info file for __builtin_return_address for the first part. Joerg From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 13:05:51 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2577D1065674 for ; Tue, 15 Apr 2008 13:05:51 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.177]) by mx1.freebsd.org (Postfix) with ESMTP id CABE28FC24 for ; Tue, 15 Apr 2008 13:05:50 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so2745452pyb.10 for ; Tue, 15 Apr 2008 06:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=rNG8HiRJ8xqps7GvWlVyClv39KynApF205YwtYBLYt0=; b=vXw7RVRBMyXOMQ7bzHfI9scDZex3NsDSoT+t/5eNQ6DFiwHbFzEE2LsdgSGiOwDx3ZzpFJcGr/lpmO1IPnCW3uXQpDmJuYHTMjOfEbeKEKl/CcpkbQTf3zRiWyBmIL8x04xm0P7BvK22jV673q6mTGJwSYkg04n8qJI9341xkMc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=iV9oWOdC3hYRDjD21EAygSAjISNWBdhV9nWkAK+2jkDMmaIQ7xz7USCR7kWuZADiEADvi3iA0y4fxOl5Ox/FJrVHvM0u49WcwdklJOzV+3jUkTa93SVFl0pTR6OsSNWzVrnaps0LDLyCCFBjueLIp3gYqYD+EBewaxzPGKLpaQQ= Received: by 10.35.60.15 with SMTP id n15mr12791284pyk.60.1208263037498; Tue, 15 Apr 2008 05:37:17 -0700 (PDT) Received: by 10.35.38.6 with HTTP; Tue, 15 Apr 2008 05:37:17 -0700 (PDT) Message-ID: Date: Tue, 15 Apr 2008 14:37:17 +0200 From: "Antoine Brodin" Sender: antoine.brodin.freebsd@gmail.com To: Volker In-Reply-To: <4803AE07.9010106@vwsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4803AE07.9010106@vwsoft.com> X-Google-Sender-Auth: b2e1ae203a43fc68 Cc: hackers@freebsd.org Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 13:05:51 -0000 On Mon, Apr 14, 2008 at 9:18 PM, Volker wrote: > Hi hackers! > > I need to hunt kernel memory allocations done by a kld. Unfortunately > the kld in question is a blob, no access to source code, so I need to > check for kmem_alloc() + kmem_malloc() + kmem_free() and print > information from there to the console screen. Hi Volker, You could patch your kld (hexadecimal editor or something like that) to call wrappers around the interesting functions that print the arguments. I hope this helps. Cheers, Antoine > >From within these functions, I need to get the name (or an ID) of the > module trying to allocate memory (and display that or - better - filter > some allocations out). From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 15 14:27:34 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9760A1065680 for ; Tue, 15 Apr 2008 14:27:34 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.176]) by mx1.freebsd.org (Postfix) with ESMTP id 4290D8FC23 for ; Tue, 15 Apr 2008 14:27:34 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so2782596pyb.10 for ; Tue, 15 Apr 2008 07:27:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=biEnFfTAO2jfbM2qHhbKO7RsU7eAhhzG3/gwrkQTCYs=; b=xH1V98Q0BIR9GttQCFPS4TZiHee5XraNdhzKo44/0rpCJfl+fsb7sq7hAzcmMfy0ifFyecQgaZe/tU8KvVFBZ7lyOOrKsdKmAbZYSv3trohgRv4NaSLEUnYC7so6joF17s3UZ4HJsau6rGqI32lrR89nXZYDXyXpAhk0raCDLf4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=f84Jk95cwYGjDvqqCpAbuQzfuC4bh+lju58Ezu035KvMv42AyENERML6usb6aU+Jo8vxASm1Itc7wBa/467c4p6BHk0YjUFspUiqwT3Q4fIAkHWaaIyS2Jmw+Wq4r+sO3Wa011G/qSd1LbL91fFd7+pgtrTEps8iVDkrO543Ofw= Received: by 10.35.86.19 with SMTP id o19mr13029218pyl.43.1208269652495; Tue, 15 Apr 2008 07:27:32 -0700 (PDT) Received: by 10.35.38.6 with HTTP; Tue, 15 Apr 2008 07:27:32 -0700 (PDT) Message-ID: Date: Tue, 15 Apr 2008 16:27:32 +0200 From: "Antoine Brodin" Sender: antoine.brodin.freebsd@gmail.com To: "Antoine Brodin" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4803AE07.9010106@vwsoft.com> X-Google-Sender-Auth: 1cfa39019ef15104 Cc: Volker , hackers@freebsd.org Subject: Re: "visualize" kernel memory allocations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2008 14:27:34 -0000 On Tue, Apr 15, 2008 at 2:37 PM, Antoine Brodin wrote: > On Mon, Apr 14, 2008 at 9:18 PM, Volker wrote: > > Hi hackers! > > > > I need to hunt kernel memory allocations done by a kld. Unfortunately > > the kld in question is a blob, no access to source code, so I need to > > check for kmem_alloc() + kmem_malloc() + kmem_free() and print > > information from there to the console screen. > > Hi Volker, > > You could patch your kld (hexadecimal editor or something like that) > to call wrappers around the interesting functions that print the > arguments. "objcopy --redefine-syms" seems to be easier to use than a hexadecimal editor. Cheers, Antoine From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 00:24:03 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B9E8106564A for ; Wed, 16 Apr 2008 00:24:03 +0000 (UTC) (envelope-from v.velox@vvelox.net) Received: from vulpes.vvelox.net (vulpes.vvelox.NET [74.200.198.26]) by mx1.freebsd.org (Postfix) with ESMTP id F359B8FC0C for ; Wed, 16 Apr 2008 00:24:02 +0000 (UTC) (envelope-from v.velox@vvelox.net) Received: from vixen42 (c-68-51-74-1.hsd1.il.comcast.net [68.51.74.1]) (Authenticated sender: v.velox) by vulpes.vvelox.net (Postfix) with ESMTP id F12A7B82E for ; Tue, 15 Apr 2008 19:04:29 -0500 (CDT) Date: Tue, 15 Apr 2008 19:05:14 -0500 From: "Zane C.B." To: hackers@freebsd.org Message-ID: <20080415190514.39abab53@vixen42> X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.9; i386-portbld-freebsd6.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: NSS question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 00:24:03 -0000 The only documentation I can really find for writing NSS stuff is the stuff for glibc. Any differences or the like I need to be aware of in regards to FreeBSD? http://www.gnu.org/software/libtool/manual/libc/Name-Service-Switch.html#Name-Service-Switch From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 11:04:25 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25F7B106566B for ; Wed, 16 Apr 2008 11:04:25 +0000 (UTC) (envelope-from miauris@gmail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.172]) by mx1.freebsd.org (Postfix) with ESMTP id F19148FC19 for ; Wed, 16 Apr 2008 11:04:24 +0000 (UTC) (envelope-from miauris@gmail.com) Received: by wf-out-1314.google.com with SMTP id 25so2415487wfa.7 for ; Wed, 16 Apr 2008 04:04:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=58WnAALxY9rBuIDgPwvHd/erLHBcAieCMBS8OJd2lNo=; b=jHMhb2AszZi+nWxBsZ7RY3PNSOklQtgzO2B4VsSSpCetDrfZWJaGUf3uXn7zpgL/xSkIcdAlidQi6z7UGhcJPKH6g2plFQXUeahc5KWcEAMnC6lXRNx17Nk+uf5dfQniUrU+nEqMa/ZgMAs0UqyPiW5R9sPEiFIan8ORJGIyDOY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=rDMiara0tqxCHwQzwJpEJ/JNycbjHv0DeGw05/Yv2o7SyMYUMUDt2XgUMUR0vJeWu3GTbhJArTZLLzu1hiQYC/MszrSdJhYxjo/ZmAz37IkEq0iFGSVNnr6fa0Mc1Aj2ciiCwGl3wyVRqKTgNohpAMxb9WgnIsTK6047KSSalnU= Received: by 10.142.53.6 with SMTP id b6mr1737281wfa.289.1208342247175; Wed, 16 Apr 2008 03:37:27 -0700 (PDT) Received: by 10.142.131.21 with HTTP; Wed, 16 Apr 2008 03:37:26 -0700 (PDT) Message-ID: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> Date: Wed, 16 Apr 2008 13:37:26 +0300 From: "M.S. Motanu" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Multipath routing - failover version X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 11:04:25 -0000 I've ported the multipath routing patch by Ed Tanzer > for FreeBSD 4.8-STABLE to FreeBSD 6.1-RELEASE. The patch modifies the kernel and the userland programs netstat and route so that for the same destination it is possible to specify two or more different gateways (paths). Switching between different paths is done by the kernel based on the link state of the interface associated with the gateway. This way when can achieve a level of redundancy at the link level (this is not a routing protocol!). The original patch did not have failover in mind, it addressed the problem of load balancing. But the same results can be achieved with this patch with minor changes to the code. The patch files and installation instructions can be downloaded from:http://miauris.freehostia.com/mpath/mpath.tar.gz Comments are welcomed! From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 16:29:21 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2D8C1065670 for ; Wed, 16 Apr 2008 16:29:21 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by mx1.freebsd.org (Postfix) with ESMTP id 7A6018FC29 for ; Wed, 16 Apr 2008 16:29:21 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: by yw-out-2324.google.com with SMTP id 2so1406875ywt.13 for ; Wed, 16 Apr 2008 09:29:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition; bh=cbeNwuoe5GWVXOURQKZVNT8I/2XRz4LBqBuN304PsOI=; b=E875QjB6tWXW0+CeEWuohnlR2k5uTQZdEdbp13sIyIkmqMiv8VgKmJge9ZEtZGxXOfJIzyeQZUn40xMg6okUxzTf/L081eAwlh2lVo9042Kr4Sat5PQ8NuWUssZD9SZ4oTrn+8+zMMK5NSihLfMu/JZYlxFkxq8ds9vWhKzUhkU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition; b=dPuiWP3r7NmJPEdcBFsmbijz4s8uuEsprBI4jbNMjpUCFcUH9ci/bFblTaHQTklN1VCvzFDxnkdrh9bMO/PDJEOmLz+i1TPtjX593Rc2GDSVaZ/X8qNxWi6V3BI+lf4s/slwAcPbXYvCEBpOteqmcFhFjgf2W3ElziP6jMYUW2o= Received: by 10.150.49.15 with SMTP id w15mr361375ybw.101.1208363359508; Wed, 16 Apr 2008 09:29:19 -0700 (PDT) Received: by 10.151.15.21 with HTTP; Wed, 16 Apr 2008 09:29:19 -0700 (PDT) Message-ID: <3c0b01820804160929i76cc04fdy975929e2a04c0368@mail.gmail.com> Date: Wed, 16 Apr 2008 12:29:19 -0400 From: "Alexander Sack" To: freebsd-drivers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org Subject: bge dropping packets issue X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 16:29:22 -0000 Hello: Sorry for cross posting but this seems to be both a driver and network/kernel issue so I figure I actually thought all lists seemed appropriate. I'm investigating an issue we are seeing with 6.1-RELEASE and the bge driver dropping packets sporadically at 100MBps speed. The machine is a 2-way Intel dual-core running 64-bit FreeBSD-6.1 Release with SMP/8GB RAM. I would post dmesg but currently I'm running a test and has a lot of instrumentation in it. Anyway, what I'm seeing with a SmartBit traffic generator connected to 4 bge cards (all BCOM_DEVICEID_BCM5704C) is sporadic packet drops as recorded by the firmware in its statistics structure (as pulled out by bge_tick()), i.e. this isn't malloc starvation of allocating mbuf clusters, etc. The firmware seems to just drop packets occasionally (depending on workload). Its get mainly aggravated when heavy disk I/O occurs from generating a network report which entails gzip'ing a very large dumpfile in /tmp and then anonymously ftping it via another interface (em). DEVICE_POLLING is being used: # sysctl -a | grep kern.polling kern.polling.idlepoll_sleeping: 1 kern.polling.stalled: 3 kern.polling.suspect: 1023 kern.polling.phase: 0 kern.polling.enable: 1 kern.polling.handlers: 6 kern.polling.residual_burst: 0 kern.polling.pending_polls: 0 kern.polling.lost_polls: 24436 kern.polling.short_ticks: 592 kern.polling.reg_frac: 20 kern.polling.user_frac: 50 kern.polling.idle_poll: 0 kern.polling.each_burst: 32 kern.polling.burst_max: 1000 kern.polling.burst: 1000 After looking at the driver for a bit, I believe the issue maybe from RX chain starvation which causes the firmware to drop packets when bge_rxeof() can not keep up. The driver uses a global locking scheme which may contribute to some of these robustness issues (this is a generalization on my part without hard facts so take it with a grain of salt, I just notice things like bge_tick() being called every cycle and competing with the ISR when it may not have too or may not have too for its entire duration, updating stats for example). My main question is currently the RX chain slots are set to a global define BGE_SSLOTS (if_bgedevreg.h) which is 256. Technically this card I believe can do up to 512 slots and the comment above said these are tunable yet not exposed via SYSCTL. Does anyone know why its not 512 by default? Is there any harm in setting it to 512 instead of 256? Why not make it tunable (512 as max)? I've increased the SSLOTS to 512 so there are more RX chain slots available (as I currently understand it, I don't have specs) and the kern.polling.each_burst to 150 (max) in an effort to try to keep the BGE driver in bge_rxeof() and to experiment a bit! This is the first exposure to this code so be gentle! :D! Has anyone seen this problem before with bge? Am I barking up the wrong tree with my initial investigation? Does anyone know if its even possible to achieve 100% packet capture with bge at its supported speeds (10/100/1000)? Thanks! -aps From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 17:41:14 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46804106564A for ; Wed, 16 Apr 2008 17:41:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outB.internet-mail-service.net (outb.internet-mail-service.net [216.240.47.225]) by mx1.freebsd.org (Postfix) with ESMTP id 350A68FC27 for ; Wed, 16 Apr 2008 17:41:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 16 Apr 2008 21:14:15 -0700 Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 986212D600D; Wed, 16 Apr 2008 10:41:13 -0700 (PDT) Message-ID: <48063A3D.4070800@elischer.org> Date: Wed, 16 Apr 2008 10:41:17 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.12 (Macintosh/20080213) MIME-Version: 1.0 To: "M.S. Motanu" References: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> In-Reply-To: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Qing Li Subject: Re: Multipath routing - failover version X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 17:41:14 -0000 M.S. Motanu wrote: > I've ported the multipath routing patch by Ed Tanzer > > > for FreeBSD 4.8-STABLE to FreeBSD 6.1-RELEASE. multipath routing was just committed to -current and will be merged back possibly. it may be worth seeing what there is in common, and if some features need to be cross-pollinated. > > The patch modifies the kernel and the userland programs netstat and route > so that for the same destination it is possible to specify two or more > different gateways (paths). > > Switching between different paths is done by the kernel based on the link > state of the interface associated with the gateway. This way when can > achieve a level of redundancy at the link level (this is not a routing > protocol!). > > The original patch did not have failover in mind, it addressed the problem > of load balancing. But the same results can be achieved with this patch > with minor changes to the code. > > The patch files and installation instructions can be downloaded > from:http://miauris.freehostia.com/mpath/mpath.tar.gz > > Comments are welcomed! > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 18:00:28 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C3841065670 for ; Wed, 16 Apr 2008 18:00:28 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from whisker.bluecoat.com (whisker.bluecoat.com [216.52.23.28]) by mx1.freebsd.org (Postfix) with ESMTP id E6EEC8FC13 for ; Wed, 16 Apr 2008 18:00:27 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from bcs-mail03.internal.cacheflow.com ([10.2.2.95]) by whisker.bluecoat.com (8.14.2/8.14.2) with ESMTP id m3GHchG1023927; Wed, 16 Apr 2008 10:38:43 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Wed, 16 Apr 2008 10:38:37 -0700 Message-ID: In-Reply-To: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Multipath routing - failover version Thread-Index: Acifsbqwo5QwhbG3SPqfAyPlWFKhSAANC0Ug References: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> From: "Li, Qing" To: "M.S. Motanu" , Cc: Subject: RE: Multipath routing - failover version X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 18:00:28 -0000 Hi, I recently incorporated multipath support into -CURRENT, for the upcoming 8.0. This patch originated from the KAME project and builds on the existing routing data structures and infrastructure. As a result I did not have to modify the userland programs, however, I think netstat can use some tweaking in its output.=20 I am in the process of incorporating additional functionalities such as allowing for preference setting, and performing active=20 health-check on the gateways and updating the routes accordingly. >=20 > Switching between different paths is done by the kernel based=20 > on the link state of the interface associated with the=20 > gateway. This way when can achieve a level of redundancy at=20 > the link level (this is not a routing protocol!). >=20 Hmm... in the current code if_unroute() would remove the interface route when the interface is down. -- Qing From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 18:20:35 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A481E1065678 for ; Wed, 16 Apr 2008 18:20:35 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id 4DB418FC14 for ; Wed, 16 Apr 2008 18:20:35 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: by yw-out-2324.google.com with SMTP id 2so1436582ywt.13 for ; Wed, 16 Apr 2008 11:20:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=gWce5972bEgiQU0Qe0OxMbA8rvfNAm3oVFyxzVaPFIs=; b=iAUmo1iSjg/sFIwharPCZkD4hVnlne4H5qYV9arouVxiFWL/Ohe/PWCgl/r+eshjoPzaaYoNRwsT1wgVJ1KCuRpbRmmWk0O7xpCYhx1AetU7DHoX9rUPqEjWjbY6IiJ+lTDrLNlIHXhLp0apTJzaDIs+qsUoPNf0AQH4d0BxJ54= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=UKKeh78Hy0q3aWAULJgqcdI/fvsKtw/151AmO4oV/EvssU+tcxhdpLXZ9onS0Bu+fE/ByyRznmIsZP3w8YsQA3ewwLcsX8GDmE2OLwYy1Pi1k2lG0w2wX5Y2lvp0Mst4YnbKQdtYSIEU9/GTCWMIUqrKo+70CKn/89J5H9OcYuQ= Received: by 10.151.85.20 with SMTP id n20mr553514ybl.28.1208370027256; Wed, 16 Apr 2008 11:20:27 -0700 (PDT) Received: by 10.151.15.21 with HTTP; Wed, 16 Apr 2008 11:20:27 -0700 (PDT) Message-ID: <3c0b01820804161120udb54ab3tea4bf7baade0061f@mail.gmail.com> Date: Wed, 16 Apr 2008 14:20:27 -0400 From: "Alexander Sack" To: Dieter In-Reply-To: <200804161732.RAA23071@sopwith.solgatos.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3c0b01820804160929i76cc04fdy975929e2a04c0368@mail.gmail.com> <200804161732.RAA23071@sopwith.solgatos.com> Cc: freebsd-net@freebsd.org, freebsd-drivers@freebsd.org, Jung-uk Kim , freebsd-hackers@freebsd.org Subject: Re: bge dropping packets issue X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 18:20:35 -0000 Dieter: Thanks, at 20Mbps! That's pretty aweful. JK: Thanks again. Wow, I searched the list and didn't see much discussion with respect to bge and packet loss! I will try the rest of that patch including pushing the TCP receive buffer up (though I don't think that's going to help in this case). The above is based on just looking at code.... I guess some follow-up questions would be: 1) Why isn't BGE_SSLOTS tunable (to a point)? Why can't that be added the driver? I noticed that CURRENT has added a lot more SYSCTL information. Moreover it seems the Linux driver can set it up to 1024. 2) bge_tick() uses the same global mutex for its callout as the rest of the driver. Moreover, it really doesn't have to hold it while updating statistics, they are reads of volatile registers anyway (blocking the ISR doesn't prevent the firmware from updating its stat struct). Would there be any interest in using a separate mutex for the callout itself and then just hold the lock for the other small calls (bge_asf_driver_up(), bge_watchdog())? I'm experimenting with right now just dropping the BGE mutex around the bge_stats_update() calls to give more time to bge_rxeof() to drain rx_bd's. I admit that bge_tick doesn't do a whole lot but it seems this driver is very sensitive to resource starvation and I'm trying to get the BGE driver to drain as much rx_bd's as possible to avoid drops due to the firmware having no place to put them! 3) How does interrupt non-DEVICE_POLLING perform? Thanks guys! -aps On Wed, Apr 16, 2008 at 5:32 AM, Dieter wrote: > > I'm investigating an issue we are seeing with 6.1-RELEASE and the bge > > driver dropping packets sporadically at 100MBps speed. > > > > Its get mainly aggravated when heavy disk I/O occurs > > > > Has anyone seen this problem before with bge? Am I barking up the > > wrong tree with my initial investigation? Does anyone know if its > > even possible to achieve 100% packet capture with bge at its supported > > speeds (10/100/1000)? > > I had a similar problem with bge and 6.0-RELEASE. Bge works great for > me in 6.2-RELEASE. So far 7.0-RELEASE looks good as well (bge-wise, > I do have unresolved issues with 7). > > My app is TCP, cranking the TCP receive buffer way up helped, as did > turning off Nagle. > > My bge is 1000, but connected at 100 since that is what the other end is. > I saw problems at less than 20 Mbps. > > There is still a problem that other drivers can lock out bge for too long. > For example kern/118093: firewire bus reset. > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 18:07:11 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E1D9106566C; Wed, 16 Apr 2008 18:07:11 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from parsely.rain.com (parsely.rain.com [199.26.172.196]) by mx1.freebsd.org (Postfix) with ESMTP id 9ABD98FC1D; Wed, 16 Apr 2008 18:07:10 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from sopwith.solgatos.com (uucp@localhost) by parsely.rain.com (8.11.4/8.11.4) with UUCP id m3GHfDl56312; Wed, 16 Apr 2008 10:41:13 -0700 (PDT) (envelope-from freebsd@sopwith.solgatos.com) Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id RAA23071; Wed, 16 Apr 2008 17:32:50 GMT Message-Id: <200804161732.RAA23071@sopwith.solgatos.com> To: "Alexander Sack" In-reply-to: Your message of "Wed, 16 Apr 2008 12:29:19 EDT." <3c0b01820804160929i76cc04fdy975929e2a04c0368@mail.gmail.com> Date: Wed, 16 Apr 2008 10:32:50 +0100 From: Dieter X-Mailman-Approved-At: Wed, 16 Apr 2008 18:25:47 +0000 Cc: freebsd-net@freebsd.org, freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: bge dropping packets issue X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 18:07:11 -0000 > I'm investigating an issue we are seeing with 6.1-RELEASE and the bge > driver dropping packets sporadically at 100MBps speed. > Its get mainly aggravated when heavy disk I/O occurs > Has anyone seen this problem before with bge? Am I barking up the > wrong tree with my initial investigation? Does anyone know if its > even possible to achieve 100% packet capture with bge at its supported > speeds (10/100/1000)? I had a similar problem with bge and 6.0-RELEASE. Bge works great for me in 6.2-RELEASE. So far 7.0-RELEASE looks good as well (bge-wise, I do have unresolved issues with 7). My app is TCP, cranking the TCP receive buffer way up helped, as did turning off Nagle. My bge is 1000, but connected at 100 since that is what the other end is. I saw problems at less than 20 Mbps. There is still a problem that other drivers can lock out bge for too long. For example kern/118093: firewire bus reset. From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 19:11:01 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3AF31065676 for ; Wed, 16 Apr 2008 19:11:01 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 717CC8FC29 for ; Wed, 16 Apr 2008 19:11:01 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so3525356pyb.10 for ; Wed, 16 Apr 2008 12:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=3auZE6egHMzl9Z9IOKVm+q5SXaqpBEI2FoOWLPlg+Xc=; b=lW+suftq3h1ml0SxYfmIUyfpoOyq59V+qP4+ftV6mmpU7EkjcJ9FkLjDJ08MJSuzDjI487nuA2zcykpvQykDvSlZQaYlshgY+XeeftEYPWM3VXpTPstRRmx54DxSJEG9aOOAm1Au/Yfud+zrI/aD1z99Ap43L5lPY2NoOfTvqp4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=q6MMdcKvJ5+NVoZcSb9F1+aU8oPakUZ1GNhFeIQ+/IoaOVlVhshZOoIvBRpyP03GYGcWjNCQjqhntuxI5HGvLvjLuM7P1UO6TYNXSaxFgeeJIZhz8FeymmqspFdrtUIDi4q9mt89SxTWdptZjfl3EV2KYbXfDFlzW43p5PA44ZA= Received: by 10.141.136.19 with SMTP id o19mr187769rvn.250.1208371469486; Wed, 16 Apr 2008 11:44:29 -0700 (PDT) Received: by 10.140.135.3 with HTTP; Wed, 16 Apr 2008 11:44:29 -0700 (PDT) Message-ID: <9a542da30804161144s3f90b9daq44966f5a449273bb@mail.gmail.com> Date: Wed, 16 Apr 2008 14:44:29 -0400 From: "=?ISO-8859-1?Q?Ermal_Lu=E7i?=" To: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: [patch] ng_iface(4) and ports/net/mpd4 allow renaming of interfaces. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 19:11:01 -0000 Hello, the patches inlined give the ng_iface(4) and the mpd4 port the ability to rename its interfaces. IE if you create a new pppoe connection instead of the line new -i ng0 pppoe pppoe you can enter new -i pppoe0 pppoe pppoe so when mpd starts it will create an ngX interface with a ngX: hook after that it will rename ngX to pppoe0 and the corresponding hook to pppoe0: The ng_iface(4) patch adds a new message NGM_IFACE_SET_IFNAME: usable through ngctl msg $path: setifname $name it is just a copy of the ioctl in if.c SIFNAME without the routing adevertising cause mostly you will do renaming before bringing the interface up. Alghotu you are not allowed to rename it after the interface is up. I wonder if it can be polished and integrated in future versions of FreeBSD?! If it is needed to patch even mpd5 i will do even that. Regards, Ermal --- src/ngfunc.c.orig 2008-04-16 13:29:08.000000000 -0400 +++ src/ngfunc.c 2008-04-16 13:29:16.000000000 -0400 @@ -249,6 +249,7 @@ struct ng_mesg reply; } u; char path[NG_PATHLEN + 1]; +#if 0 char *eptr; int ifnum; @@ -258,9 +259,10 @@ ifnum = (int)strtoul(ifname + strlen(NG_IFACE_IFACE_NAME), &eptr, 10); if (ifnum < 0 || *eptr != '\0') return(-1); +#endif /* See if interface exists */ - snprintf(path, sizeof(path), "%s%d:", NG_IFACE_IFACE_NAME, ifnum); + snprintf(path, sizeof(path), "%s:", ifname); if (NgSendMsg(b->csock, path, NGM_GENERIC_COOKIE, NGM_NODEINFO, NULL, 0) < 0) return(0); if (NgRecvMsg(b->csock, &u.reply, sizeof(u), NULL) < 0) { @@ -270,7 +272,7 @@ /* It exists */ if (buf != NULL) - snprintf(buf, max, "%s%d", NG_IFACE_IFACE_NAME, ifnum); + snprintf(buf, max, "%s", ifname); return(1); } @@ -294,30 +296,10 @@ struct nodeinfo *const ni = (struct nodeinfo *)(void *)u.reply.data; struct ngm_rmhook rm; struct ngm_mkpeer mp; + struct ngm_name nm; + char path[NG_PATHLEN + 1]; int rtn = 0; - /* If ifname is not null, create interfaces until it gets created */ - if (ifname != NULL) { - int count; - - for (count = 0; count < MAX_IFACE_CREATE; count++) { - switch (NgFuncIfaceExists(b, ifname, buf, max)) { - case 1: /* ok now it exists */ - return(0); - case 0: /* nope, create another one */ - NgFuncCreateIface(b, NULL, NULL, 0); - break; - case -1: /* something weird happened */ - return(-1); - default: - assert(0); - } - } - Log(LG_ERR, ("[%s] created %d interfaces, that's too many!", - b->name, count)); - return(-1); - } - /* Create iface node (as a temporary peer of the socket node) */ snprintf(mp.type, sizeof(mp.type), "%s", NG_IFACE_NODE_TYPE); snprintf(mp.ourhook, sizeof(mp.ourhook), "%s", TEMPHOOK); @@ -328,7 +310,6 @@ b->name, NG_IFACE_NODE_TYPE, ".", mp.ourhook, strerror(errno))); return(-1); } - /* Get the new node's name */ if (NgSendMsg(b->csock, TEMPHOOK, NGM_GENERIC_COOKIE, NGM_NODEINFO, NULL, 0) < 0) { @@ -342,6 +323,28 @@ rtn = -1; goto done; } + +if (ifname != NULL) { + /* Set the new node's name */ + bzero(path, sizeof(path)); + snprintf(path, sizeof(path), "%s:", ni->name); +snprintf(nm.name, sizeof(nm.name), "%s", ifname); + if (NgSendMsg(b->csock, path, + NGM_IFACE_COOKIE, NGM_IFACE_SET_IFNAME, nm.name, sizeof(nm.name)) < 0) { + Log(LG_ERR, ("[%s] %s: %s", b->name, "NGM_NODEINFO", strerror(errno))); + rtn = -1; + goto done; + } + + /* Set the new node's name */ + if (NgSendMsg(b->csock, path, + NGM_GENERIC_COOKIE, NGM_NAME, &nm, sizeof(nm)) < 0) { + Log(LG_ERR, ("[%s] %s: %s", b->name, "NGM_NODEINFO", strerror(errno))); + rtn = -1; + goto done; + } + snprintf(buf, max, "%s", ifname); +} else snprintf(buf, max, "%s", ni->name); done: @@ -355,7 +358,7 @@ } /* Done */ - return(rtn); + return (rtn); } /* Index: ng_iface.c =================================================================== RCS file: /home/ncvs/src/sys/netgraph/ng_iface.c,v retrieving revision 1.47 diff -u -r1.47 ng_iface.c --- ng_iface.c 2 Jun 2006 23:14:40 -0000 1.47 +++ ng_iface.c 16 Apr 2008 17:03:47 -0000 @@ -69,9 +69,11 @@ #include #include #include +#include #include #include +#include #include #include @@ -163,6 +165,13 @@ }, { NGM_IFACE_COOKIE, + NGM_IFACE_SET_IFNAME, + "setifname", + &ng_parse_string_type, + NULL + }, + { + NGM_IFACE_COOKIE, NGM_IFACE_POINT2POINT, "point2point", NULL, @@ -587,6 +596,10 @@ struct ng_mesg *resp = NULL; int error = 0; struct ng_mesg *msg; + char *new_name; + size_t namelen, onamelen; + struct sockaddr_dl *sdl = NULL; + struct ifaddr *ifa = NULL; NGI_GET_MSG(item, msg); switch (msg->header.typecookie) { @@ -601,6 +614,49 @@ strlcpy(resp->data, ifp->if_xname, IFNAMSIZ); break; + case NGM_IFACE_SET_IFNAME: + + new_name = (char *)msg->data; + /* Announce the departure of the interface. */ + //new_name[strlen(new_name)] = '\0'; + + /* Deny request if interface is UP */ + if ((ifp->if_flags & IFF_UP) != 0) { + error = EBUSY; + break; + } + + //rt_ifannouncemsg(ifp, IFAN_DEPARTURE); + EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); + + strlcpy(ifp->if_xname, new_name, sizeof(ifp->if_xname)); + ifa = ifp->if_addr; + IFA_LOCK(ifa); + sdl = (struct sockaddr_dl *)ifa->ifa_addr; + namelen = strlen(new_name) + 1; + onamelen = sdl->sdl_nlen; + /* + * Move the address if needed. This is safe because we + * allocate space for a name of length IFNAMSIZ when we + * create this in if_attach(). + */ + if (namelen != onamelen) { + bcopy(sdl->sdl_data + onamelen, + sdl->sdl_data + namelen, sdl->sdl_alen); + } + bcopy(new_name, sdl->sdl_data, namelen); + sdl->sdl_nlen = namelen; + sdl = (struct sockaddr_dl *)ifa->ifa_netmask; + bzero(sdl->sdl_data, onamelen); + while (namelen != 0) + sdl->sdl_data[--namelen] = 0xff; + IFA_UNLOCK(ifa); + + EVENTHANDLER_INVOKE(ifnet_arrival_event, ifp); + /* Announce the return of the interface. */ + //rt_ifannouncemsg(ifp, IFAN_ARRIVAL); + break; + case NGM_IFACE_POINT2POINT: case NGM_IFACE_BROADCAST: { Index: ng_iface.h =================================================================== RCS file: /home/ncvs/src/sys/netgraph/ng_iface.h,v retrieving revision 1.9 diff -u -r1.9 ng_iface.h --- ng_iface.h 13 Feb 2005 16:36:41 -0000 1.9 +++ ng_iface.h 16 Apr 2008 17:03:48 -0000 @@ -70,6 +70,7 @@ NGM_IFACE_POINT2POINT, NGM_IFACE_BROADCAST, NGM_IFACE_GET_IFINDEX, + NGM_IFACE_SET_IFNAME, }; #endif /* _NETGRAPH_NG_IFACE_H_ */ From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 19:33:34 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F2AA106564A for ; Wed, 16 Apr 2008 19:33:34 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id B6F528FC15 for ; Wed, 16 Apr 2008 19:33:33 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by yw-out-2324.google.com with SMTP id 2so1455667ywt.13 for ; Wed, 16 Apr 2008 12:33:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=inHJo33ufiNHgchGwTSiDdVzUppDmDFZbB9MINwm5bg=; b=w+NSQn8WjDmc+Mmg7Ne3/Qmzv5QXQyFQ3e9yhNAI1YKssbGvaFiJrYv0v7JNL0HcD/cKBibLGtU/1kpgJAhQP+j3POSYn2kDQ38n+LxmESmq8ClyNTl87XuBaTG0v9sWINA3tuY8erkwxe7jDMhZhC2rEfCXZh77URFDbUNqHGk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=Wl+wLlCVuwJgVj5RHDXPHE6vPXkoRaYrI9+WqGhu0oZVlI8lk0KzFCXQ0SwcGWqI3OYGfQjYU9EXZJeFTwpJFlhIuDzRRVgfVhWhqP8lQbDhOpFzSCOtEk6OLwKCqoSJtUtU50SrUBoaRzCa/PiVo2XbiLOIrPmfgfrytKH3Rfw= Received: by 10.150.135.2 with SMTP id i2mr556825ybd.197.1208372789674; Wed, 16 Apr 2008 12:06:29 -0700 (PDT) Received: by 10.150.156.14 with HTTP; Wed, 16 Apr 2008 12:06:29 -0700 (PDT) Message-ID: <5f67a8c40804161206r4273cbb2sb42f478ca0d3d4fc@mail.gmail.com> Date: Wed, 16 Apr 2008 15:06:29 -0400 From: "Zaphod Beeblebrox" To: "Li, Qing" In-Reply-To: MIME-Version: 1.0 References: <1dd0a33d0804160337o3090ac08g4a2cbc3be0d58b19@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "M.S. Motanu" , freebsd-hackers@freebsd.org Subject: Re: Multipath routing - failover version X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 19:33:34 -0000 On Wed, Apr 16, 2008 at 1:38 PM, Li, Qing wrote: > I recently incorporated multipath support into -CURRENT, > for the upcoming 8.0. This patch originated from the KAME > The most annoying feature of the -stable routing is that you cannot add an IP to an interface if the route already exists. An example would be our datacentre where two routers both have vlan interfaces on the same subnet and they run OSPF (quagga). If machine you add the IP to the vlan on machine A, you need to stop OSPF on machine B before it will allow you to add the IP on it's vlan. Operationally, this is suboptimal. Is this gnat fixed? Obviously, also, a machine should prefer it's own interface routes to routes provided by external routing protocols. Is anyone involved in this pushing these changes into quagga? From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 22:32:22 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CE191065673 for ; Wed, 16 Apr 2008 22:32:22 +0000 (UTC) (envelope-from ravi.murty@intel.com) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx1.freebsd.org (Postfix) with ESMTP id 6A2E18FC24 for ; Wed, 16 Apr 2008 22:32:22 +0000 (UTC) (envelope-from ravi.murty@intel.com) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 16 Apr 2008 15:31:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.25,666,1199692800"; d="scan'208,217";a="271297372" Received: from orsmsx334.amr.corp.intel.com (HELO orsmsx334.jf.intel.com) ([10.22.226.45]) by orsmga002.jf.intel.com with ESMTP; 16 Apr 2008 15:32:19 -0700 Received: from orsmsx416.amr.corp.intel.com ([10.22.226.46]) by orsmsx334.jf.intel.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 Apr 2008 15:32:21 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 16 Apr 2008 15:32:20 -0700 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: md_spinlock_count? Thread-Index: AcigEbwGeWcf+4KGRVqcV42pRHFz4Q== From: "Murty, Ravi" To: X-OriginalArrivalTime: 16 Apr 2008 22:32:21.0835 (UTC) FILETIME=[BC8DB1B0:01C8A011] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: md_spinlock_count? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 22:32:22 -0000 Hello All, =20 I was looking at the code that creates a new process (fork) with a single thread coming out on the other side. I can't figure out a couple of things. =20 1. Why is the md_spinlock_count for the new thread set to 1 and not to 0. This happens in cpu_fork and cpu_set_upcall under the amd64 tree.=20 2. If this was the "per-cpu" idle thread and the AP was booting up (running init_secondary) why does it grab sched_lock and call spinlock_exit. It would seem simpler to set the count of the idle thread to 0 and not have to call spinlock_exit. The only answer I can come up with is the fact that a non-zero spinlock_count prevents interrupts from getting disabled/renabled to some unknown value?=20 =20 Thanks Ravi Murty =20 From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 16 22:02:38 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60B4D106566C for ; Wed, 16 Apr 2008 22:02:38 +0000 (UTC) (envelope-from ravi.murty@intel.com) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx1.freebsd.org (Postfix) with ESMTP id 379F78FC16 for ; Wed, 16 Apr 2008 22:02:38 +0000 (UTC) (envelope-from ravi.murty@intel.com) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP; 16 Apr 2008 14:32:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.25,666,1199692800"; d="scan'208,217";a="271274528" Received: from orsmsx335.amr.corp.intel.com (HELO orsmsx335.jf.intel.com) ([10.22.226.40]) by orsmga002.jf.intel.com with ESMTP; 16 Apr 2008 14:33:26 -0700 Received: from orsmsx416.amr.corp.intel.com ([10.22.226.46]) by orsmsx335.jf.intel.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 Apr 2008 14:33:28 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 16 Apr 2008 14:33:27 -0700 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: md_spinlock_count? Thread-Index: AcigCYHX55FKH/ilSyKoSaQWgpt/wg== From: "Murty, Ravi" To: X-OriginalArrivalTime: 16 Apr 2008 21:33:28.0879 (UTC) FILETIME=[82BF63F0:01C8A009] X-Mailman-Approved-At: Wed, 16 Apr 2008 23:08:58 +0000 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: md_spinlock_count? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 22:02:38 -0000 Hello All, =20 I was looking at the code that creates a new process (fork) with a single thread coming out on the other side. I can't figure out a couple of things. =20 1. Why is the md_spinlock_count for the new thread set to 1 and not to 0. This happens in cpu_fork and cpu_set_upcall under the amd64 tree. 2. If this was the "per-cpu" idle thread and the AP was booting up (running init_secondary) why does it grab sched_lock and call spinlock_exit. It would seem simpler to set the count of the idle thread to 0 and not have to call spinlock_exit. The only answer I can come up with is the fact that a non-zero spinlock_count prevents interrupts from getting disabled/renabled to some unknown value? =20 Thanks Ravi Murty From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 17 00:12:04 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1DF211065672 for ; Thu, 17 Apr 2008 00:12:04 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outW.internet-mail-service.net (outw.internet-mail-service.net [216.240.47.246]) by mx1.freebsd.org (Postfix) with ESMTP id 06D6E8FC16 for ; Thu, 17 Apr 2008 00:12:03 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Thu, 17 Apr 2008 04:15:53 -0700 Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 79A6A2D6004; Wed, 16 Apr 2008 17:12:03 -0700 (PDT) Message-ID: <480695D7.5050600@elischer.org> Date: Wed, 16 Apr 2008 17:12:07 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.12 (Macintosh/20080213) MIME-Version: 1.0 To: "Murty, Ravi" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: md_spinlock_count? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2008 00:12:04 -0000 Murty, Ravi wrote: > Hello All, > > > > I was looking at the code that creates a new process (fork) with a > single thread coming out on the other side. I can't figure out a couple > of things. > > > > 1. Why is the md_spinlock_count for the new thread set to 1 and not > to 0. This happens in cpu_fork and cpu_set_upcall under the amd64 tree. > 2. If this was the "per-cpu" idle thread and the AP was booting up > (running init_secondary) why does it grab sched_lock and call > spinlock_exit. It would seem simpler to set the count of the idle thread > to 0 and not have to call spinlock_exit. The only answer I can come up > with is the fact that a non-zero spinlock_count prevents interrupts from > getting disabled/renabled to some unknown value? Which version/branch? (and a filename and linenumber would be good too) You might make it easy for us to answer you :-) > > > > Thanks > > Ravi Murty > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 17 13:44:56 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AFEC1065678 for ; Thu, 17 Apr 2008 13:44:56 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 70F338FC26 for ; Thu, 17 Apr 2008 13:44:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from zion.baldwin.cx (unknown [208.65.89.154]) by elvis.mu.org (Postfix) with ESMTP id 4B27C1A4D84; Thu, 17 Apr 2008 06:44:55 -0700 (PDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 17 Apr 2008 09:29:56 -0400 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200804170929.57192.jhb@freebsd.org> Cc: "Murty, Ravi" Subject: Re: md_spinlock_count? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2008 13:44:56 -0000 On Wednesday 16 April 2008 06:32:20 pm Murty, Ravi wrote: > Hello All, > > > > I was looking at the code that creates a new process (fork) with a > single thread coming out on the other side. I can't figure out a couple > of things. > > > > 1. Why is the md_spinlock_count for the new thread set to 1 and not > to 0. This happens in cpu_fork and cpu_set_upcall under the amd64 tree. Threads begin life during a context switch and during a context switch you always own a spin lock that is explicitly handed off to you during mi_switch(). The new threads actually start life in fork_trampoline() which calls fork_exit(). fork_exit() starts off by dropping the spin lock, so we need to set the thread to start out as if it was holding a spin lock. > 2. If this was the "per-cpu" idle thread and the AP was booting up > (running init_secondary) why does it grab sched_lock and call > spinlock_exit. It would seem simpler to set the count of the idle thread > to 0 and not have to call spinlock_exit. The only answer I can come up > with is the fact that a non-zero spinlock_count prevents interrupts from > getting disabled/renabled to some unknown value? First, you need the lock to enter the scheduler (when it calls cpu_throw() or sched_throw()) so you can start executing tasks. We also don't want to enable interrupts until we have entered the scheduler and are fully up and running. The code in HEAD and other branches has a big comment to explain the extra spinlock_exit() and it is related to md_spinlock_count being 1 for new threads as explained above as CPUs don't start up in fork_trampoline() but use a different code path and so need to account for the md_spinlock_count = 1 differently. Here is the comment in sched_throw() in sched_4bsd.c on HEAD: /* * Correct spinlock nesting. The idle thread context that we are * borrowing was created so that it would start out with a single * spin lock (sched_lock) held in fork_trampoline(). Since we've * explicitly acquired locks in this function, the nesting count * is now 2 rather than 1. Since we are nested, calling * spinlock_exit() will simply adjust the counts without allowing * spin lock using code to interrupt us. */ -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 19 01:18:09 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 724161065682 for ; Sat, 19 Apr 2008 01:18:09 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell.rawbw.com (shell.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 612CB8FC0A for ; Sat, 19 Apr 2008 01:18:09 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from eagle.syrec.org (ip224.carlyle.sfo.ygnition.net [24.219.144.224]) (authenticated bits=0) by shell.rawbw.com (8.13.6/8.13.6) with ESMTP id m3J1I8mL071340 for ; Fri, 18 Apr 2008 18:18:08 -0700 (PDT) Message-ID: <4809484F.3010607@rawbw.com> Date: Fri, 18 Apr 2008 18:18:07 -0700 From: Yuri User-Agent: Thunderbird 2.0.0.12 (X11/20080405) MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org References: <47FCF12D.3070902@rawbw.com> In-Reply-To: <47FCF12D.3070902@rawbw.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: C++ exceptions are broken in FreeBSD with gcc-compiled code? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: yuri@rawbw.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2008 01:18:09 -0000 I had some mixup of gcc versions so I got some false errors with system compiler. System gcc-4.2.1 works ok, as well as all gcc-4.2.X But the newest gcc-4.3.0 breaks. Executable aborts: Abort trap: 6 Stack trace in debugger is: Program received signal SIGABRT, Aborted. 0x28250a67 in kill () from /lib/libc.so.7 (gdb) bt #0 0x28250a67 in kill () from /lib/libc.so.7 #1 0x282509c6 in raise () from /lib/libc.so.7 #2 0x2824f5da in abort () from /lib/libc.so.7 #3 0x281759d6 in uw_init_context_1 (context=0xbfbfe5c4, outer_cfa=0xbfbfe660, outer_ra=0x28127fe8) at ../../../src/libgcc/../gcc/unwind-dw2.c:1249 #4 0x28175f3e in _Unwind_RaiseException (exc=0x81030b0) at ../../../src/libgcc/../gcc/unwind.inc:93 #5 0x28127fe8 in __cxa_throw (obj=0x81030d0, tinfo=0x8048da0, dest=0x80488ac <_ZNSsD1Ev@plt>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:71 #6 0x08048bcc in main () at exc-2.C:7 And shared libraries loaded correspond to this compiler. Yuri From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 19 17:18:40 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC6691065673 for ; Sat, 19 Apr 2008 17:18:40 +0000 (UTC) (envelope-from fbsd06+VE=21e66bf6@mlists.homeunix.com) Received: from turtle-out.mxes.net (turtle-out.mxes.net [216.86.168.191]) by mx1.freebsd.org (Postfix) with ESMTP id 730718FC22 for ; Sat, 19 Apr 2008 17:18:40 +0000 (UTC) (envelope-from fbsd06+VE=21e66bf6@mlists.homeunix.com) Received: from mxout-03.mxes.net (mxout-03.mxes.net [216.86.168.178]) by turtle-in.mxes.net (Postfix) with ESMTP id 2EB7916475E for ; Sat, 19 Apr 2008 12:57:00 -0400 (EDT) Received: from gumby.homeunix.com. (unknown [87.81.140.128]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTP id AE96023E3F9 for ; Sat, 19 Apr 2008 12:56:57 -0400 (EDT) Date: Sat, 19 Apr 2008 17:56:55 +0100 From: RW To: freebsd-hackers@freebsd.org Message-ID: <20080419175655.51a37bb2@gumby.homeunix.com.> X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.9; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Yarrow's Counter X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2008 17:18:40 -0000 The random number generator in FreeBSD's Yarrow implementation uses AES256 in counter mode. When a reseed occurs the generator is reinitialised like this: - generate a new cypher-key from the pool[s] and the old key - zero the counter - encrypt the (zeroed) counter with the new key My question is: why zero the counter? If it's not zeroed then the old counter is encrypted instead, and after a few reseeds the counter will accumulate an independent 256 bits of entropy, rather than being a function of the new key. Should I submit a patch, it's simply a matter of deleting two lines in reseed() in sys/dev/random/yarrow.c. yarrow_hash_finish(&context, temp); yarrow_encrypt_init(&random_state.key, temp); /* 4. Recompute the counter */ for (i = 0; i < 4; i++) <--- random_state.counter[i] = 0; <--- yarrow_encrypt(&random_state.key, random_state.counter, temp); memcpy(random_state.counter, temp, sizeof(random_state.counter)); From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 19 18:17:51 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCA3C106564A for ; Sat, 19 Apr 2008 18:17:51 +0000 (UTC) (envelope-from randyhyde@earthlink.net) Received: from elasmtp-mealy.atl.sa.earthlink.net (elasmtp-mealy.atl.sa.earthlink.net [209.86.89.69]) by mx1.freebsd.org (Postfix) with ESMTP id 89DC98FC2D for ; Sat, 19 Apr 2008 18:17:51 +0000 (UTC) (envelope-from randyhyde@earthlink.net) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=VDO31OgNWwYMXMpM25DqNovxtBAO+C/tJG+w4pA9XUXoCKBonTyVoQo4w6ScpfWb; h=Message-ID:Date:From:Reply-To:To:Subject:Mime-Version:Content-Type:Content-Transfer-Encoding:X-Mailer:X-ELNK-Trace:X-Originating-IP; Received: from [209.86.224.48] (helo=elwamui-rustique.atl.sa.earthlink.net) by elasmtp-mealy.atl.sa.earthlink.net with esmtpa (Exim 4.67) (envelope-from ) id 1JnHON-00077g-LX for freebsd-hackers@freebsd.org; Sat, 19 Apr 2008 14:02:27 -0400 Received: from 71.9.7.20 by webmail.pas.earthlink.net with HTTP; Sat, 19 Apr 2008 14:02:27 -0400 Message-ID: <29853546.1208628147652.JavaMail.root@elwamui-rustique.atl.sa.earthlink.net> Date: Sat, 19 Apr 2008 11:02:27 -0700 (GMT-07:00) From: Randall Hyde To: freebsd-hackers@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailer: EarthLink Zoo Mail 1.0 X-ELNK-Trace: eba5e0c9192a36dcd6dd28457998182d7e972de0d01da940443a92b74ef4b8e9135d2dc2d1c3c7eb350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 209.86.224.48 Subject: chdir/rmdir X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Randall Hyde List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2008 18:17:51 -0000 Hi, I recently made a couple of calls like the following // currently in /x/y/z chdir( "/x/y" ); rmdir( "/x/y/z" ); When I did at "gwd" call, it returned "/x/y/z" along with ENOTDIR. Is this a known issue? I'm making low-level (assembly) calls via int 0x80 to do the above (not C stdlib), though I can't imagine that's causing the problem (then again, I've written broken code before :-(, this wouldn't be the first time I've messed up, though I have studied the code very closely and the same function calls work fine in other contexts). Thanks for any input, Randy Hyde P.S. I noticed that the man pages said something about using open on "." and fchdir to more robustly switch back to some previous directory; this is what has me wondering if there is a problem with executing statements like the above. From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 19 19:30:21 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA70D106566C for ; Sat, 19 Apr 2008 19:30:21 +0000 (UTC) (envelope-from rea-fbsd@codelabs.ru) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.177.45]) by mx1.freebsd.org (Postfix) with ESMTP id 593808FC2B for ; Sat, 19 Apr 2008 19:30:21 +0000 (UTC) (envelope-from rea-fbsd@codelabs.ru) DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=one; d=codelabs.ru; h=Received:Date:From:To:Cc:Message-ID:References:MIME-Version:Content-Type:Content-Disposition:In-Reply-To:Sender:Subject; b=ZDth4WBnG/l4OjKYjLlF9e/qTjwc3I1Gr9qMSsIcqXMadH6JY9uOEOaL0CKzWN4U07OdFdXz2aERFnqFBhTfa875g5Nk2I9lzLlFb53OkHeXYth8Hwv+xhoEdUgIJpq8yEmHgBFzM3oA2kJ9YWBrQCaX31Ds8+f+I/CU75Gl/UI=; Received: from amnesiac.at.no.dns (ppp83-237-104-51.pppoe.mtu-net.ru [83.237.104.51]) by 0.mx.codelabs.ru with esmtpsa (TLSv1:AES256-SHA:256) id 1JnIlP-000ONp-Pr; Sat, 19 Apr 2008 23:30:19 +0400 Date: Sat, 19 Apr 2008 23:30:23 +0400 From: Eygene Ryabinkin To: RW Message-ID: References: <20080419175655.51a37bb2@gumby.homeunix.com.> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20080419175655.51a37bb2@gumby.homeunix.com.> Sender: rea-fbsd@codelabs.ru Cc: freebsd-hackers@freebsd.org Subject: Re: Yarrow's Counter X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2008 19:30:21 -0000 Good day. Sat, Apr 19, 2008 at 05:56:55PM +0100, RW wrote: > The random number generator in FreeBSD's Yarrow implementation uses > AES256 in counter mode. When a reseed occurs the generator is > reinitialised like this: > > - generate a new cypher-key from the pool[s] and the old key > - zero the counter > - encrypt the (zeroed) counter with the new key The latter two are better explained as "generate new counter as the result of encryption of a number 'zero' with the new key". > My question is: why zero the counter? It is per paper about Yarrow design: see http://www.schneier.com/paper-yarrow.html page 11, section 5.3, step 4. > If it's not zeroed then the old counter is encrypted instead, and after > a few reseeds the counter will accumulate an independent 256 bits of > entropy, rather than being a function of the new key. As the seventh paragraph of section 5.3 says, "There is no security reason why we would set a new value for the counter C". And deriving the new value of C from the old one will not add any additional entropy -- you're producing the old C and new key from the same "entropy source", so this won't give you more entropy: you have two dependent values. Moreover, as the last paragraph of page 9 says "...the counter value C is assumed to be known to the attacker", Yarrow was designed with this motto in mind. As I see it, the key reasoning is that for the perfect encryption function in the counter mode, there is no reason to keep the counter to be secret: it is just nonce, nothing more. Only the key should be kept safe. > Should I submit a patch, it's simply a matter of deleting two > lines in reseed() in sys/dev/random/yarrow.c. > > > yarrow_hash_finish(&context, temp); > yarrow_encrypt_init(&random_state.key, temp); > > /* 4. Recompute the counter */ > > for (i = 0; i < 4; i++) <--- > random_state.counter[i] = 0; <--- > > yarrow_encrypt(&random_state.key, random_state.counter, temp); > memcpy(random_state.counter, temp, sizeof(random_state.counter)); I would not do it without consultations with Yarrow's creators: this modification seems not to help anything, but can break something. But your mileage may vary, as usual ;)) -- Eygene