From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 06:31:25 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B294516A4CE for ; Mon, 15 Mar 2004 06:31:25 -0800 (PST) Received: from darkness.comp.waw.pl (unknown [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED7EA43D2D for ; Mon, 15 Mar 2004 06:31:24 -0800 (PST) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id B54B3ACAE0; Mon, 15 Mar 2004 15:31:23 +0100 (CET) Date: Mon, 15 Mar 2004 15:31:23 +0100 From: Pawel Jakub Dawidek To: freebsd-arch@freebsd.org Message-ID: <20040315143123.GB8930@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mxv5cy4qt+RJ9ypb" Content-Disposition: inline User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 Subject: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 14:31:25 -0000 --mxv5cy4qt+RJ9ypb Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello. Ok, bitstring(9) is broken for kernel use atm, because leak of calloc() function. What we should do about it? Is this patch acceptable or maybe we should add calloc() macro? http://people.freebsd.org/~pjd/patches/bitstring.h.patch Or maybe there is calloc() somewhere and I'm missing something? --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --mxv5cy4qt+RJ9ypb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAVb47ForvXbEpPzQRAnYKAKDVTw3Oy3Ww6VGDIVPoWuzBQv+S8wCggyhH CGmnmRn2AL9EpKWQ5ohukJA= =iFL3 -----END PGP SIGNATURE----- --mxv5cy4qt+RJ9ypb-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 06:52:55 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B3BA316A4CE; Mon, 15 Mar 2004 06:52:55 -0800 (PST) Received: from smtp.des.no (flood.des.no [217.116.83.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E64143D53; Mon, 15 Mar 2004 06:52:55 -0800 (PST) (envelope-from des@des.no) Received: by smtp.des.no (Pony Express, from userid 666) id B1561530E; Mon, 15 Mar 2004 15:52:54 +0100 (CET) Received: from dwp.des.no (des.no [80.203.228.37]) by smtp.des.no (Pony Express) with ESMTP id 97101530A; Mon, 15 Mar 2004 15:52:48 +0100 (CET) Received: by dwp.des.no (Postfix, from userid 2602) id 820EE33CA7; Mon, 15 Mar 2004 15:52:48 +0100 (CET) To: Pawel Jakub Dawidek References: <20040315143123.GB8930@darkness.comp.waw.pl> From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Mon, 15 Mar 2004 15:52:48 +0100 In-Reply-To: <20040315143123.GB8930@darkness.comp.waw.pl> (Pawel Jakub Dawidek's message of "Mon, 15 Mar 2004 15:31:23 +0100") Message-ID: User-Agent: Gnus/5.090024 (Oort Gnus v0.24) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on flood.des.no X-Spam-Level: X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.63 cc: freebsd-arch@freebsd.org Subject: Re: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 14:52:55 -0000 Pawel Jakub Dawidek writes: > Ok, bitstring(9) is broken for kernel use atm, because leak of calloc() > function. What we should do about it? No, it isn't broken. You just have to use bitstr_size() to figure out how much space it needs and do the alloc yourself. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 07:02:48 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DF0AE16A4CE; Mon, 15 Mar 2004 07:02:48 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id C256243D39; Mon, 15 Mar 2004 07:02:48 -0800 (PST) (envelope-from mux@freebsd.org) Received: by elvis.mu.org (Postfix, from userid 1920) id ABDAA5C79A; Mon, 15 Mar 2004 07:02:48 -0800 (PST) Date: Mon, 15 Mar 2004 16:02:48 +0100 From: Maxime Henrion To: Dag-Erling Sm?rgrav Message-ID: <20040315150248.GL35475@elvis.mu.org> References: <20040315143123.GB8930@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i cc: Pawel Jakub Dawidek cc: freebsd-arch@freebsd.org Subject: Re: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 15:02:49 -0000 Dag-Erling Sm?rgrav wrote: > Pawel Jakub Dawidek writes: > > Ok, bitstring(9) is broken for kernel use atm, because leak of calloc() > > function. What we should do about it? > > No, it isn't broken. You just have to use bitstr_size() to figure out > how much space it needs and do the alloc yourself. That is, reimplement bit_alloc(). This makes 0 sense, we should indeed fix bit_alloc() as Pawel suggested. Cheers, Maxime From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 07:10:22 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BB0A16A4CE; Mon, 15 Mar 2004 07:10:22 -0800 (PST) Received: from smtp.des.no (flood.des.no [217.116.83.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F3A843D1F; Mon, 15 Mar 2004 07:10:22 -0800 (PST) (envelope-from des@des.no) Received: by smtp.des.no (Pony Express, from userid 666) id 20FB75310; Mon, 15 Mar 2004 16:10:21 +0100 (CET) Received: from dwp.des.no (des.no [80.203.228.37]) by smtp.des.no (Pony Express) with ESMTP id 91068530E; Mon, 15 Mar 2004 16:10:12 +0100 (CET) Received: by dwp.des.no (Postfix, from userid 2602) id 3D1E833CA7; Mon, 15 Mar 2004 16:10:12 +0100 (CET) To: Maxime Henrion References: <20040315143123.GB8930@darkness.comp.waw.pl> <20040315150248.GL35475@elvis.mu.org> From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Mon, 15 Mar 2004 16:10:12 +0100 In-Reply-To: <20040315150248.GL35475@elvis.mu.org> (Maxime Henrion's message of "Mon, 15 Mar 2004 16:02:48 +0100") Message-ID: User-Agent: Gnus/5.090024 (Oort Gnus v0.24) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on flood.des.no X-Spam-Level: X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.63 cc: Pawel Jakub Dawidek cc: freebsd-arch@freebsd.org Subject: Re: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 15:10:22 -0000 Maxime Henrion writes: > Dag-Erling Sm?rgrav wrote: > > Pawel Jakub Dawidek writes: > > > Ok, bitstring(9) is broken for kernel use atm, because leak of calloc= () > > > function. What we should do about it? > > No, it isn't broken. You just have to use bitstr_size() to figure out > > how much space it needs and do the alloc yourself. > That is, reimplement bit_alloc(). This makes 0 sense, we should indeed > fix bit_alloc() as Pawel suggested. What doesn't make sense is to assume that there is no difference between the kernel and userland and that you can write code which will work in both. There are bugs in the kernel which I can't fix properly because Somebody[tm] decided they wanted to use sbufs in userland, and dumbed down the code so the changes I need to make are no longer possible. So we have to live with workarounds... That being said, I have no objection to Pawel's patch. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 07:39:21 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C898C16A4CE; Mon, 15 Mar 2004 07:39:21 -0800 (PST) Received: from darkness.comp.waw.pl (unknown [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D4A043D49; Mon, 15 Mar 2004 07:39:21 -0800 (PST) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id D8335AC974; Mon, 15 Mar 2004 16:39:19 +0100 (CET) Date: Mon, 15 Mar 2004 16:39:19 +0100 From: Pawel Jakub Dawidek To: Dag-Erling Sm?rgrav Message-ID: <20040315153919.GC8930@darkness.comp.waw.pl> References: <20040315143123.GB8930@darkness.comp.waw.pl> <20040315150248.GL35475@elvis.mu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+xNpyl7Qekk2NvDX" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-arch@freebsd.org Subject: Re: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 15:39:21 -0000 --+xNpyl7Qekk2NvDX Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 15, 2004 at 04:10:12PM +0100, Dag-Erling Sm?rgrav wrote: +> That being said, I have no objection to Pawel's patch. Just want to double check: Should I go ahead and commit it? Or you want think this over? The only user- and kernel-land friendly solution will be to reimplement bit_alloc() as bit_alloc(bits, type, flags). Arguments 'type' and 'flags' will be unused in userland and for userland-only application we can provide bit_ualloc(bits) or something. But this looks a bit hackish. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --+xNpyl7Qekk2NvDX Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAVc4nForvXbEpPzQRAv1KAKCMIBdjGCTXz15wRUhH4aXsb3RooQCfQeFY fHuyfTWiMFMebUD4yIoPNP0= =RTSx -----END PGP SIGNATURE----- --+xNpyl7Qekk2NvDX-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 07:43:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D69316A4CE for ; Mon, 15 Mar 2004 07:43:47 -0800 (PST) Received: from darkness.comp.waw.pl (unknown [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0650343D1D for ; Mon, 15 Mar 2004 07:43:47 -0800 (PST) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 3808BAC974; Mon, 15 Mar 2004 16:43:46 +0100 (CET) Date: Mon, 15 Mar 2004 16:43:46 +0100 From: Pawel Jakub Dawidek To: Dag-Erling Sm?rgrav Message-ID: <20040315154346.GD8930@darkness.comp.waw.pl> References: <20040315143123.GB8930@darkness.comp.waw.pl> <20040315150248.GL35475@elvis.mu.org> <20040315153919.GC8930@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9UV9rz0O2dU/yYYn" Content-Disposition: inline In-Reply-To: <20040315153919.GC8930@darkness.comp.waw.pl> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-arch@freebsd.org Subject: Re: Bitstring(9) in kernel. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 15:43:47 -0000 --9UV9rz0O2dU/yYYn Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 15, 2004 at 04:39:19PM +0100, Pawel Jakub Dawidek wrote: +> On Mon, Mar 15, 2004 at 04:10:12PM +0100, Dag-Erling Sm?rgrav wrote: +> +> That being said, I have no objection to Pawel's patch. +>=20 +> Just want to double check: Should I go ahead and commit it? +> Or you want think this over? +>=20 +> The only user- and kernel-land friendly solution will be to +> reimplement bit_alloc() as bit_alloc(bits, type, flags). +> Arguments 'type' and 'flags' will be unused in userland and for +> userland-only application we can provide bit_ualloc(bits) +> or something. But this looks a bit hackish. No, maybe just create bit_malloc(bits, type, flags) and don't touch bit_alloc() for backwards compatibility. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --9UV9rz0O2dU/yYYn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAVc8yForvXbEpPzQRArDkAJwLQOMY2yabBoi1HA3XKdkhSEE1QQCcDFri 3rrxmtp7sN5QpoWQ6lETQ0U= =e55N -----END PGP SIGNATURE----- --9UV9rz0O2dU/yYYn-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 19:18:46 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CFAF16A4CE for ; Mon, 15 Mar 2004 19:18:46 -0800 (PST) Received: from smtp.distributel.net (cns2.distributel.NET [66.38.181.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 18A1943D46 for ; Mon, 15 Mar 2004 19:18:46 -0800 (PST) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by smtp.distributel.net (8.12.6/8.12.6) with ESMTP id i2G3IiM2033027; Mon, 15 Mar 2004 22:18:44 -0500 (EST) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) i2G3H36p003829; Mon, 15 Mar 2004 22:17:03 -0500 (EST) (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost)i2G3H3vm003828; Mon, 15 Mar 2004 22:17:03 -0500 (EST) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Mon, 15 Mar 2004 22:17:03 -0500 From: Bosko Milekic To: Doug Rabson Message-ID: <20040316031702.GA3794@technokratis.com> References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040218182136.S5430@mail.chesapeake.net> <1077181355.28133.13.camel@herring.nlsystems.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1077181355.28133.13.camel@herring.nlsystems.com> User-Agent: Mutt/1.4.1i cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 03:18:46 -0000 On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > On Wed, 18 Feb 2004, Doug Rabson wrote: > > > So, I was reading the latest installment of the SCO soap on slashdot > > > today (bizarre - they seem to be claiming that they own all code that > > > was ever linked to a System V kernel) and I ended up trying to figure > > > out exactly what this RCU thing is that they claim is infringing their > > > right to obtain money with menaces. > > > > > > I read one of the original papers at > > > http://www.rdrop.com/users/paulmck/paper/rclockpdcsproof.pdf and its a > > > surprisingly simple idea. Basically for certain data structures which > > > are read-mostly, you can make the entire read path lock-free at the > > > expense of making updates quite a bit more expensive. They claim that > > > its much faster than reader-writer locks both for light contention and > > > for heavy contention. > > > > > > I imagine that a FreeBSD implementation of RCU wouldn't actually be too > > > hard and it might be well worth it as an alternative way of managing > > > concurrency, e.g. for the routing cache and the name cache (and probably > > > lots of other things). > > > > > > > I've looked into this a bit myself. My general feeling is that it would > > not be terribly difficult to do, but I would prefer to start with stronger > > primitives and work our way into more efficient mechanisms. I say this > > for two reasons. First, as a general rule of optimizations, you only > > optimize the things that need it. I think we should wait and measure > > contention and then optimize things. Secondly, we need to get more > > confidence in the correctness of simpler mechanisms in most subsystems > > before we go to something more complex. It will be easier to move to RCU > > once a subsystem is already protected by mutexes. > > > > I think that this is a good path to go down, but I really don't think > > we're ready yet. I'd rather see energy spent protecting code than > > building more infrastructure. > > I completely agree. I was just musing about this as a future addition to > the locking toolbox. Its certainly not worth trying before enough of the > kernel is outside the giant lock to make it worthwhile. As Jeff said and you agree, it is probably too early for this now. Also, I've looked at the paper you quote before SCO's announcement (which by the way I had no idea about until now), and I think we'll eventually do just as well in the common case without going to the RCU model at all. -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/ "It is impossible for anyone to begin to learn what he believes he already knows." -- Epictetus (c.a.d. 55-c135) From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 19:22:11 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 50FD916A4CE for ; Mon, 15 Mar 2004 19:22:11 -0800 (PST) Received: from smtp.distributel.net (cns2.distributel.NET [66.38.181.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id E859843D39 for ; Mon, 15 Mar 2004 19:22:10 -0800 (PST) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by smtp.distributel.net (8.12.6/8.12.6) with ESMTP id i2G3MAM2033385 for ; Mon, 15 Mar 2004 22:22:10 -0500 (EST) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) i2G3KT6p003848 for ; Mon, 15 Mar 2004 22:20:29 -0500 (EST) (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost)i2G3KTwh003847 for freebsd-arch@freebsd.org; Mon, 15 Mar 2004 22:20:29 -0500 (EST) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Mon, 15 Mar 2004 22:20:29 -0500 From: Bosko Milekic To: freebsd-arch@freebsd.org Message-ID: <20040316032029.GA3735@technokratis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: mbuma: network buffers & UMA X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 03:22:11 -0000 The post below has received exactly zero attention on the freebsd-current mailing list. I'm assuming this is because there are other exciting threads going on there right now, such as style debates and lovely feature requests. :-) I would like to commit this RSN. I need feedback from the Release Engineering people on how close we are to a RELEASE and if it's OK for me to break netstat -m stats for a few weeks or, if not, what the appropriate course of action is. Fellow earthlings, I would like to request some testing on diverse architectures of the following patch (~130K): http://people.freebsd.org/~bmilekic/code/mbuma-v2.diff In short, the above accomplishes several things and is the result of one of my now mature p4 repositories which I have been able to again address following a generous hardware donation. This basically gets rid of the existing mbuf allocator and replaces it with some routines which get hooked on top of UMA, the existing general-purpose SMP-friendly allocator in -CURRENT, after adding some extensions to UMA which hopefully make allocations faster than they would be without them. The current list of implications (whether good or bad is for you to decide): - NMBCLUSTERS, as a compile-time option, is now gone. The kern.ipc.nmbclusters tunable and sysctl still exist and are both read+write. Currently, the sysctl does not effect anything but will soon be made to allow for runtime (dynamic) modification of the maximum number of mbuf clusters allocatable cap. If you set the tunable to zero, though, the number of clusters allocatable is unbounded and the system will scale according to demand. Be careful... leaving these unbounded may not be what you really want. - Memory allocated to network buffers can (and now will) be reclaimed by UMA when required and, for example, after a large network spike. - Current on-par performance with the existing allocator. Soon some modifications to UMA that improve performance drastically in the common case may change that, though - for the better. - For developers: M_WAIT behavior now is the same as for all other UMA allocations, and is no longer special for network buffers. - Currently netstat(1) mbuf stats ('netstat -m') are broken and instead the following is displayed: fermat% netstat -m Mbuf statistics are currently unavailable via netstat(1), see UPDATING. This is only a TEMPORARY measure in -CURRENT. For current mbuf and cluster allocation stats, see sysctl vm.zone I don't plan to fix them prior to doing some other work (which may change the way stats work, anyway) in UMA. Hopefully this won't be too much of a problem for most of you. - So far has been very stable on my dual Athlon and a friend's x86 UP machine (I think it's UP, anyway). Some other architectures, particular 64-bit ones, would be worth testing. - Some things (e.g., dev/nsp) I just noticed might be broken by the above changes due to slight time-delay between the vendor p4 branch and the CVS checked out repo I used to generate the final diff. If something you're using is broken by the diff, it is only broken slightly, and you'll notice it immediately during kernel compile, so you can fix it (it's generally just changing some includes) or Email it to me and I'll send you a custom diff. This only applies to the current diff posted above and will not be committed. That's all I could think of right now. I would like to commit this soon as I have other things in the pipeline that build on top of it. I should also mention that there are various other implications in addition to the above but are more relevant to the developer than the user, so they'll probably appear in a really large commit log. Reviews and constructive criticism is both welcomed and appreciated. Please leave me in the To: line - although I am subscribed to -current, I'd like to see feedback in my main box. Regards, -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/ "It is impossible for anyone to begin to learn what he believes he already knows." -- Epictetus (c.a.d. 55-c135) From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 21:22:01 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D639F16A4CE for ; Mon, 15 Mar 2004 21:22:01 -0800 (PST) Received: from sohgo.tanimura.dyndns.org (IP1A1374.kng.mesh.ad.jp [61.203.105.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24A2C43D45 for ; Mon, 15 Mar 2004 21:22:00 -0800 (PST) (envelope-from tanimura@tanimura.dyndns.org) Received: from sohgo.tanimura.dyndns.org (localhost [IPv6:::1]) ESMTP id i2G5LtlY017831 ; Tue, 16 Mar 2004 14:21:55 +0900 (JST) Received: (from uucp@localhost) (8.12.11/3.7W-submit-carrots-Tokyu-Meguro) with UUCP id i2G5LrcO017830 ; Tue, 16 Mar 2004 14:21:53 +0900 (JST) Received: from urban.nkth.tanimura.dyndns.org (localhost [IPv6:::1]) with ESMTP id i2G5J0V6023193 ; Tue, 16 Mar 2004 14:19:00 +0900 (JST) Message-Id: <200403160519.i2G5J0V6023193@urban> Date: Tue, 16 Mar 2004 14:19:00 +0900 From: Seigo Tanimura To: arch@FreeBSD.org User-Agent: Wanderlust/2.10.1 (Watching The Wheels) SEMI/1.14.5 (Awara-Onsen) FLIM/1.14.5 (Demachiyanagi) APEL/10.6 MULE XEmacs/21.4 (patch 14) (Reasonable Discussion) (i386--freebsd) Organization: My Home MIME-Version: 1.0 (generated by SEMI 1.14.5 - "Awara-Onsen") Content-Type: text/plain; charset=US-ASCII cc: Seigo Tanimura Subject: Is MTX_CONTESTED evil? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 05:22:01 -0000 _mtx_unlock_sleep() currently wakes up only one thread being blocked, and leaves MTX_CONTESTED on a mutex. According to Solaris Internals, that strategy adds an overhead to check for MTX_CONTESTED on a mutex, even though it is not held by any thread. The thread waken up cannot grab the mutex immediately by _obtain_lock() and have to go through _mtx_lock_sleep(). The penalty tends to be large for a mutex with a high contention, and we have at least one of such a mutex - Giant. What would it be like if we axed MTX_CONTEST and let _mtx_unlock_sleep() wake up all of the blocked threads? -- Seigo Tanimura From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 00:53:03 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80D7216A4CE for ; Tue, 16 Mar 2004 00:53:03 -0800 (PST) Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF35243D1D for ; Tue, 16 Mar 2004 00:53:02 -0800 (PST) (envelope-from mime@traveller.cz) Received: from prg.traveller.cz (localhost [127.0.0.1]) i2G8r1Kh043815; Tue, 16 Mar 2004 09:53:01 +0100 (CET) Received: from localhost (mime@localhost)i2G8r1GB043812; Tue, 16 Mar 2004 09:53:01 +0100 (CET) Date: Tue, 16 Mar 2004 09:53:00 +0100 (CET) From: Michal Mertl To: Bosko Milekic In-Reply-To: <20040316032029.GA3735@technokratis.com> Message-ID: <20040316094551.Q41416@prg.traveller.cz> References: <20040316032029.GA3735@technokratis.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@freebsd.org Subject: Re: mbuma: network buffers & UMA X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 08:53:03 -0000 I downloaded your patch and compiled a kernel on SMP AMD64 with WITNESS with it. Apart from breaking MAC (does not compile) all seems ok. I've never liked the idea of multiple allocators and hard limits so let me say that I believe your proposed change is a good one. On Mon, 15 Mar 2004, Bosko Milekic wrote: > > The post below has received exactly zero attention on the > freebsd-current mailing list. I'm assuming this is because there are > other exciting threads going on there right now, such as style debates > and lovely feature requests. :-) > > I would like to commit this RSN. I need feedback from the Release > Engineering people on how close we are to a RELEASE and if it's OK for > me to break netstat -m stats for a few weeks or, if not, what the > appropriate course of action is. > > > Fellow earthlings, > > I would like to request some testing on diverse architectures of the > following patch (~130K): > > http://people.freebsd.org/~bmilekic/code/mbuma-v2.diff > > In short, the above accomplishes several things and is the result of > one of my now mature p4 repositories which I have been able to again > address following a generous hardware donation. > > This basically gets rid of the existing mbuf allocator and replaces > it with some routines which get hooked on top of UMA, the existing > general-purpose SMP-friendly allocator in -CURRENT, after adding > some extensions to UMA which hopefully make allocations faster than > they would be without them. > > The current list of implications (whether good or bad is for you to > decide): > > - NMBCLUSTERS, as a compile-time option, is now gone. The > kern.ipc.nmbclusters tunable and sysctl still exist and are both > read+write. Currently, the sysctl does not effect anything but > will soon be made to allow for runtime (dynamic) modification of > the maximum number of mbuf clusters allocatable cap. If you set > the tunable to zero, though, the number of clusters allocatable is > unbounded and the system will scale according to demand. Be > careful... leaving these unbounded may not be what you really > want. > > - Memory allocated to network buffers can (and now will) be > reclaimed by UMA when required and, for example, after a large > network spike. > > - Current on-par performance with the existing allocator. Soon some > modifications to UMA that improve performance drastically in the > common case may change that, though - for the better. > > - For developers: M_WAIT behavior now is the same as for all other > UMA allocations, and is no longer special for network buffers. > > - Currently netstat(1) mbuf stats ('netstat -m') are broken and > instead the following is displayed: > > fermat% netstat -m > Mbuf statistics are currently unavailable via netstat(1), see > UPDATING. > This is only a TEMPORARY measure in -CURRENT. > For current mbuf and cluster allocation stats, see sysctl vm.zone > > I don't plan to fix them prior to doing some other work (which may > change the way stats work, anyway) in UMA. Hopefully this won't > be too much of a problem for most of you. > > - So far has been very stable on my dual Athlon and a friend's x86 > UP machine (I think it's UP, anyway). Some other architectures, > particular 64-bit ones, would be worth testing. > > - Some things (e.g., dev/nsp) I just noticed might be broken by the > above changes due to slight time-delay between the vendor p4 branch > and the CVS checked out repo I used to generate the final diff. > If something you're using is broken by the diff, it is only broken > slightly, and you'll notice it immediately during kernel compile, > so you can fix it (it's generally just changing some includes) or > Email it to me and I'll send you a custom diff. This only > applies to the current diff posted above and will not be > committed. > > > That's all I could think of right now. I would like to commit this > soon as I have other things in the pipeline that build on top of it. > I should also mention that there are various other implications in > addition to the above but are more relevant to the developer than the > user, so they'll probably appear in a really large commit log. > > Reviews and constructive criticism is both welcomed and appreciated. > Please leave me in the To: line - although I am subscribed to > -current, I'd like to see feedback in my main box. > > Regards, > -- > Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org > TECHNOkRATIS Consulting Services * http://www.technokratis.com/ > > "It is impossible for anyone to begin to learn what he believes > he already knows." -- Epictetus (c.a.d. 55-c135) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > -- Michal Mertl From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 01:15:04 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B91516A4CE for ; Tue, 16 Mar 2004 01:15:04 -0800 (PST) Received: from herring.rabson.org (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB48A43D46 for ; Tue, 16 Mar 2004 01:15:03 -0800 (PST) (envelope-from dfr@nlsystems.com) Received: from herring.rabson.org (herring.rabson.org [10.0.0.2]) by herring.rabson.org (8.12.11/8.12.11) with ESMTP id i2G9EqUY012694; Tue, 16 Mar 2004 09:14:52 GMT (envelope-from dfr@nlsystems.com) From: Doug Rabson To: Bosko Milekic Date: Tue, 16 Mar 2004 09:14:51 +0000 User-Agent: KMail/1.6.1 References: <1077137806.28133.10.camel@herring.nlsystems.com> <1077181355.28133.13.camel@herring.nlsystems.com> <20040316031702.GA3794@technokratis.com> In-Reply-To: <20040316031702.GA3794@technokratis.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403160914.51727.dfr@nlsystems.com> X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.63 X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on herring.rabson.org X-Virus-Scanned: ClamAV version 'clamd / ClamAV version 0.65', clamav-milter version '0.60p' cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 09:15:04 -0000 On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > I think that this is a good path to go down, but I really don't > > > think we're ready yet. I'd rather see energy spent protecting > > > code than building more infrastructure. > > > > I completely agree. I was just musing about this as a future > > addition to the locking toolbox. Its certainly not worth trying > > before enough of the kernel is outside the giant lock to make it > > worthwhile. > > As Jeff said and you agree, it is probably too early for this now. > Also, I've looked at the paper you quote before SCO's announcement > (which by the way I had no idea about until now), and I think we'll > eventually do just as well in the common case without going to the > RCU model at all. Its a pretty neat idea though. I like the sound of being able to e.g. read from the namecache without needing to take an expensive lock. With the way 5-CURRENT works, we would probably still need to suppress context switching which is expensive on intel processors in the current implementation. I guess that could be fixed using some kind of lazy-cli scheme. The main barrier here (apart from the need to push Giant down in more places and stabilise the existing implementation) is IBM's patent. The SCO IP claim (bogus as it is) only covers Sequent's original implementation. From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 01:34:21 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 294B116A4CE for ; Tue, 16 Mar 2004 01:34:21 -0800 (PST) Received: from mailtoaster1.pipeline.ch (mailtoaster1.pipeline.ch [62.48.0.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F03243D2D for ; Tue, 16 Mar 2004 01:34:20 -0800 (PST) (envelope-from andre@freebsd.org) Received: (qmail 34902 invoked from network); 16 Mar 2004 09:34:19 -0000 Received: from unknown (HELO freebsd.org) ([62.48.0.53]) (envelope-sender ) by mailtoaster1.pipeline.ch (qmail-ldap-1.03) with SMTP for ; 16 Mar 2004 09:34:19 -0000 Message-ID: <4056CA1B.51595D0F@freebsd.org> Date: Tue, 16 Mar 2004 10:34:19 +0100 From: Andre Oppermann X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Bosko Milekic References: <20040316032029.GA3735@technokratis.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-arch@freebsd.org Subject: Re: mbuma: network buffers & UMA X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 09:34:21 -0000 Bosko Milekic wrote: > > The post below has received exactly zero attention on the > freebsd-current mailing list. I'm assuming this is because there are > other exciting threads going on there right now, such as style debates > and lovely feature requests. :-) > > I would like to commit this RSN. I need feedback from the Release > Engineering people on how close we are to a RELEASE and if it's OK for > me to break netstat -m stats for a few weeks or, if not, what the > appropriate course of action is. I haven't looked at the diff in detail (the largest part seems to be the removal of subr_mbuf.c) but it sounds very exciting! I would love to have a UMA based mbuf system in 5-CURRENT and 5.3R. -- Andre > Fellow earthlings, > > I would like to request some testing on diverse architectures of the > following patch (~130K): > > http://people.freebsd.org/~bmilekic/code/mbuma-v2.diff > > In short, the above accomplishes several things and is the result of > one of my now mature p4 repositories which I have been able to again > address following a generous hardware donation. > > This basically gets rid of the existing mbuf allocator and replaces > it with some routines which get hooked on top of UMA, the existing > general-purpose SMP-friendly allocator in -CURRENT, after adding > some extensions to UMA which hopefully make allocations faster than > they would be without them. > > The current list of implications (whether good or bad is for you to > decide): > > - NMBCLUSTERS, as a compile-time option, is now gone. The > kern.ipc.nmbclusters tunable and sysctl still exist and are both > read+write. Currently, the sysctl does not effect anything but > will soon be made to allow for runtime (dynamic) modification of > the maximum number of mbuf clusters allocatable cap. If you set > the tunable to zero, though, the number of clusters allocatable is > unbounded and the system will scale according to demand. Be > careful... leaving these unbounded may not be what you really > want. > > - Memory allocated to network buffers can (and now will) be > reclaimed by UMA when required and, for example, after a large > network spike. > > - Current on-par performance with the existing allocator. Soon some > modifications to UMA that improve performance drastically in the > common case may change that, though - for the better. > > - For developers: M_WAIT behavior now is the same as for all other > UMA allocations, and is no longer special for network buffers. > > - Currently netstat(1) mbuf stats ('netstat -m') are broken and > instead the following is displayed: > > fermat% netstat -m > Mbuf statistics are currently unavailable via netstat(1), see > UPDATING. > This is only a TEMPORARY measure in -CURRENT. > For current mbuf and cluster allocation stats, see sysctl vm.zone > > I don't plan to fix them prior to doing some other work (which may > change the way stats work, anyway) in UMA. Hopefully this won't > be too much of a problem for most of you. > > - So far has been very stable on my dual Athlon and a friend's x86 > UP machine (I think it's UP, anyway). Some other architectures, > particular 64-bit ones, would be worth testing. > > - Some things (e.g., dev/nsp) I just noticed might be broken by the > above changes due to slight time-delay between the vendor p4 branch > and the CVS checked out repo I used to generate the final diff. > If something you're using is broken by the diff, it is only broken > slightly, and you'll notice it immediately during kernel compile, > so you can fix it (it's generally just changing some includes) or > Email it to me and I'll send you a custom diff. This only > applies to the current diff posted above and will not be > committed. > > That's all I could think of right now. I would like to commit this > soon as I have other things in the pipeline that build on top of it. > I should also mention that there are various other implications in > addition to the above but are more relevant to the developer than the > user, so they'll probably appear in a really large commit log. > > Reviews and constructive criticism is both welcomed and appreciated. > Please leave me in the To: line - although I am subscribed to > -current, I'd like to see feedback in my main box. > > Regards, > -- > Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org > TECHNOkRATIS Consulting Services * http://www.technokratis.com/ > > "It is impossible for anyone to begin to learn what he believes > he already knows." -- Epictetus (c.a.d. 55-c135) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 01:50:26 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBA7016A4CE for ; Tue, 16 Mar 2004 01:50:26 -0800 (PST) Received: from smtp02.syd.iprimus.net.au (smtp02.syd.iprimus.net.au [210.50.76.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1982F43D1F for ; Tue, 16 Mar 2004 01:50:26 -0800 (PST) (envelope-from tim@robbins.dropbear.id.au) Received: from robbins.dropbear.id.au (203.134.130.100) by smtp02.syd.iprimus.net.au (7.0.024) id 402CF87000A1FBBB; Tue, 16 Mar 2004 20:50:24 +1100 Received: by robbins.dropbear.id.au (Postfix, from userid 1000) id F0967415D; Tue, 16 Mar 2004 20:51:12 +1100 (EST) Date: Tue, 16 Mar 2004 20:51:12 +1100 From: Tim Robbins To: Bosko Milekic Message-ID: <20040316095112.GA33276@cat.robbins.dropbear.id.au> References: <20040316032029.GA3735@technokratis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040316032029.GA3735@technokratis.com> User-Agent: Mutt/1.4.1i cc: freebsd-arch@freebsd.org Subject: Re: mbuma: network buffers & UMA X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 09:50:27 -0000 On Mon, Mar 15, 2004 at 10:20:29PM -0500, Bosko Milekic wrote: > The post below has received exactly zero attention on the > freebsd-current mailing list. I'm assuming this is because there are > other exciting threads going on there right now, such as style debates > and lovely feature requests. :-) > > I would like to commit this RSN. I need feedback from the Release > Engineering people on how close we are to a RELEASE and if it's OK for > me to break netstat -m stats for a few weeks or, if not, what the > appropriate course of action is. I'm unable to test this right now, but I like the idea of using UMA for mbuf allocations. I also like the keg allocator idea - it will be useful in quite a few other places. Tim From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 06:22:52 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B373D16A4CE for ; Tue, 16 Mar 2004 06:22:52 -0800 (PST) Received: from smtp.distributel.net (cns2.distributel.NET [66.38.181.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4E79643D41 for ; Tue, 16 Mar 2004 06:22:52 -0800 (PST) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by smtp.distributel.net (8.12.6/8.12.6) with ESMTP id i2GEMoM2076810; Tue, 16 Mar 2004 09:22:51 -0500 (EST) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) i2GELA6p006848; Tue, 16 Mar 2004 09:21:10 -0500 (EST) (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost)i2GELAEG006847; Tue, 16 Mar 2004 09:21:10 -0500 (EST) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Tue, 16 Mar 2004 09:21:10 -0500 From: Bosko Milekic To: Doug Rabson Message-ID: <20040316142110.GA6802@technokratis.com> References: <1077137806.28133.10.camel@herring.nlsystems.com> <1077181355.28133.13.camel@herring.nlsystems.com> <20040316031702.GA3794@technokratis.com> <200403160914.51727.dfr@nlsystems.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200403160914.51727.dfr@nlsystems.com> User-Agent: Mutt/1.4.1i cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 14:22:52 -0000 On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > > > I think that this is a good path to go down, but I really don't > > > > think we're ready yet. I'd rather see energy spent protecting > > > > code than building more infrastructure. > > > > > > I completely agree. I was just musing about this as a future > > > addition to the locking toolbox. Its certainly not worth trying > > > before enough of the kernel is outside the giant lock to make it > > > worthwhile. > > > > As Jeff said and you agree, it is probably too early for this now. > > Also, I've looked at the paper you quote before SCO's announcement > > (which by the way I had no idea about until now), and I think we'll > > eventually do just as well in the common case without going to the > > RCU model at all. > > Its a pretty neat idea though. I like the sound of being able to e.g. > read from the namecache without needing to take an expensive lock. With > the way 5-CURRENT works, we would probably still need to suppress > context switching which is expensive on intel processors in the current > implementation. I guess that could be fixed using some kind of lazy-cli > scheme. Should be really easy to fix in the common case without having to get really fancy (e.g., just interlock against ourselves on a private per-thread flag) to merely delay cli until the first interrupt. We shouldn't need to mask out per CPU or anything fancy like that. > The main barrier here (apart from the need to push Giant down in more > places and stabilise the existing implementation) is IBM's patent. The > SCO IP claim (bogus as it is) only covers Sequent's original > implementation. Yeah, that sucks. -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/ "It is impossible for anyone to begin to learn what he believes he already knows." -- Epictetus (c.a.d. 55-c135) From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 06:53:13 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4992416A4CE for ; Tue, 16 Mar 2004 06:53:13 -0800 (PST) Received: from mail.qubesoft.com (gate.qubesoft.com [217.169.36.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id 696FA43D1D for ; Tue, 16 Mar 2004 06:53:12 -0800 (PST) (envelope-from dfr@nlsystems.com) Received: from bluebottle.qubesoft.com (bluebottle.qubesoft.com [192.168.1.2]) by mail.qubesoft.com (8.12.9/8.12.9) with ESMTP id i2GEr9kM048171; Tue, 16 Mar 2004 14:53:10 GMT (envelope-from dfr@nlsystems.com) Received: from builder02.qubesoft.com (builder02.qubesoft.com [192.168.1.8]) i2GEr9vG001092; Tue, 16 Mar 2004 14:53:09 GMT (envelope-from dfr@nlsystems.com) From: Doug Rabson To: Bosko Milekic In-Reply-To: <20040316142110.GA6802@technokratis.com> References: <1077137806.28133.10.camel@herring.nlsystems.com> <1077181355.28133.13.camel@herring.nlsystems.com> <20040316031702.GA3794@technokratis.com> <200403160914.51727.dfr@nlsystems.com> <20040316142110.GA6802@technokratis.com> Content-Type: text/plain Message-Id: <1079448788.10695.1.camel@builder02.qubesoft.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 16 Mar 2004 14:53:08 +0000 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 'clamd / ClamAV version 0.65', clamav-milter version '0.60p' cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 14:53:13 -0000 On Tue, 2004-03-16 at 14:21, Bosko Milekic wrote: > On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > > > > > I think that this is a good path to go down, but I really don't > > > > > think we're ready yet. I'd rather see energy spent protecting > > > > > code than building more infrastructure. > > > > > > > > I completely agree. I was just musing about this as a future > > > > addition to the locking toolbox. Its certainly not worth trying > > > > before enough of the kernel is outside the giant lock to make it > > > > worthwhile. > > > > > > As Jeff said and you agree, it is probably too early for this now. > > > Also, I've looked at the paper you quote before SCO's announcement > > > (which by the way I had no idea about until now), and I think we'll > > > eventually do just as well in the common case without going to the > > > RCU model at all. > > > > Its a pretty neat idea though. I like the sound of being able to e.g. > > read from the namecache without needing to take an expensive lock. With > > the way 5-CURRENT works, we would probably still need to suppress > > context switching which is expensive on intel processors in the current > > implementation. I guess that could be fixed using some kind of lazy-cli > > scheme. > > Should be really easy to fix in the common case without having to get > really fancy (e.g., just interlock against ourselves on a private > per-thread flag) to merely delay cli until the first interrupt. We > shouldn't need to mask out per CPU or anything fancy like that. I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu fields an interrupt and the soft cli flag is set, it just clears the interrupt flag in the trapframe and returns. It all works out the same in the end. > > > The main barrier here (apart from the need to push Giant down in more > > places and stabilise the existing implementation) is IBM's patent. The > > SCO IP claim (bogus as it is) only covers Sequent's original > > implementation. > > Yeah, that sucks. Agreed. From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:06:12 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A0D4816A4CE for ; Tue, 16 Mar 2004 08:06:12 -0800 (PST) Received: from mail1.speakeasy.net (mail1.speakeasy.net [216.254.0.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15F3443D2D for ; Tue, 16 Mar 2004 08:06:12 -0800 (PST) (envelope-from john@baldwin.cx) Received: (qmail 12782 invoked from network); 16 Mar 2004 16:06:09 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 16 Mar 2004 16:06:09 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2GG5m28065141; Tue, 16 Mar 2004 11:05:49 -0500 (EST) (envelope-from john@baldwin.cx) From: John Baldwin To: arch@FreeBSD.org Date: Tue, 16 Mar 2004 10:09:48 -0500 User-Agent: KMail/1.6 References: <200403160519.i2G5J0V6023193@urban> In-Reply-To: <200403160519.i2G5J0V6023193@urban> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403161009.48938.john@baldwin.cx> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: Seigo Tanimura Subject: Re: Is MTX_CONTESTED evil? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:06:12 -0000 On Tuesday 16 March 2004 12:19 am, Seigo Tanimura wrote: > _mtx_unlock_sleep() currently wakes up only one thread being blocked, > and leaves MTX_CONTESTED on a mutex. According to Solaris Internals, > that strategy adds an overhead to check for MTX_CONTESTED on a mutex, > even though it is not held by any thread. The thread waken up cannot > grab the mutex immediately by _obtain_lock() and have to go through > _mtx_lock_sleep(). The penalty tends to be large for a mutex with a > high contention, and we have at least one of such a mutex - Giant. > > What would it be like if we axed MTX_CONTEST and let > _mtx_unlock_sleep() wake up all of the blocked threads? We wouldn't be able to axe MTX_CONTEST. We also use it to determine on unlock if we can unlock easily or if we have waiters that we need to awake. The only way we might be able to axe MTX_CONTEST would be to penalize every unlock operation requiring a turnstile lookup (spin lock acquire/release + hash table lookup) even unlocks of an uncontested mutex. However, what I think you want to do is get rid of the mtx_lock == MTX_CONTESTED case and use turnstile_wakeup() rather than turnstile_signal()? Is that what you are asking? That is something we can try at some point in the future, but we would need to benchmark both ways. What we might can do is add a kernel option MUTEX_WAKE_ALL or some such that uses the Solaris behavior. Having it be an option like ADAPTIVE_MUTEXES makes it easier to benchmark both cases. -- John Baldwin <>< http://www.baldwin.cx/~john/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:07:03 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B6C0416A4CE for ; Tue, 16 Mar 2004 08:07:03 -0800 (PST) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 816A743D46 for ; Tue, 16 Mar 2004 08:07:03 -0800 (PST) (envelope-from john@baldwin.cx) Received: (qmail 20171 invoked from network); 16 Mar 2004 16:06:52 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 16 Mar 2004 16:06:52 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2GG5m29065141; Tue, 16 Mar 2004 11:06:01 -0500 (EST) (envelope-from john@baldwin.cx) From: John Baldwin To: arch@FreeBSD.org Date: Tue, 16 Mar 2004 10:12:09 -0500 User-Agent: KMail/1.6 References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> In-Reply-To: <1079448788.10695.1.camel@builder02.qubesoft.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403161012.09174.john@baldwin.cx> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: Bosko Milekic cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:07:04 -0000 On Tuesday 16 March 2004 09:53 am, Doug Rabson wrote: > On Tue, 2004-03-16 at 14:21, Bosko Milekic wrote: > > On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > > > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > I think that this is a good path to go down, but I really don't > > > > > > think we're ready yet. I'd rather see energy spent protecting > > > > > > code than building more infrastructure. > > > > > > > > > > I completely agree. I was just musing about this as a future > > > > > addition to the locking toolbox. Its certainly not worth trying > > > > > before enough of the kernel is outside the giant lock to make it > > > > > worthwhile. > > > > > > > > As Jeff said and you agree, it is probably too early for this now. > > > > Also, I've looked at the paper you quote before SCO's announcement > > > > (which by the way I had no idea about until now), and I think we'll > > > > eventually do just as well in the common case without going to the > > > > RCU model at all. > > > > > > Its a pretty neat idea though. I like the sound of being able to e.g. > > > read from the namecache without needing to take an expensive lock. With > > > the way 5-CURRENT works, we would probably still need to suppress > > > context switching which is expensive on intel processors in the current > > > implementation. I guess that could be fixed using some kind of lazy-cli > > > scheme. > > > > Should be really easy to fix in the common case without having to get > > really fancy (e.g., just interlock against ourselves on a private > > per-thread flag) to merely delay cli until the first interrupt. We > > shouldn't need to mask out per CPU or anything fancy like that. > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > fields an interrupt and the soft cli flag is set, it just clears the > interrupt flag in the trapframe and returns. It all works out the same > in the end. I have this partly implemented, but because the VM86 code really sucks (it just always enables interrupts) the invariants checks I have don't make it to single user mode. I haven't decided what to do about VM86 yet, and I also haven't handled the problem of switching away from interrupt context (for ithread preemption) and switching back and making that all work correctly w/o possibly dropping interrupts. -- John Baldwin <>< http://www.baldwin.cx/~john/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:07:03 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B6EA916A4CF for ; Tue, 16 Mar 2004 08:07:03 -0800 (PST) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E4AB43D48 for ; Tue, 16 Mar 2004 08:07:03 -0800 (PST) (envelope-from john@baldwin.cx) Received: (qmail 20171 invoked from network); 16 Mar 2004 16:06:52 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 16 Mar 2004 16:06:52 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2GG5m29065141; Tue, 16 Mar 2004 11:06:01 -0500 (EST) (envelope-from john@baldwin.cx) From: John Baldwin To: arch@FreeBSD.org Date: Tue, 16 Mar 2004 10:12:09 -0500 User-Agent: KMail/1.6 References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> In-Reply-To: <1079448788.10695.1.camel@builder02.qubesoft.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403161012.09174.john@baldwin.cx> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: Bosko Milekic cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:07:04 -0000 On Tuesday 16 March 2004 09:53 am, Doug Rabson wrote: > On Tue, 2004-03-16 at 14:21, Bosko Milekic wrote: > > On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > > > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > I think that this is a good path to go down, but I really don't > > > > > > think we're ready yet. I'd rather see energy spent protecting > > > > > > code than building more infrastructure. > > > > > > > > > > I completely agree. I was just musing about this as a future > > > > > addition to the locking toolbox. Its certainly not worth trying > > > > > before enough of the kernel is outside the giant lock to make it > > > > > worthwhile. > > > > > > > > As Jeff said and you agree, it is probably too early for this now. > > > > Also, I've looked at the paper you quote before SCO's announcement > > > > (which by the way I had no idea about until now), and I think we'll > > > > eventually do just as well in the common case without going to the > > > > RCU model at all. > > > > > > Its a pretty neat idea though. I like the sound of being able to e.g. > > > read from the namecache without needing to take an expensive lock. With > > > the way 5-CURRENT works, we would probably still need to suppress > > > context switching which is expensive on intel processors in the current > > > implementation. I guess that could be fixed using some kind of lazy-cli > > > scheme. > > > > Should be really easy to fix in the common case without having to get > > really fancy (e.g., just interlock against ourselves on a private > > per-thread flag) to merely delay cli until the first interrupt. We > > shouldn't need to mask out per CPU or anything fancy like that. > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > fields an interrupt and the soft cli flag is set, it just clears the > interrupt flag in the trapframe and returns. It all works out the same > in the end. I have this partly implemented, but because the VM86 code really sucks (it just always enables interrupts) the invariants checks I have don't make it to single user mode. I haven't decided what to do about VM86 yet, and I also haven't handled the problem of switching away from interrupt context (for ithread preemption) and switching back and making that all work correctly w/o possibly dropping interrupts. -- John Baldwin <>< http://www.baldwin.cx/~john/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:19:11 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0907716A4CE; Tue, 16 Mar 2004 08:19:11 -0800 (PST) Received: from mail.qubesoft.com (gate.qubesoft.com [217.169.36.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1660543D31; Tue, 16 Mar 2004 08:19:10 -0800 (PST) (envelope-from dfr@nlsystems.com) Received: from bluebottle.qubesoft.com (bluebottle.qubesoft.com [192.168.1.2]) by mail.qubesoft.com (8.12.9/8.12.9) with ESMTP id i2GGIokM050624; Tue, 16 Mar 2004 16:18:50 GMT (envelope-from dfr@nlsystems.com) Received: from builder02.qubesoft.com (builder02.qubesoft.com [192.168.1.8]) i2GGImvG002805; Tue, 16 Mar 2004 16:18:49 GMT (envelope-from dfr@nlsystems.com) From: Doug Rabson To: John Baldwin In-Reply-To: <200403161012.09174.john@baldwin.cx> References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> <200403161012.09174.john@baldwin.cx> Content-Type: text/plain Message-Id: <1079453928.10695.6.camel@builder02.qubesoft.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 16 Mar 2004 16:18:48 +0000 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 'clamd / ClamAV version 0.65', clamav-milter version '0.60p' cc: arch@FreeBSD.org cc: Bosko Milekic cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:19:11 -0000 On Tue, 2004-03-16 at 15:12, John Baldwin wrote: > On Tuesday 16 March 2004 09:53 am, Doug Rabson wrote: > > On Tue, 2004-03-16 at 14:21, Bosko Milekic wrote: > > > On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > > > > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > > > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > > I think that this is a good path to go down, but I really don't > > > > > > > think we're ready yet. I'd rather see energy spent protecting > > > > > > > code than building more infrastructure. > > > > > > > > > > > > I completely agree. I was just musing about this as a future > > > > > > addition to the locking toolbox. Its certainly not worth trying > > > > > > before enough of the kernel is outside the giant lock to make it > > > > > > worthwhile. > > > > > > > > > > As Jeff said and you agree, it is probably too early for this now. > > > > > Also, I've looked at the paper you quote before SCO's announcement > > > > > (which by the way I had no idea about until now), and I think we'll > > > > > eventually do just as well in the common case without going to the > > > > > RCU model at all. > > > > > > > > Its a pretty neat idea though. I like the sound of being able to e.g. > > > > read from the namecache without needing to take an expensive lock. With > > > > the way 5-CURRENT works, we would probably still need to suppress > > > > context switching which is expensive on intel processors in the current > > > > implementation. I guess that could be fixed using some kind of lazy-cli > > > > scheme. > > > > > > Should be really easy to fix in the common case without having to get > > > really fancy (e.g., just interlock against ourselves on a private > > > per-thread flag) to merely delay cli until the first interrupt. We > > > shouldn't need to mask out per CPU or anything fancy like that. > > > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > fields an interrupt and the soft cli flag is set, it just clears the > > interrupt flag in the trapframe and returns. It all works out the same > > in the end. > > I have this partly implemented, but because the VM86 code really sucks (it > just always enables interrupts) the invariants checks I have don't make it to > single user mode. I haven't decided what to do about VM86 yet, and I also > haven't handled the problem of switching away from interrupt context (for > ithread preemption) and switching back and making that all work correctly w/o > possibly dropping interrupts. I guess I did gloss over a lot of the 'interesting' bits like not losing edge triggered interrupts etc. Still, that part is well understood since its the core of the splX implementation in 4.x and before. I'm not quite sure what you are getting at wrt. ithread preemption? How does that interact with critical_enter/critical_exit? From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:19:11 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0907716A4CE; Tue, 16 Mar 2004 08:19:11 -0800 (PST) Received: from mail.qubesoft.com (gate.qubesoft.com [217.169.36.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1660543D31; Tue, 16 Mar 2004 08:19:10 -0800 (PST) (envelope-from dfr@nlsystems.com) Received: from bluebottle.qubesoft.com (bluebottle.qubesoft.com [192.168.1.2]) by mail.qubesoft.com (8.12.9/8.12.9) with ESMTP id i2GGIokM050624; Tue, 16 Mar 2004 16:18:50 GMT (envelope-from dfr@nlsystems.com) Received: from builder02.qubesoft.com (builder02.qubesoft.com [192.168.1.8]) i2GGImvG002805; Tue, 16 Mar 2004 16:18:49 GMT (envelope-from dfr@nlsystems.com) From: Doug Rabson To: John Baldwin In-Reply-To: <200403161012.09174.john@baldwin.cx> References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> <200403161012.09174.john@baldwin.cx> Content-Type: text/plain Message-Id: <1079453928.10695.6.camel@builder02.qubesoft.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 16 Mar 2004 16:18:48 +0000 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 'clamd / ClamAV version 0.65', clamav-milter version '0.60p' cc: arch@FreeBSD.org cc: Bosko Milekic cc: freebsd-arch@FreeBSD.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:19:11 -0000 On Tue, 2004-03-16 at 15:12, John Baldwin wrote: > On Tuesday 16 March 2004 09:53 am, Doug Rabson wrote: > > On Tue, 2004-03-16 at 14:21, Bosko Milekic wrote: > > > On Tue, Mar 16, 2004 at 09:14:51AM +0000, Doug Rabson wrote: > > > > On Tuesday 16 March 2004 03:17, Bosko Milekic wrote: > > > > > On Thu, Feb 19, 2004 at 09:02:35AM +0000, Doug Rabson wrote: > > > > > > On Wed, 2004-02-18 at 23:26, Jeff Roberson wrote: > > > > > > > I think that this is a good path to go down, but I really don't > > > > > > > think we're ready yet. I'd rather see energy spent protecting > > > > > > > code than building more infrastructure. > > > > > > > > > > > > I completely agree. I was just musing about this as a future > > > > > > addition to the locking toolbox. Its certainly not worth trying > > > > > > before enough of the kernel is outside the giant lock to make it > > > > > > worthwhile. > > > > > > > > > > As Jeff said and you agree, it is probably too early for this now. > > > > > Also, I've looked at the paper you quote before SCO's announcement > > > > > (which by the way I had no idea about until now), and I think we'll > > > > > eventually do just as well in the common case without going to the > > > > > RCU model at all. > > > > > > > > Its a pretty neat idea though. I like the sound of being able to e.g. > > > > read from the namecache without needing to take an expensive lock. With > > > > the way 5-CURRENT works, we would probably still need to suppress > > > > context switching which is expensive on intel processors in the current > > > > implementation. I guess that could be fixed using some kind of lazy-cli > > > > scheme. > > > > > > Should be really easy to fix in the common case without having to get > > > really fancy (e.g., just interlock against ourselves on a private > > > per-thread flag) to merely delay cli until the first interrupt. We > > > shouldn't need to mask out per CPU or anything fancy like that. > > > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > fields an interrupt and the soft cli flag is set, it just clears the > > interrupt flag in the trapframe and returns. It all works out the same > > in the end. > > I have this partly implemented, but because the VM86 code really sucks (it > just always enables interrupts) the invariants checks I have don't make it to > single user mode. I haven't decided what to do about VM86 yet, and I also > haven't handled the problem of switching away from interrupt context (for > ithread preemption) and switching back and making that all work correctly w/o > possibly dropping interrupts. I guess I did gloss over a lot of the 'interesting' bits like not losing edge triggered interrupts etc. Still, that part is well understood since its the core of the splX implementation in 4.x and before. I'm not quite sure what you are getting at wrt. ithread preemption? How does that interact with critical_enter/critical_exit? From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:35:53 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A810616A4CE; Tue, 16 Mar 2004 08:35:53 -0800 (PST) Received: from smtp.distributel.net (cns2.distributel.NET [66.38.181.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5793443D1F; Tue, 16 Mar 2004 08:35:53 -0800 (PST) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by smtp.distributel.net (8.12.6/8.12.6) with ESMTP id i2GGZeM2095507; Tue, 16 Mar 2004 11:35:40 -0500 (EST) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) i2GGXx6p007404; Tue, 16 Mar 2004 11:33:59 -0500 (EST) (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost)i2GGXx4p007403; Tue, 16 Mar 2004 11:33:59 -0500 (EST) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Tue, 16 Mar 2004 11:33:59 -0500 From: Bosko Milekic To: John Baldwin Message-ID: <20040316163359.GA7365@technokratis.com> References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> <200403161012.09174.john@baldwin.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200403161012.09174.john@baldwin.cx> User-Agent: Mutt/1.4.1i cc: arch@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:35:53 -0000 On Tue, Mar 16, 2004 at 10:12:09AM -0500, John Baldwin wrote: > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > fields an interrupt and the soft cli flag is set, it just clears the > > interrupt flag in the trapframe and returns. It all works out the same > > in the end. No because if you're in the middle of modifying a pcpu flag and you take an interrupt you may get scheduled over to another CPU. You would also have to temporarily pin down the thread to the current CPU unless you do it with an in-thread flag. By the way, the point of all this is that with a combination of soft-disables and short-term scheduler pinning we should be able to take the common memory allocation case in UMA out of any lock requirements (lockless in common case). > I have this partly implemented, but because the VM86 code really sucks (it > just always enables interrupts) the invariants checks I have don't make it to > single user mode. I haven't decided what to do about VM86 yet, and I also > haven't handled the problem of switching away from interrupt context (for > ithread preemption) and switching back and making that all work correctly w/o > possibly dropping interrupts. Do you have your current changes in a separate p4 repo somewhere? > -- > John Baldwin <>< http://www.baldwin.cx/~john/ > "Power Users Use the Power to Serve" = http://www.FreeBSD.org -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/ "It is impossible for anyone to begin to learn what he believes he already knows." -- Epictetus (c.a.d. 55-c135) From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 08:35:53 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A810616A4CE; Tue, 16 Mar 2004 08:35:53 -0800 (PST) Received: from smtp.distributel.net (cns2.distributel.NET [66.38.181.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5793443D1F; Tue, 16 Mar 2004 08:35:53 -0800 (PST) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by smtp.distributel.net (8.12.6/8.12.6) with ESMTP id i2GGZeM2095507; Tue, 16 Mar 2004 11:35:40 -0500 (EST) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) i2GGXx6p007404; Tue, 16 Mar 2004 11:33:59 -0500 (EST) (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost)i2GGXx4p007403; Tue, 16 Mar 2004 11:33:59 -0500 (EST) (envelope-from bmilekic@technokratis.com) X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Tue, 16 Mar 2004 11:33:59 -0500 From: Bosko Milekic To: John Baldwin Message-ID: <20040316163359.GA7365@technokratis.com> References: <1077137806.28133.10.camel@herring.nlsystems.com> <20040316142110.GA6802@technokratis.com> <1079448788.10695.1.camel@builder02.qubesoft.com> <200403161012.09174.john@baldwin.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200403161012.09174.john@baldwin.cx> User-Agent: Mutt/1.4.1i cc: arch@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 16:35:53 -0000 On Tue, Mar 16, 2004 at 10:12:09AM -0500, John Baldwin wrote: > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > fields an interrupt and the soft cli flag is set, it just clears the > > interrupt flag in the trapframe and returns. It all works out the same > > in the end. No because if you're in the middle of modifying a pcpu flag and you take an interrupt you may get scheduled over to another CPU. You would also have to temporarily pin down the thread to the current CPU unless you do it with an in-thread flag. By the way, the point of all this is that with a combination of soft-disables and short-term scheduler pinning we should be able to take the common memory allocation case in UMA out of any lock requirements (lockless in common case). > I have this partly implemented, but because the VM86 code really sucks (it > just always enables interrupts) the invariants checks I have don't make it to > single user mode. I haven't decided what to do about VM86 yet, and I also > haven't handled the problem of switching away from interrupt context (for > ithread preemption) and switching back and making that all work correctly w/o > possibly dropping interrupts. Do you have your current changes in a separate p4 repo somewhere? > -- > John Baldwin <>< http://www.baldwin.cx/~john/ > "Power Users Use the Power to Serve" = http://www.FreeBSD.org -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/ "It is impossible for anyone to begin to learn what he believes he already knows." -- Epictetus (c.a.d. 55-c135) From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 09:21:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B884016A4FB for ; Tue, 16 Mar 2004 09:21:10 -0800 (PST) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8BF6D43D31 for ; Tue, 16 Mar 2004 09:21:10 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: (qmail 16648 invoked from network); 16 Mar 2004 17:21:09 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 16 Mar 2004 17:21:09 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2GHL528065517; Tue, 16 Mar 2004 12:21:05 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Bosko Milekic Date: Tue, 16 Mar 2004 12:14:56 -0500 User-Agent: KMail/1.6 References: <1077137806.28133.10.camel@herring.nlsystems.com> <200403161012.09174.john@baldwin.cx> <20040316163359.GA7365@technokratis.com> In-Reply-To: <20040316163359.GA7365@technokratis.com> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200403161214.31276.john@baldwin.cx> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 17:21:10 -0000 On Tuesday 16 March 2004 11:33 am, Bosko Milekic wrote: > On Tue, Mar 16, 2004 at 10:12:09AM -0500, John Baldwin wrote: > > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > > fields an interrupt and the soft cli flag is set, it just clears the > > > interrupt flag in the trapframe and returns. It all works out the same > > > in the end. > > No because if you're in the middle of modifying a pcpu flag and you > take an interrupt you may get scheduled over to another CPU. You > would also have to temporarily pin down the thread to the current CPU > unless you do it with an in-thread flag. > > By the way, the point of all this is that with a combination of > soft-disables and short-term scheduler pinning we should be able to > take the common memory allocation case in UMA out of any lock > requirements (lockless in common case). > > > I have this partly implemented, but because the VM86 code really sucks > > (it just always enables interrupts) the invariants checks I have don't > > make it to single user mode. I haven't decided what to do about VM86 > > yet, and I also haven't handled the problem of switching away from > > interrupt context (for ithread preemption) and switching back and making > > that all work correctly w/o possibly dropping interrupts. > > Do you have your current changes in a separate p4 repo somewhere? The jhb_acpipci_crit branch. Like I said, though, it doesn't boot. There are lots of implementation notes in //depot/user/jhb/acpicpi_crit/notes. -- John Baldwin <>< http://www.baldwin.cx/~john/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 09:21:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B48A916A4E4 for ; Tue, 16 Mar 2004 09:21:10 -0800 (PST) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C0EB43D39 for ; Tue, 16 Mar 2004 09:21:10 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: (qmail 16648 invoked from network); 16 Mar 2004 17:21:09 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 16 Mar 2004 17:21:09 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2GHL528065517; Tue, 16 Mar 2004 12:21:05 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Bosko Milekic Date: Tue, 16 Mar 2004 12:14:56 -0500 User-Agent: KMail/1.6 References: <1077137806.28133.10.camel@herring.nlsystems.com> <200403161012.09174.john@baldwin.cx> <20040316163359.GA7365@technokratis.com> In-Reply-To: <20040316163359.GA7365@technokratis.com> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200403161214.31276.john@baldwin.cx> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: Read Copy Update X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 17:21:10 -0000 On Tuesday 16 March 2004 11:33 am, Bosko Milekic wrote: > On Tue, Mar 16, 2004 at 10:12:09AM -0500, John Baldwin wrote: > > > I was imagining a a pcpu flag which was a 'soft cli', i.e. if a cpu > > > fields an interrupt and the soft cli flag is set, it just clears the > > > interrupt flag in the trapframe and returns. It all works out the same > > > in the end. > > No because if you're in the middle of modifying a pcpu flag and you > take an interrupt you may get scheduled over to another CPU. You > would also have to temporarily pin down the thread to the current CPU > unless you do it with an in-thread flag. > > By the way, the point of all this is that with a combination of > soft-disables and short-term scheduler pinning we should be able to > take the common memory allocation case in UMA out of any lock > requirements (lockless in common case). > > > I have this partly implemented, but because the VM86 code really sucks > > (it just always enables interrupts) the invariants checks I have don't > > make it to single user mode. I haven't decided what to do about VM86 > > yet, and I also haven't handled the problem of switching away from > > interrupt context (for ithread preemption) and switching back and making > > that all work correctly w/o possibly dropping interrupts. > > Do you have your current changes in a separate p4 repo somewhere? The jhb_acpipci_crit branch. Like I said, though, it doesn't boot. There are lots of implementation notes in //depot/user/jhb/acpicpi_crit/notes. -- John Baldwin <>< http://www.baldwin.cx/~john/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 13:32:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BEBF816A4CE for ; Tue, 16 Mar 2004 13:32:10 -0800 (PST) Received: from carver.gumbysoft.com (carver.gumbysoft.com [66.220.23.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A2ACC43D1D for ; Tue, 16 Mar 2004 13:32:10 -0800 (PST) (envelope-from dwhite@gumbysoft.com) Received: by carver.gumbysoft.com (Postfix, from userid 1000) id 9571772DD2; Tue, 16 Mar 2004 13:32:10 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by carver.gumbysoft.com (Postfix) with ESMTP id 931E572DB5; Tue, 16 Mar 2004 13:32:10 -0800 (PST) Date: Tue, 16 Mar 2004 13:32:10 -0800 (PST) From: Doug White To: Seigo Tanimura In-Reply-To: <200403160519.i2G5J0V6023193@urban> Message-ID: <20040316133013.P42715@carver.gumbysoft.com> References: <200403160519.i2G5J0V6023193@urban> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Is MTX_CONTESTED evil? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 21:32:10 -0000 On Tue, 16 Mar 2004, Seigo Tanimura wrote: > What would it be like if we axed MTX_CONTEST and let > _mtx_unlock_sleep() wake up all of the blocked threads? Excuse my naievete, but this sounds like it would trigger a thundering-herd response on a heavily contested mutex (like Giant). That would be Bad. :) -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 11:58:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 843E216A4CE; Wed, 17 Mar 2004 11:58:10 -0800 (PST) Received: from gateway.posi.net (adsl-63-201-89-201.dsl.snfc21.pacbell.net [63.201.89.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5EA0E43D1D; Wed, 17 Mar 2004 11:58:10 -0800 (PST) (envelope-from kbyanc@posi.net) Received: from localhost (localhost [127.0.0.1]) by gateway.posi.net (Postfix) with ESMTP id BB6EE6A0505; Wed, 17 Mar 2004 11:58:59 -0800 (PST) Date: Wed, 17 Mar 2004 11:58:59 -0800 (PST) From: Kelly Yancey To: Tim Robbins In-Reply-To: <20040316095112.GA33276@cat.robbins.dropbear.id.au> Message-ID: <20040317115328.W44453@gateway.posi.net> References: <20040316032029.GA3735@technokratis.com> <20040316095112.GA33276@cat.robbins.dropbear.id.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Bosko Milekic cc: freebsd-arch@freebsd.org Subject: Re: mbuma: network buffers & UMA X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2004 19:58:10 -0000 On Tue, 16 Mar 2004, Tim Robbins wrote: > I'm unable to test this right now, but I like the idea of using UMA > for mbuf allocations. I also like the keg allocator idea - it will be > useful in quite a few other places. > Keg allocator? Isn't that a job for donations@ ? Kelly -- Kelly Yancey - kbyanc@{posi.net,FreeBSD.org} - kelly@nttmcl.com From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 21:21:55 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD7D216A4CE for ; Fri, 19 Mar 2004 21:21:55 -0800 (PST) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D2DF43D31 for ; Fri, 19 Mar 2004 21:21:55 -0800 (PST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de (cicely5.cicely.de [IPv6:3ffe:400:8d0:301:200:92ff:fe9b:20e7]) (authenticated bits=0) i2K5KuUS050307 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK) for ; Sat, 20 Mar 2004 06:21:01 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [IPv6:3ffe:400:8d0:301::12]) by cicely5.cicely.de (8.12.10/8.12.10) with ESMTP id i2K5K9hn021873 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 20 Mar 2004 06:20:09 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.12.11/8.12.11) with ESMTP id i2K5K95G034049 for ; Sat, 20 Mar 2004 06:20:09 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.12.11/8.12.11/Submit) id i2K5K8PL034048 for arch@freebsd.org; Sat, 20 Mar 2004 06:20:08 +0100 (CET) (envelope-from ticso) Date: Sat, 20 Mar 2004 06:20:08 +0100 From: Bernd Walter To: arch@freebsd.org Message-ID: <20040320052007.GC33602@cicely12.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD cicely12.cicely.de 5.2-CURRENT alpha User-Agent: Mutt/1.5.6i X-Spam-Status: No, hits=-4.9 required=3.0 tests=BAYES_00 autolearn=ham version=2.61 X-Spam-Report: * -4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on cicely5.cicely.de Subject: Opinions about using USB serial numbers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: ticso@cicely.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2004 05:21:55 -0000 I have a lot to do with systems using many USB devices of the same kind. Currently device are named acording to their probing order. That result in the problematic that device names can change after reboot or replugging in different order. I would like to add functionality for creating alias devnodes including the serial numbers (if the device has one setup). E.g. for a ucom(4) device you will get /dev/ucom0 and ucom.serialnumber. If the device has no serial numer there will be just /dev/ucom0 as befor. Another solution could be to create the nodes in a tree form: /dev/usbchannel0/hubport1/pubport2/ucom That would work for devices without serial numbers too, but I personally dislike this way. -- B.Walter BWCT http://www.bwct.de ticso@bwct.de info@bwct.de From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 17:43:15 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B341516A4CE for ; Sat, 20 Mar 2004 17:43:15 -0800 (PST) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4515843D41 for ; Sat, 20 Mar 2004 17:43:15 -0800 (PST) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.10/8.12.9) with ESMTP id i2L1h9kj059741; Sat, 20 Mar 2004 18:43:10 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sat, 20 Mar 2004 18:43:36 -0700 (MST) Message-Id: <20040320.184336.88475380.imp@bsdimp.com> To: ticso@cicely.de, ticso@cicely12.cicely.de From: "M. Warner Losh" In-Reply-To: <20040320052007.GC33602@cicely12.cicely.de> References: <20040320052007.GC33602@cicely12.cicely.de> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org Subject: Re: Opinions about using USB serial numbers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2004 01:43:15 -0000 In message: <20040320052007.GC33602@cicely12.cicely.de> Bernd Walter writes: : I have a lot to do with systems using many USB devices of the same : kind. : Currently device are named acording to their probing order. : That result in the problematic that device names can change after : reboot or replugging in different order. : : I would like to add functionality for creating alias devnodes : including the serial numbers (if the device has one setup). : E.g. for a ucom(4) device you will get /dev/ucom0 and ucom.serialnumber. : If the device has no serial numer there will be just /dev/ucom0 : as befor. : : Another solution could be to create the nodes in a tree form: : /dev/usbchannel0/hubport1/pubport2/ucom : That would work for devices without serial numbers too, but I : personally dislike this way. I'd rather that we have a generic binding of a device instance number to a pnp location. However, having said that, you can use devd to extract the information from the node and then create a symbolic link... Warner From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 23:15:00 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF79316A4CE for ; Sat, 20 Mar 2004 23:15:00 -0800 (PST) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id EDDE643D2F for ; Sat, 20 Mar 2004 23:14:59 -0800 (PST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de (cicely5.cicely.de [IPv6:3ffe:400:8d0:301:200:92ff:fe9b:20e7]) (authenticated bits=0) i2L7CHUS094930 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sun, 21 Mar 2004 08:12:19 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [IPv6:3ffe:400:8d0:301::12]) by cicely5.cicely.de (8.12.10/8.12.10) with ESMTP id i2L7B3hn032151 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 21 Mar 2004 08:11:04 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.12.11/8.12.11) with ESMTP id i2L7B3Ok063365; Sun, 21 Mar 2004 08:11:03 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.12.11/8.12.11/Submit) id i2L7B19s063364; Sun, 21 Mar 2004 08:11:01 +0100 (CET) (envelope-from ticso) Date: Sun, 21 Mar 2004 08:11:01 +0100 From: Bernd Walter To: "M. Warner Losh" Message-ID: <20040321071100.GH33602@cicely12.cicely.de> References: <20040320052007.GC33602@cicely12.cicely.de> <20040320.184336.88475380.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040320.184336.88475380.imp@bsdimp.com> X-Operating-System: FreeBSD cicely12.cicely.de 5.2-CURRENT alpha User-Agent: Mutt/1.5.6i X-Spam-Status: No, hits=-4.9 required=3.0 tests=BAYES_00 autolearn=ham version=2.61 X-Spam-Report: * -4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on cicely5.cicely.de cc: ticso@cicely12.cicely.de cc: arch@freebsd.org cc: ticso@cicely.de Subject: Re: Opinions about using USB serial numbers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: ticso@cicely.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2004 07:15:01 -0000 On Sat, Mar 20, 2004 at 06:43:36PM -0700, M. Warner Losh wrote: > In message: <20040320052007.GC33602@cicely12.cicely.de> > Bernd Walter writes: > : I have a lot to do with systems using many USB devices of the same > : kind. > : Currently device are named acording to their probing order. > : That result in the problematic that device names can change after > : reboot or replugging in different order. > : > : I would like to add functionality for creating alias devnodes > : including the serial numbers (if the device has one setup). > : E.g. for a ucom(4) device you will get /dev/ucom0 and ucom.serialnumber. > : If the device has no serial numer there will be just /dev/ucom0 > : as befor. > : > : Another solution could be to create the nodes in a tree form: > : /dev/usbchannel0/hubport1/pubport2/ucom > : That would work for devices without serial numbers too, but I > : personally dislike this way. > > I'd rather that we have a generic binding of a device instance number > to a pnp location. Currently we have non really working - even cam binding is evil because it's a boot time only. > However, having said that, you can use devd to extract the information > from the node and then create a symbolic link... OK - lets try it with devd. Usefull devd support is required anyway. That's what I get on plugging in a device: Processing event '? at on uhub2' Pushing table Processing nomatch event Popping table Processing event '+ubser0 at on uhub2' Pushing table device-name=ubser0 Processing attach event I think the '?' one is because no driver attaches at device level. ubser is an interface level driver as most USB ones. What I need is the following for devices: vendor-id, device-id, device-class, device-protocoll, vendor-string, device-string, serial-id. At interface level additionally: interface-class, interface-subclass, interface-protocoll, interface-string. I've seen some places where bus_child_pnpinfo_str() and bus_child_location_str() was used for extended informations. Is this all I have to implement? Can I put the above data in it or are there any restrictions? It seems that dv_pnpinfo[128] could be to small for all of them. We can discuss if strings are required, but also USB serial number is a (unicode)string and strings can be 254 bytes long. What about whitespace in strings? Does usbd still serve any usefull purpose after having the above data via devctl? -- B.Walter BWCT http://www.bwct.de ticso@bwct.de info@bwct.de