From owner-freebsd-arch@FreeBSD.ORG Sun Sep 21 10:30:00 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3BA0616A4B3 for ; Sun, 21 Sep 2003 10:30:00 -0700 (PDT) Received: from blake.polstra.com (mail.polstra.com [206.213.73.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id F048E43F75 for ; Sun, 21 Sep 2003 10:29:58 -0700 (PDT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [206.213.73.20]) by blake.polstra.com (8.12.9/8.12.9) with ESMTP id h8LHTkIL021458; Sun, 21 Sep 2003 10:29:46 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20030920.214835.101559646.imp@bsdimp.com> Date: Sun, 21 Sep 2003 10:29:46 -0700 (PDT) From: John Polstra To: "M. Warner Losh" X-Bogosity: No, tests=bogofilter, spamicity=0.493654, version=0.14.5 cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Sep 2003 17:30:00 -0000 On 21-Sep-2003 M. Warner Losh wrote: > John Polstra writes: >: there will be no link changes except at bootstrap and shutdown. > > For server machines. For laptops, these events happen more often. > However, for most laptops, the rate that they happen is typically on > less than 1/hour. Still rare enough to not worry about optimizing it > and your other points are good. I just wanted to make sure that it > wasn't optimized to the point where disconnecting the cable from the > laptop to move it accross the room, or another room doesn't cause huge > problems. Don't worry, I have a laptop too. :-) John From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 02:10:51 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D8BA716A4C0 for ; Mon, 22 Sep 2003 02:10:51 -0700 (PDT) Received: from herring.nlsystems.com (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D20443FA3 for ; Mon, 22 Sep 2003 02:10:50 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from [10.0.0.2] (herring.nlsystems.com [10.0.0.2]) by herring.nlsystems.com (8.12.9/8.12.8) with ESMTP id h8M9AbY9065337 for ; Mon, 22 Sep 2003 10:10:38 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: arch@freebsd.org Content-Type: text/plain Message-Id: <1064221837.15078.14.camel@herring.nlsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 22 Sep 2003 10:10:37 +0100 Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-2.3 required=5.0 tests=EXCUSE_3,USER_AGENT_XIMIAN version=2.55 X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) Subject: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 09:10:52 -0000 I believe that I have the kobj multiple inheritance changes about ready for committing now. I have locked up the class handling in kobj and I've re-done the method dispatch so that it is MP-safe without needing locks (I would appreciate a close look at that part by another pair of eyes). You can get the current state of this patch at http://people.freebsd.org/~dfr/kobj-mi-22092003.diff. I've included in this patch a couple of uses of the two main new features. I have changed the cardbus driver so that it derives from the pci driver. This allows many pci methods to be removed from the cardbus method table and should allow many of those methods to be staticised in the pci driver again. I have also edited a bunch of pci drivers with explicit cardbus attachments so that they just list as pci drivers (where the cardbus attachment is identical to the pci attachment). This demonstrates the other inheritance feature which searches for drivers in both the bus devclass and the bus devclass' parent. This effectively allows all pci drivers to get into the cardbus probe. If a particular driver needs to treat its cardbus attachment specially, it can still do this by adding a special cardbus driver (e.g. with a cardbus specific probe or attach method) to the cardbus devclass (exactly as it does now). From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 07:11:35 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D10F516A4B3 for ; Mon, 22 Sep 2003 07:11:35 -0700 (PDT) Received: from mailhub.fokus.fraunhofer.de (mailhub.fokus.fraunhofer.de [193.174.154.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 64CE143FEC for ; Mon, 22 Sep 2003 07:11:34 -0700 (PDT) (envelope-from brandt@fokus.fraunhofer.de) Received: from beagle (beagle [193.175.132.100])h8MEBEU28381; Mon, 22 Sep 2003 16:11:14 +0200 (MEST) Date: Mon, 22 Sep 2003 16:11:14 +0200 (CEST) From: Harti Brandt To: "M. Warner Losh" In-Reply-To: <20030920.214835.101559646.imp@bsdimp.com> Message-ID: <20030922160106.Y6621@beagle.fokus.fraunhofer.de> References: <20030920141158.B97439@xorpc.icir.org> <20030920.214835.101559646.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: jdp@polstra.com cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: harti@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 14:11:35 -0000 On Sat, 20 Sep 2003, M. Warner Losh wrote: MWL>In message: MWL> John Polstra writes: MWL>: there will be no link changes except at bootstrap and shutdown. MWL> MWL>For server machines. For laptops, these events happen more often. MWL>However, for most laptops, the rate that they happen is typically on MWL>less than 1/hour. Still rare enough to not worry about optimizing it MWL>and your other points are good. I just wanted to make sure that it MWL>wasn't optimized to the point where disconnecting the cable from the MWL>laptop to move it accross the room, or another room doesn't cause huge MWL>problems. Perhaps the polling should be configurable., I struggled with this a year ago in the xl driver. I have an application that does real-time satellite simulation over two ethernet links with HZ=10000. This works really perfect (timing errors are not larger than 200usecs) except for the MII polling. With help from msilby we could cut down the mii polling delay from 8msecs to below 1msec. But, because that's still too much for my application, I have simply commented out the polling calls in the mii source. I suppose there are other application (servers) where one could simply switch them off. This could perhaps be done with a sysctl in mii(4). harti -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.fraunhofer.de, harti@freebsd.org From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 08:47:24 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E06716A4BF for ; Mon, 22 Sep 2003 08:47:24 -0700 (PDT) Received: from pimout2-ext.prodigy.net (pimout2-ext.prodigy.net [207.115.63.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 812244400E for ; Mon, 22 Sep 2003 08:46:36 -0700 (PDT) (envelope-from mweinstein@sbcglobal.net) Received: from sbcglobal.net (adsl-67-118-118-40.dsl.snfc21.pacbell.net [67.118.118.40])h8MFkYhH118104 for ; Mon, 22 Sep 2003 11:46:35 -0400 Message-ID: <3F6F1926.9060206@sbcglobal.net> Date: Mon, 22 Sep 2003 08:45:42 -0700 From: "Mark R. Weinstein" User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: BSD Versions -- Older X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 15:47:24 -0000 I am doing some research on the development of the BSD Unix operating system, and I am looking for some older (i.e. BSD 4.2 or 4.3) versions of BSD from the mid to late 1980's. Do you by chance know where I could obtain those older versions (the platform is not important)? Thanks in advance, Mark. From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 08:50:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 098BD16A4B3; Mon, 22 Sep 2003 08:50:36 -0700 (PDT) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1DFDB43F3F; Mon, 22 Sep 2003 08:50:33 -0700 (PDT) (envelope-from sam@errno.com) Received: from melange.errno.com (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.9) with ESMTP id h8MFoG0x029598 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Mon, 22 Sep 2003 08:50:19 -0700 (PDT) (envelope-from sam@errno.com) Date: Mon, 22 Sep 2003 08:50:15 -0700 From: Sam Leffler To: harti@freebsd.org, "M. Warner Losh" Message-ID: <839561335.1064220615@melange.errno.com> In-Reply-To: <20030922160106.Y6621@beagle.fokus.fraunhofer.de> References: <20030920141158.B97439@xorpc.icir.org> <20030920.214835.101559646.imp@bsdimp.com> <20030922160106.Y6621@beagle.fokus.fraunhofer.de> X-Mailer: Mulberry/3.0.3 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline cc: freebsd-arch@freebsd.org cc: jdp@polstra.com Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 15:50:36 -0000 > On Sat, 20 Sep 2003, M. Warner Losh wrote: > > MWL>In message: > MWL> John Polstra writes: > MWL>: there will be no link changes except at bootstrap and shutdown. > MWL> > MWL>For server machines. For laptops, these events happen more often. > MWL>However, for most laptops, the rate that they happen is typically on > MWL>less than 1/hour. Still rare enough to not worry about optimizing it > MWL>and your other points are good. I just wanted to make sure that it > MWL>wasn't optimized to the point where disconnecting the cable from the > MWL>laptop to move it across the room, or another room doesn't cause huge > MWL>problems. > > Perhaps the polling should be configurable., I struggled with this a year > ago in the xl driver. I have an application that does real-time satellite > simulation over two ethernet links with HZ=10000. This works really > perfect (timing errors are not larger than 200usecs) except for the MII > polling. With help from msilby we could cut down the mii polling delay > from 8msecs to below 1msec. But, because that's still too much for my > application, I have simply commented out the polling calls in the mii > source. I suppose there are other application (servers) where one could > simply switch them off. This could perhaps be done with a sysctl in > mii(4). I don't believe removing functionality is the right approach (though for specialized applications it might be what you have to do). I brought up the issue because it is widespread and has noticeable impact on performance. Some of this is a byproduct of the semi-mechanical way in which drivers have been converted from spl's to mutex's. It also appears some of the current work can be removed entirely for certain devices. I think we can move the mii bus polling/tick processing out from under the driver lock so these operations do not interfere with interrupt processing. But I'm not sure if this device-dependent; hence I didn't just add a lock to mii and sweep the drivers. I'm willing to add mii locking but can't deal with all the drivers. Before I do something in mii I'd like to understand better whether it's worthwhile or whether we'll still have to grab the driver lock to do the work. I suppose if I just add locking to mii then those drivers that are unchanged will just incur double locking until they are "fixed". Sam From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 08:52:31 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADD9916A4C2 for ; Mon, 22 Sep 2003 08:52:31 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 96A6E43FE0 for ; Mon, 22 Sep 2003 08:52:30 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9p1/8.12.3) with ESMTP id h8MFqTGA041818; Mon, 22 Sep 2003 09:52:29 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Sep 2003 09:52:25 -0600 (MDT) Message-Id: <20030922.095225.85015472.imp@bsdimp.com> To: dfr@nlsystems.com From: "M. Warner Losh" In-Reply-To: <1064221837.15078.14.camel@herring.nlsystems.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 15:52:31 -0000 In message: <1064221837.15078.14.camel@herring.nlsystems.com> Doug Rabson writes: : This effectively allows all pci : drivers to get into the cardbus probe. If a particular driver needs to : treat its cardbus attachment specially, it can still do this by adding a : special cardbus driver (e.g. with a cardbus specific probe or attach : method) to the cardbus devclass (exactly as it does now). So if there's devices that can only be "base" pci, and have issues with all other types of pci-like buses, is there a way to say "only on pci bus, but none of the derived buses"? Or is it better to list those derived buses that are known to cause problems? I'd imagine that these devices would be rare, but I've worked on one.... Also, we're violating the PC Card spec by not matching the CIS values, but reading the vendor/device instead. Technically, this is a violation and those registers aren't reqiured to be defined. So far, nobody has showed up with devices that don't have them, but I thought I'd point this out. It has been theorized that this is because so many designs share silicon with PCI. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 09:09:32 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FC3016A4B3 for ; Mon, 22 Sep 2003 09:09:32 -0700 (PDT) Received: from mail.qubesoft.com (gate.qubesoft.com [217.169.36.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2079443FE9 for ; Mon, 22 Sep 2003 09:09:31 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from bluebottle.qubesoft.com (bluebottle.qubesoft.com [192.168.1.2]) by mail.qubesoft.com (8.12.9/8.12.9) with ESMTP id h8MG93FH010108; Mon, 22 Sep 2003 17:09:04 +0100 (BST) (envelope-from dfr@nlsystems.com) Received: from builder02.qubesoft.com (builder02.qubesoft.com [192.168.1.8]) h8MG8qAc094125; Mon, 22 Sep 2003 17:09:03 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: "M. Warner Losh" In-Reply-To: <20030922.095225.85015472.imp@bsdimp.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <20030922.095225.85015472.imp@bsdimp.com> Content-Type: text/plain Message-Id: <1064246931.26368.15.camel@builder02.qubesoft.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 22 Sep 2003 17:08:52 +0100 Content-Transfer-Encoding: 7bit cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 16:09:32 -0000 On Mon, 2003-09-22 at 16:52, M. Warner Losh wrote: > In message: <1064221837.15078.14.camel@herring.nlsystems.com> > Doug Rabson writes: > : This effectively allows all pci > : drivers to get into the cardbus probe. If a particular driver needs to > : treat its cardbus attachment specially, it can still do this by adding a > : special cardbus driver (e.g. with a cardbus specific probe or attach > : method) to the cardbus devclass (exactly as it does now). > > So if there's devices that can only be "base" pci, and have issues > with all other types of pci-like buses, is there a way to say "only > on pci bus, but none of the derived buses"? Or is it better to list > those derived buses that are known to cause problems? I'd imagine > that these devices would be rare, but I've worked on one.... If there is a device which only ever appears in base pci physically, then, trivially, the driver for that hardware will never match to a cardbus device (since there is no physical manifestation of a cardbus version of the hardware). If there is cardbus hardware which we have a base pci driver for and which causes real problems when it attaches to cardbus hardware, then I guess you could always include a dummy 'do nothing' cardbus attachment which would match first. Remember that cardbus-specific drivers are searched before we fall back to pci-generic drivers so you can always win the driver election with a cardbus-specific driver. > > Also, we're violating the PC Card spec by not matching the CIS values, > but reading the vendor/device instead. Technically, this is a > violation and those registers aren't reqiured to be defined. So far, > nobody has showed up with devices that don't have them, but I thought > I'd point this out. It has been theorized that this is because so > many designs share silicon with PCI. If any hardware which doesn't support vendor/device ever appears, then a driver for it would need a cardbus-specific attachment. This can be pretty simple: static device_method_t foo_pci_methods[] { DEVMETHOD(device_probe, foo_pci_probe), DEVMETHOD(device_attach, foo_pci_attach), ... { 0, 0 } }; DEFINE_CLASS(foo, foo_pci_driver, foo_pci_methods, sizeof(struct foo_softc)); /* override just the probe for cardbus */ static device_method_t foo_cardbus_methods[] { DEVMETHOD(device_probe, foo_cardbus_probe), { 0, 0 } } DEFINE_CLASS_INHERITS1(foo, foo_cardbus_driver, foo_cardbus_methods, sizeof(struct foo_softc), foo_pci_driver); From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 09:11:53 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2347716A4B3 for ; Mon, 22 Sep 2003 09:11:53 -0700 (PDT) Received: from gidgate.gid.co.uk (gid.co.uk [194.32.164.225]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15CA243FF3 for ; Mon, 22 Sep 2003 09:11:45 -0700 (PDT) (envelope-from rb@gid.co.uk) Received: (from rb@localhost) by gidgate.gid.co.uk (8.11.7/8.11.6) id h8MGBXb03964; Mon, 22 Sep 2003 17:11:33 +0100 (BST) (envelope-from rb) Message-Id: <4.3.2.7.2.20030922171105.02d79e50@gid.co.uk> X-Sender: rbmail@gid.co.uk X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 22 Sep 2003 17:11:31 +0100 To: "Mark R. Weinstein" , freebsd-arch@freebsd.org From: Bob Bishop In-Reply-To: <3F6F1926.9060206@sbcglobal.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: Re: BSD Versions -- Older X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 16:11:53 -0000 Hi, See http://www.mckusick.com/csrg/index.html At 16:45 22/9/03, Mark R. Weinstein wrote: >I am doing some research on the development of the BSD Unix operating >system, and I am looking for some older (i.e. BSD 4.2 or 4.3) versions of >BSD from the mid to late 1980's. Do you by chance know where I could >obtain those older versions (the platform is not important)? >Thanks in advance, Mark. > >_______________________________________________ >freebsd-arch@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-arch >To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -- Bob Bishop +44 (0)118 977 4017 rb@gid.co.uk fax +44 (0)118 989 4254 From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 12:46:59 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4736216A4B3 for ; Mon, 22 Sep 2003 12:46:59 -0700 (PDT) Received: from magic.adaptec.com (magic-mail.adaptec.com [216.52.22.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7080D43FFD for ; Mon, 22 Sep 2003 12:46:56 -0700 (PDT) (envelope-from gibbs@scsiguy.com) Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11]) by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h8MJkuR28469; Mon, 22 Sep 2003 12:46:56 -0700 Received: from [10.100.253.70] (aslan.btc.adaptec.com [10.100.253.70]) by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id MAA26589; Mon, 22 Sep 2003 12:46:55 -0700 (PDT) Date: Mon, 22 Sep 2003 13:50:06 -0600 From: "Justin T. Gibbs" To: Doug Rabson , arch@freebsd.org Message-ID: <1423490000.1064260204@aslan.btc.adaptec.com> In-Reply-To: <1064221837.15078.14.camel@herring.nlsystems.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> X-Mailer: Mulberry/3.1.0b6 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Justin T. Gibbs" List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 19:46:59 -0000 > I believe that I have the kobj multiple inheritance changes about ready > for committing now. I have locked up the class handling in kobj and I've > re-done the method dispatch so that it is MP-safe without needing locks > (I would appreciate a close look at that part by another pair of eyes). I've only just glanced at these patches, but I don't see how the method cache is now MP safe. Aren't you still vulnerable to a cache collision from two different threads performing an operation on the same class? I still believe that the concept of inherited interfaces is better way to achieve multiple inheritance. The methods I may want to inherit need not be associated with what we currently call a device class. The nice thing about your approach is that it doesn't require a massive rototilling of the drivers, but I fear it doesn't go far enough toward providing flexible inheritence. -- Justin From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 13:11:40 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D27016A4B3 for ; Mon, 22 Sep 2003 13:11:40 -0700 (PDT) Received: from herring.nlsystems.com (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5E6743FE0 for ; Mon, 22 Sep 2003 13:11:38 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from [10.0.0.2] (herring.nlsystems.com [10.0.0.2]) by herring.nlsystems.com (8.12.9/8.12.8) with ESMTP id h8MKBMY9075986; Mon, 22 Sep 2003 21:11:22 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: "Justin T. Gibbs" In-Reply-To: <1423490000.1064260204@aslan.btc.adaptec.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <1423490000.1064260204@aslan.btc.adaptec.com> Content-Type: text/plain Message-Id: <1064261482.68463.13.camel@herring.nlsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 22 Sep 2003 21:11:22 +0100 Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-4.9 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_XIMIAN version=2.55 X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 20:11:40 -0000 On Mon, 2003-09-22 at 20:50, Justin T. Gibbs wrote: > > I believe that I have the kobj multiple inheritance changes about ready > > for committing now. I have locked up the class handling in kobj and I've > > re-done the method dispatch so that it is MP-safe without needing locks > > (I would appreciate a close look at that part by another pair of eyes). > > I've only just glanced at these patches, but I don't see how the > method cache is now MP safe. Aren't you still vulnerable to a cache > collision from two different threads performing an operation on the > same class? Thats the cunning part :-). The key is that the cache entries now point directly at the method structures instead of being copies of the structure. The main race condition theoretically present in the old code was that one thread could read a cache entry part way through that entry being updated by another thread. Since the cache entries are now simple pointers, that can't happen. The other main change is that kobj_lookup_method will return the method structure pointer as well as updating the cache entry. The caller only uses the pointer returned from kobj_lookup_method - this ensures that the correct method is called even if another thread re-writes the cache entry with a different value, e.g. if two methods hash to the same cache entry. For the more common case of two threads independantly looking up the same method, both will write the exact same value to the cache. > I still believe that the concept of inherited interfaces is better > way to achieve multiple inheritance. The methods I may want to > inherit need not be associated with what we currently call a device > class. The nice thing about your approach is that it doesn't require > a massive rototilling of the drivers, but I fear it doesn't go far > enough toward providing flexible inheritence. It was the mass rototilling which I wanted to avoid. It still seems like a pretty good thing to maintain some kind of API compatibility between FreeBSD 4, 5 and 6 and this method can support that. Your original scheme was 'single inheritance of multiple interfaces'. This patch can support that since you can derive from as many base classes as you like - each base class will be looked at if the previous class doesn't find a match. From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 13:42:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 97CE016A4B3 for ; Mon, 22 Sep 2003 13:42:28 -0700 (PDT) Received: from magic.adaptec.com (magic-mail.adaptec.com [216.52.22.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id E737644011 for ; Mon, 22 Sep 2003 13:42:26 -0700 (PDT) (envelope-from gibbs@scsiguy.com) Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11]) by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h8MKgQR10754; Mon, 22 Sep 2003 13:42:26 -0700 Received: from [10.100.253.70] (aslan.btc.adaptec.com [10.100.253.70]) by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id NAA20588; Mon, 22 Sep 2003 13:42:26 -0700 (PDT) Date: Mon, 22 Sep 2003 14:45:40 -0600 From: "Justin T. Gibbs" To: Doug Rabson Message-ID: <1448210000.1064263540@aslan.btc.adaptec.com> In-Reply-To: <1064261482.68463.13.camel@herring.nlsystems.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <1423490000.1064260204@aslan.btc.adaptec.com> <1064261482.68463.13.camel@herring.nlsystems.com> X-Mailer: Mulberry/3.1.0b6 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Justin T. Gibbs" List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 20:42:28 -0000 > On Mon, 2003-09-22 at 20:50, Justin T. Gibbs wrote: >> > I believe that I have the kobj multiple inheritance changes about ready >> > for committing now. I have locked up the class handling in kobj and I've >> > re-done the method dispatch so that it is MP-safe without needing locks >> > (I would appreciate a close look at that part by another pair of eyes). >> >> I've only just glanced at these patches, but I don't see how the >> method cache is now MP safe. Aren't you still vulnerable to a cache >> collision from two different threads performing an operation on the >> same class? > > Thats the cunning part :-). The key is that the cache entries now point > directly at the method structures instead of being copies of the > structure. The main race condition theoretically present in the old code > was that one thread could read a cache entry part way through that entry > being updated by another thread. Since the cache entries are now simple > pointers, that can't happen. The code assumes that pointer accesses are atomic, which I didn't think was guaranteed on all machines we support. That's why I didn't think it was safe. >> I still believe that the concept of inherited interfaces is better >> way to achieve multiple inheritance. The methods I may want to >> inherit need not be associated with what we currently call a device >> class. The nice thing about your approach is that it doesn't require >> a massive rototilling of the drivers, but I fear it doesn't go far >> enough toward providing flexible inheritence. > > It was the mass rototilling which I wanted to avoid. It still seems like > a pretty good thing to maintain some kind of API compatibility between > FreeBSD 4, 5 and 6 and this method can support that. Your original > scheme was 'single inheritance of multiple interfaces'. This patch can > support that since you can derive from as many base classes as you like > - each base class will be looked at if the previous class doesn't find a > match. There were quite a few differences: 1) Inheritence was not limited to only inheriting from a base interface. The method lookup traversed all the way to the root. 2) Default methods were removed and classes were forced to explicitly list supported interfaces. I found this to be much less confusing. 3) Short-hand for inheriting all of the interfaces of a "parent class" were provided. This just meant having the ability to list classes to inherit from so that the kobj class compiling code could iterate through all the interfaces of specified classes. 4) The method cache was removed in favor of a direct indexing into the interface's static method table. Interface lookup was explicit, the hope being that one interface lookup could be amortized over multiple method calls. This would allow using kobj interfaces for more heavy weight tasks such as exporting the correct XPT interface in CAM to low-level drivers (FC, SPI, SATA, SAS, etc). 5) I also planned to add a way to do "super" invocations. This would cut down on lots of the bloat in things like cardbus. The goal was to be able to do things like, using cardbus as an example, inheret all of the interfaces of PCI adding in the CIS parsing interface shared with pccard. CIS parsing is just a list of methods that doesn't make sense to be a "driver class" on its own. I guess I have bigger dreams than most for how to use newbus... Do you also have a proposal for handling ivar in the multiple-inheritance scheme? This is required to make multiple inheritance really useful. -- Justin From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 14:31:30 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CE2616A4BF for ; Mon, 22 Sep 2003 14:31:30 -0700 (PDT) Received: from herring.nlsystems.com (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id 68FD743FB1 for ; Mon, 22 Sep 2003 14:31:22 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from [10.0.0.2] (herring.nlsystems.com [10.0.0.2]) by herring.nlsystems.com (8.12.9/8.12.8) with ESMTP id h8MLV9Y9076333; Mon, 22 Sep 2003 22:31:10 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: "Justin T. Gibbs" In-Reply-To: <1448210000.1064263540@aslan.btc.adaptec.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <1423490000.1064260204@aslan.btc.adaptec.com> <1064261482.68463.13.camel@herring.nlsystems.com> <1448210000.1064263540@aslan.btc.adaptec.com> Content-Type: text/plain Message-Id: <1064266269.68463.42.camel@herring.nlsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 22 Sep 2003 22:31:09 +0100 Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-4.9 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_XIMIAN version=2.55 X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 21:31:30 -0000 On Mon, 2003-09-22 at 21:45, Justin T. Gibbs wrote: > > On Mon, 2003-09-22 at 20:50, Justin T. Gibbs wrote: > >> > I believe that I have the kobj multiple inheritance changes about ready > >> > for committing now. I have locked up the class handling in kobj and I've > >> > re-done the method dispatch so that it is MP-safe without needing locks > >> > (I would appreciate a close look at that part by another pair of eyes). > >> > >> I've only just glanced at these patches, but I don't see how the > >> method cache is now MP safe. Aren't you still vulnerable to a cache > >> collision from two different threads performing an operation on the > >> same class? > > > > Thats the cunning part :-). The key is that the cache entries now point > > directly at the method structures instead of being copies of the > > structure. The main race condition theoretically present in the old code > > was that one thread could read a cache entry part way through that entry > > being updated by another thread. Since the cache entries are now simple > > pointers, that can't happen. > > The code assumes that pointer accesses are atomic, which I didn't > think was guaranteed on all machines we support. That's why I didn't > think it was safe. I think that we are guaranteed that a pointer read from a memory location will be a whole copy of some value written to that location (i.e. not a combination of part of one write with part of another) That might not be true for bde's exotic i386/64bit long platform. With this scheme, a cache entry will always be either NULL or will point at some instance of struct kobj_method. A thread performing a lookup will read the cache entry once and then simply work with the pointer it has read. That value will either be NULL or will point at a read-only structure (i.e. one which some other thread is guaranteed not to change). If the value read from the cache is the right one (has the right desc pointer), we have a hit. At this point, it doesn't matter if another thread writes to the cache entry, we are using a copy of the pointer which it contained so we are safe from anything another thread might do to the cache. On the other hand, if the value read was NULL or had the wrong desc pointer, we do the slow lookup. This slow lookup does all of its work with read-only structures and is also safe from other threads. The last thing it does is write the value it found to the cache (note that this is simply to help subsequent lookups - we don't read back from the cache so it doesn't matter if something else is written to our cache entry. The value from the slow lookup is returned in a register and the function is called. If two threads look up the same method simultaneously, they will both write the same value to the cache entry and will both call the same function. If two threads collide for different methods, they will both write different entries but will still call the right function. It would be theoretically possible for a pathological case to end up doing the slow lookup in that case more often than normal but I would be surprised if that ever happened in the wild. I could possibly shave an instruction off the inline lookup code by initialising each cache entry to point at a bogus method structure instead of initialising it to NULL. > > >> I still believe that the concept of inherited interfaces is better > >> way to achieve multiple inheritance. The methods I may want to > >> inherit need not be associated with what we currently call a device > >> class. The nice thing about your approach is that it doesn't require > >> a massive rototilling of the drivers, but I fear it doesn't go far > >> enough toward providing flexible inheritence. > > > > It was the mass rototilling which I wanted to avoid. It still seems like > > a pretty good thing to maintain some kind of API compatibility between > > FreeBSD 4, 5 and 6 and this method can support that. Your original > > scheme was 'single inheritance of multiple interfaces'. This patch can > > support that since you can derive from as many base classes as you like > > - each base class will be looked at if the previous class doesn't find a > > match. > > There were quite a few differences: > > 1) Inheritence was not limited to only inheriting from a base interface. > The method lookup traversed all the way to the root. This proposed scheme also traverses through base classes of base classes up to the roots. > > 2) Default methods were removed and classes were forced to explicitly > list supported interfaces. I found this to be much less confusing. This is a pretty good idea. It might be a good idea to take this in a few steps - i.e. get the basic MI into the system and stable. Then add some handy base classes containing the various default methods. After all the drivers are updated to use those base classes, remove the support for default methods. > > 3) Short-hand for inheriting all of the interfaces of a "parent class" > were provided. This just meant having the ability to list classes > to inherit from so that the kobj class compiling code could iterate > through all the interfaces of specified classes. > > 4) The method cache was removed in favor of a direct indexing into > the interface's static method table. Interface lookup was explicit, > the hope being that one interface lookup could be amortized over > multiple method calls. This would allow using kobj interfaces > for more heavy weight tasks such as exporting the correct XPT > interface in CAM to low-level drivers (FC, SPI, SATA, SAS, etc). Interface lookup using current kobj can be done expliticly. There is nothing to say that you must call the generated inline function - you can just as well call KOBJOPLOOKUP yourself e.g. if you need to call the method a few hundred thousand times. Given that the amortised cost of calling a kobj method is only about 20% slower than a direct function call, this kind of thing should only be needed in extreme cases. > > 5) I also planned to add a way to do "super" invocations. This would > cut down on lots of the bloat in things like cardbus. I've been thinking a bit about that. In C++, the normal way is to explicitly call the base method: void foo() { do important stuff; BaseClass::foo(); OtherBaseClass::foo(); } This version is easy and is what cardbus does now. You simply export the function and call it directly. This is what happens for instance in cardbus_read_ivar - it handles its own variables and then calls pci_read_ivar for everything else. It would be possible to do something with dynamic lookups using the ops cache of the base class but I haven't thought of a way of presenting the API which isn't messy. > > The goal was to be able to do things like, using cardbus as an example, > inheret all of the interfaces of PCI adding in the CIS parsing interface > shared with pccard. CIS parsing is just a list of methods that doesn't > make sense to be a "driver class" on its own. I think that you can do exactly this with what I've proposed. > > I guess I have bigger dreams than most for how to use newbus... Big dreams are good :-) > > Do you also have a proposal for handling ivar in the multiple-inheritance > scheme? This is required to make multiple inheritance really useful. Off the top of my head, assign ranges to drivers at compile time in such a way that the ivar indices of one driver are guaranteed to be different from the indices of another driver. Requires a centralised registry but we have one - the CVS repository would serve. From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 15:21:12 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1802F16A4BF for ; Mon, 22 Sep 2003 15:21:12 -0700 (PDT) Received: from magic.adaptec.com (magic-mail.adaptec.com [216.52.22.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4598F43FB1 for ; Mon, 22 Sep 2003 15:21:10 -0700 (PDT) (envelope-from gibbs@scsiguy.com) Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11]) by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h8MMLAR19249; Mon, 22 Sep 2003 15:21:10 -0700 Received: from [10.100.253.70] (aslan.btc.adaptec.com [10.100.253.70]) by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id PAA09082; Mon, 22 Sep 2003 15:21:09 -0700 (PDT) Date: Mon, 22 Sep 2003 16:24:23 -0600 From: "Justin T. Gibbs" To: Doug Rabson Message-ID: <1494190000.1064269463@aslan.btc.adaptec.com> In-Reply-To: <1064266269.68463.42.camel@herring.nlsystems.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <1423490000.1064260204@aslan.btc.adaptec.com> <1064261482.68463.13.camel@herring.nlsystems.com> <1448210000.1064263540@aslan.btc.adaptec.com> <1064266269.68463.42.camel@herring.nlsystems.com> X-Mailer: Mulberry/3.1.0b6 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Justin T. Gibbs" List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 22:21:12 -0000 >> The code assumes that pointer accesses are atomic, which I didn't >> think was guaranteed on all machines we support. That's why I didn't >> think it was safe. > > I think that we are guaranteed that a pointer read from a memory > location will be a whole copy of some value written to that location > (i.e. not a combination of part of one write with part of another) That > might not be true for bde's exotic i386/64bit long platform. Okay. That was my only concern. For some reason I thought that some early Sparc64 machines only enforced load/store atomicity on 16bit entities and that was only true with some platform specific magic. >> There were quite a few differences: >> >> 1) Inheritence was not limited to only inheriting from a base interface. >> The method lookup traversed all the way to the root. > > This proposed scheme also traverses through base classes of base classes > up to the roots. I see the recursion now. >> 4) The method cache was removed in favor of a direct indexing into >> the interface's static method table. Interface lookup was explicit, >> the hope being that one interface lookup could be amortized over >> multiple method calls. This would allow using kobj interfaces >> for more heavy weight tasks such as exporting the correct XPT >> interface in CAM to low-level drivers (FC, SPI, SATA, SAS, etc). > > Interface lookup using current kobj can be done expliticly. There is > nothing to say that you must call the generated inline function - you > can just as well call KOBJOPLOOKUP yourself e.g. if you need to call the > method a few hundred thousand times. Given that the amortised cost of > calling a kobj method is only about 20% slower than a direct function > call, this kind of thing should only be needed in extreme cases. The problem is not entirely that of speed. The cache is large even for classes that may only export or use a few methods. This also means that an active cache requires more than one cache line even if only a few methods are used repeatedly. Lastly, the lookup code is a bit large for an inline. All of this is fine for interfaces largely used for device configuration, but this makes the current scheme less interesting in, for example, a device driver's main code path. >> 5) I also planned to add a way to do "super" invocations. This would >> cut down on lots of the bloat in things like cardbus. > > I've been thinking a bit about that. In C++, the normal way is to > explicitly call the base method: > > void foo() { > do important stuff; > BaseClass::foo(); > OtherBaseClass::foo(); > } > > This version is easy and is what cardbus does now. You simply export the > function and call it directly. This is what happens for instance in > cardbus_read_ivar - it handles its own variables and then calls > pci_read_ivar for everything else. Yes. This is one of the sore points of C++. You shouldn't have to know where a method implementation resides in order to call it. It makes it difficult in add layers to a class or interface hierarchy since all consumers of a moved method must be updated. > It would be possible to do something with dynamic lookups using the ops > cache of the base class but I haven't thought of a way of presenting the > API which isn't messy. A method typically knows the interface that it belongs to. At interface compile time, super methods could be recorded into an alternate method table within the interface. This would allow chained super calls all the way to the root without having to explicitly walk interface pointers or do cache lookups. The only time you then need to massively update the code is if you move a method that makes use of the super feature to a different interface. >> Do you also have a proposal for handling ivar in the multiple-inheritance >> scheme? This is required to make multiple inheritance really useful. > > Off the top of my head, assign ranges to drivers at compile time in such > a way that the ivar indices of one driver are guaranteed to be different > from the indices of another driver. Requires a centralised registry but > we have one - the CVS repository would serve. This doesn't work well for binary only modules that create their own interfaces. Ivars should work even in this case. I also hope to avoid having large sparce ivar tables if possible. My current thought is to use variables for the ivar indexes within an interface so that any offsets needed to deal with parent interfaces can be made during interface registration. This becomes easier to deal with in the interface scheme since, with classes able to inherit from multiple interfaces, there is no need for interfaces to be polymorphic, so it is easy to determine how much space to allocate for the parent interface. The same scheme could be used for method table allocations. -- Justin From owner-freebsd-arch@FreeBSD.ORG Mon Sep 22 18:02:09 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A7F1516A4B3 for ; Mon, 22 Sep 2003 18:02:09 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8FC0D43FF5 for ; Mon, 22 Sep 2003 18:02:08 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9p1/8.12.3) with ESMTP id h8N126GA048942; Mon, 22 Sep 2003 19:02:07 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Sep 2003 19:02:08 -0600 (MDT) Message-Id: <20030922.190208.46315325.imp@bsdimp.com> To: dfr@nlsystems.com From: "M. Warner Losh" In-Reply-To: <1064266269.68463.42.camel@herring.nlsystems.com> References: <1064261482.68463.13.camel@herring.nlsystems.com> <1448210000.1064263540@aslan.btc.adaptec.com> <1064266269.68463.42.camel@herring.nlsystems.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: gibbs@scsiguy.com cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2003 01:02:09 -0000 In message: <1064266269.68463.42.camel@herring.nlsystems.com> Doug Rabson writes: : > The goal was to be able to do things like, using cardbus as an example, : > inheret all of the interfaces of PCI adding in the CIS parsing interface : > shared with pccard. CIS parsing is just a list of methods that doesn't : > make sense to be a "driver class" on its own. : : I think that you can do exactly this with what I've proposed. I think so too. However, from a practical point of view, I'm not so sure that sharing the CIS parsing code is a good idea. I'm not sure they are similar enough that it would be a significant win. The nuts and bolts of parsing the containers of the CIS entries is the same, but the details of digging out the information and doing resource stuff is rather different. The pccard parsing is rather fragile right now and would require a big re-write to make this a viable option. It does make a good example, however, if we were doing things completely from scratch. However, there are a number of other problems in newcard right now that I'd strong recommend against such a re-write for a while yet... Warner From owner-freebsd-arch@FreeBSD.ORG Tue Sep 23 00:46:01 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C80C16A4B3 for ; Tue, 23 Sep 2003 00:46:01 -0700 (PDT) Received: from mailhub.fokus.fraunhofer.de (mailhub.fokus.fraunhofer.de [193.174.154.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 84D5C43FDD for ; Tue, 23 Sep 2003 00:45:59 -0700 (PDT) (envelope-from brandt@fokus.fraunhofer.de) Received: from beagle (beagle [193.175.132.100])h8N7jeU09251; Tue, 23 Sep 2003 09:45:40 +0200 (MEST) Date: Tue, 23 Sep 2003 09:45:39 +0200 (CEST) From: Harti Brandt To: Sam Leffler In-Reply-To: <839561335.1064220615@melange.errno.com> Message-ID: <20030923094042.B9690@beagle.fokus.fraunhofer.de> References: <20030920141158.B97439@xorpc.icir.org> <20030922160106.Y6621@beagle.fokus.fraunhofer.de> <839561335.1064220615@melange.errno.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@freebsd.org cc: "M. Warner Losh" cc: jdp@polstra.com Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: harti@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2003 07:46:01 -0000 On Mon, 22 Sep 2003, Sam Leffler wrote: SL>> On Sat, 20 Sep 2003, M. Warner Losh wrote: SL>> SL>> MWL>In message: SL>> MWL> John Polstra writes: SL>> MWL>: there will be no link changes except at bootstrap and shutdown. SL>> MWL> SL>> MWL>For server machines. For laptops, these events happen more often. SL>> MWL>However, for most laptops, the rate that they happen is typically on SL>> MWL>less than 1/hour. Still rare enough to not worry about optimizing it SL>> MWL>and your other points are good. I just wanted to make sure that it SL>> MWL>wasn't optimized to the point where disconnecting the cable from the SL>> MWL>laptop to move it across the room, or another room doesn't cause huge SL>> MWL>problems. SL>> SL>> Perhaps the polling should be configurable., I struggled with this a year SL>> ago in the xl driver. I have an application that does real-time satellite SL>> simulation over two ethernet links with HZ=10000. This works really SL>> perfect (timing errors are not larger than 200usecs) except for the MII SL>> polling. With help from msilby we could cut down the mii polling delay SL>> from 8msecs to below 1msec. But, because that's still too much for my SL>> application, I have simply commented out the polling calls in the mii SL>> source. I suppose there are other application (servers) where one could SL>> simply switch them off. This could perhaps be done with a sysctl in SL>> mii(4). SL> SL>I don't believe removing functionality is the right approach (though for SL>specialized applications it might be what you have to do). I brought up SL>the issue because it is widespread and has noticeable impact on SL>performance. Some of this is a byproduct of the semi-mechanical way in SL>which drivers have been converted from spl's to mutex's. It also appears SL>some of the current work can be removed entirely for certain devices. I was not talking about removing any functionality just to make it configurable. There are just cases, where you cannot afford the 1msec or 10msec or whatever delay the polling incures. IF it is possible to overcome this by appropriate changes to the locking strategy, then this is of course the preferable solution. I can live with this stuff removed locally in my tree until a better solution is found. harti SL>I think we can move the mii bus polling/tick processing out from under the SL>driver lock so these operations do not interfere with interrupt processing. SL>But I'm not sure if this device-dependent; hence I didn't just add a lock SL>to mii and sweep the drivers. I'm willing to add mii locking but can't SL>deal with all the drivers. Before I do something in mii I'd like to SL>understand better whether it's worthwhile or whether we'll still have to SL>grab the driver lock to do the work. I suppose if I just add locking to SL>mii then those drivers that are unchanged will just incur double locking SL>until they are "fixed". -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.fraunhofer.de, harti@freebsd.org From owner-freebsd-arch@FreeBSD.ORG Tue Sep 23 01:31:05 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 398A116A4BF for ; Tue, 23 Sep 2003 01:31:05 -0700 (PDT) Received: from herring.nlsystems.com (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id 36BEF43FE9 for ; Tue, 23 Sep 2003 01:31:03 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from [10.0.0.2] (herring.nlsystems.com [10.0.0.2]) by herring.nlsystems.com (8.12.9/8.12.8) with ESMTP id h8N8UoY9079833; Tue, 23 Sep 2003 09:30:50 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: "Justin T. Gibbs" In-Reply-To: <1494190000.1064269463@aslan.btc.adaptec.com> References: <1064221837.15078.14.camel@herring.nlsystems.com> <1423490000.1064260204@aslan.btc.adaptec.com> <1064261482.68463.13.camel@herring.nlsystems.com> <1448210000.1064263540@aslan.btc.adaptec.com> <1064266269.68463.42.camel@herring.nlsystems.com> <1494190000.1064269463@aslan.btc.adaptec.com> Content-Type: text/plain Message-Id: <1064305850.68463.67.camel@herring.nlsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 23 Sep 2003 09:30:50 +0100 Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-4.9 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_XIMIAN version=2.55 X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2003 08:31:05 -0000 On Mon, 2003-09-22 at 23:24, Justin T. Gibbs wrote: > >> The code assumes that pointer accesses are atomic, which I didn't > >> think was guaranteed on all machines we support. That's why I didn't > >> think it was safe. > > > > I think that we are guaranteed that a pointer read from a memory > > location will be a whole copy of some value written to that location > > (i.e. not a combination of part of one write with part of another) That > > might not be true for bde's exotic i386/64bit long platform. > > Okay. That was my only concern. For some reason I thought that > some early Sparc64 machines only enforced load/store atomicity on 16bit > entities and that was only true with some platform specific magic. I thought that this was only a problem for locked read-modify-write cycles. The kobj lookup scheme doesn't require any kind of locked memory access. > > >> There were quite a few differences: > >> > >> 1) Inheritence was not limited to only inheriting from a base interface. > >> The method lookup traversed all the way to the root. > > > > This proposed scheme also traverses through base classes of base classes > > up to the roots. > > I see the recursion now. Recursion is necessary otherwise you can't guarantee that the base class functions correctly. > > >> 4) The method cache was removed in favor of a direct indexing into > >> the interface's static method table. Interface lookup was explicit, > >> the hope being that one interface lookup could be amortized over > >> multiple method calls. This would allow using kobj interfaces > >> for more heavy weight tasks such as exporting the correct XPT > >> interface in CAM to low-level drivers (FC, SPI, SATA, SAS, etc). > > > > Interface lookup using current kobj can be done expliticly. There is > > nothing to say that you must call the generated inline function - you > > can just as well call KOBJOPLOOKUP yourself e.g. if you need to call the > > method a few hundred thousand times. Given that the amortised cost of > > calling a kobj method is only about 20% slower than a direct function > > call, this kind of thing should only be needed in extreme cases. > > The problem is not entirely that of speed. The cache is large even > for classes that may only export or use a few methods. This also > means that an active cache requires more than one cache line even > if only a few methods are used repeatedly. Lastly, the lookup > code is a bit large for an inline. All of this is fine for interfaces > largely used for device configuration, but this makes the current > scheme less interesting in, for example, a device driver's main code > path. It is true that the cache is often larger than the number of methods. I have reduced its size in bytes by a factor of two with this patch though so it currently stands at 2k bytes on a 32bit platform (4k on a 64bit platform). This cost is per class though so its amortised over all the instances of that class. Also, the cache should only exist for active classes (i.e. classes with instances). Classes which don't have instances have their caches freed. On i386, the method dispatch with this patch is nine instructions for the common case (cache hit) plus six instructions for the uncommon case (cache miss). I think I can shave the common case down to seven instructions. This doesn't sound too expensive to me and like I said, if you are calling a method several hundred thousand times, you could consider cacheing the result of KOBJOPLOOKUP. > > >> 5) I also planned to add a way to do "super" invocations. This would > >> cut down on lots of the bloat in things like cardbus. > > > > I've been thinking a bit about that. In C++, the normal way is to > > explicitly call the base method: > > > > void foo() { > > do important stuff; > > BaseClass::foo(); > > OtherBaseClass::foo(); > > } > > > > This version is easy and is what cardbus does now. You simply export the > > function and call it directly. This is what happens for instance in > > cardbus_read_ivar - it handles its own variables and then calls > > pci_read_ivar for everything else. > > Yes. This is one of the sore points of C++. You shouldn't have to > know where a method implementation resides in order to call it. It > makes it difficult in add layers to a class or interface hierarchy > since all consumers of a moved method must be updated. Unfortunately, when you have multiple base classes, there is no general way of deciding which hase class to use for your 'super'. I haven't found the explicit naming of the base class to be much of a problem when writing in C++ (and I've written a lot of code in C++ over the last few years). > > > It would be possible to do something with dynamic lookups using the ops > > cache of the base class but I haven't thought of a way of presenting the > > API which isn't messy. > > A method typically knows the interface that it belongs to. At interface > compile time, super methods could be recorded into an alternate method > table within the interface. This would allow chained super calls all the > way to the root without having to explicitly walk interface pointers or do > cache lookups. The only time you then need to massively update the code is > if you move a method that makes use of the super feature to a different > interface. This works when you have the 'single inheritance of multiple interfaces' scheme supported by your original work. I'm not quite sure how it would work when you have multiple base classes, all of which could potentially implement the method. > > >> Do you also have a proposal for handling ivar in the multiple-inheritance > >> scheme? This is required to make multiple inheritance really useful. > > > > Off the top of my head, assign ranges to drivers at compile time in such > > a way that the ivar indices of one driver are guaranteed to be different > > from the indices of another driver. Requires a centralised registry but > > we have one - the CVS repository would serve. > > This doesn't work well for binary only modules that create their own > interfaces. Ivars should work even in this case. I also hope to avoid > having large sparce ivar tables if possible. My current thought is > to use variables for the ivar indexes within an interface so that any > offsets needed to deal with parent interfaces can be made during interface > registration. This becomes easier to deal with in the interface scheme > since, with classes able to inherit from multiple interfaces, there is > no need for interfaces to be polymorphic, so it is easy to determine how > much space to allocate for the parent interface. The same scheme could > be used for method table allocations. Hmm. Some kind of SYSINIT-driven ivar index allocator, perhaps? From owner-freebsd-arch@FreeBSD.ORG Tue Sep 23 01:47:56 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8E58216A4B3 for ; Tue, 23 Sep 2003 01:47:56 -0700 (PDT) Received: from mx.nsu.ru (mx.nsu.ru [212.192.164.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D8EB43FE1 for ; Tue, 23 Sep 2003 01:47:52 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from mail by mx.nsu.ru with drweb-scanned (Exim 3.35 #1 (Debian)) id 1A1ir1-0004Fa-00; Tue, 23 Sep 2003 15:49:03 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mx.nsu.ru with esmtp (Exim 3.35 #1 (Debian)) id 1A1ir0-0004DI-00; Tue, 23 Sep 2003 15:49:02 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.9/8.12.9) with ESMTP id h8N8o6vt075842; Tue, 23 Sep 2003 15:50:06 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.9/8.12.9/Submit) id h8N8o5VM075750; Tue, 23 Sep 2003 15:50:05 +0700 (NOVST) Date: Tue, 23 Sep 2003 15:50:05 +0700 From: Alexey Dokuchaev To: Bob Bishop Message-ID: <20030923085005.GB71368@regency.nsu.ru> References: <3F6F1926.9060206@sbcglobal.net> <4.3.2.7.2.20030922171105.02d79e50@gid.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4.3.2.7.2.20030922171105.02d79e50@gid.co.uk> User-Agent: Mutt/1.4.1i X-Envelope-To: rb@gid.co.uk, mweinstein@sbcglobal.net, freebsd-arch@freebsd.org cc: freebsd-arch@freebsd.org cc: "Mark R. Weinstein" Subject: Re: BSD Versions -- Older X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2003 08:47:56 -0000 On Mon, Sep 22, 2003 at 05:11:31PM +0100, Bob Bishop wrote: > Hi, > > See http://www.mckusick.com/csrg/index.html These, while being definitely very nice, are for cash only. Try out google on "Unix Archive". I've even done it for you. http://www.tuhs.org/archive_sites.html Pick up the mirror and you're all set. ./danfe > > At 16:45 22/9/03, Mark R. Weinstein wrote: > >I am doing some research on the development of the BSD Unix operating > >system, and I am looking for some older (i.e. BSD 4.2 or 4.3) versions of > >BSD from the mid to late 1980's. Do you by chance know where I could > >obtain those older versions (the platform is not important)? > >Thanks in advance, Mark. > > > >_______________________________________________ > >freebsd-arch@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-arch > >To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > > -- > Bob Bishop +44 (0)118 977 4017 > rb@gid.co.uk fax +44 (0)118 989 4254 > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Sep 23 14:15:34 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4587E16A4B3 for ; Tue, 23 Sep 2003 14:15:34 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id D881343FE0 for ; Tue, 23 Sep 2003 14:15:32 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9p1/8.12.3) with ESMTP id h8NLFUGA067802; Tue, 23 Sep 2003 15:15:31 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 23 Sep 2003 15:15:30 -0600 (MDT) Message-Id: <20030923.151530.84357823.imp@bsdimp.com> To: dfr@nlsystems.com From: "M. Warner Losh" In-Reply-To: <1064305850.68463.67.camel@herring.nlsystems.com> References: <1064266269.68463.42.camel@herring.nlsystems.com> <1494190000.1064269463@aslan.btc.adaptec.com> <1064305850.68463.67.camel@herring.nlsystems.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: gibbs@scsiguy.com cc: arch@freebsd.org Subject: Re: kobj multiple inheritance X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2003 21:15:34 -0000 In message: <1064305850.68463.67.camel@herring.nlsystems.com> Doug Rabson writes: : Hmm. Some kind of SYSINIT-driven ivar index allocator, perhaps? I've logn thought this is an excellent idea. Have a 32 bit name space. 16 allocated to an interface and 16 that are private to that interface. That should be plenty of bits, and the read/write ivar routines would still be simple. Hide it behind a macro, and it doesn't matter the sizes. You'd change: switch (which) { case PCI_IVAR_ETHADDR: ... } to if (IVAR_SELECTOR(which) != pci_ivar) return (EIO); /* or pass it on? */ switch (IVAR_PRIVATE(which)) { case PCI_IVAR_ETHADDR: ... } Which isn't burdonsome at all. If you have more than 65,000 interfaces in the system, then you have bigger issues :-) Warner From owner-freebsd-arch@FreeBSD.ORG Wed Sep 24 14:03:43 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CAC316A4BF for ; Wed, 24 Sep 2003 14:03:43 -0700 (PDT) Received: from mail.speakeasy.net (mail14.speakeasy.net [216.254.0.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id C08B443FEA for ; Wed, 24 Sep 2003 14:03:42 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 7120 invoked from network); 24 Sep 2003 21:03:42 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 24 Sep 2003 21:03:42 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h8OL3c6Y024435 for ; Wed, 24 Sep 2003 17:03:39 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Wed, 24 Sep 2003 17:03:42 -0400 (EDT) From: John Baldwin To: arch@FreeBSD.org X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) Subject: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Sep 2003 21:03:43 -0000 Now that we have 'nooptions' and 'nodevice' in kernel config files as well as the ability to include other config files, I'd like to tweak config(8) so that it automatically includes sys/conf/DEFAULT (or DEFAULTS) and sys/${MACHINE}/conf/DEFAULT (S) when generating a kernel config and then get rid of all the 'NO_*' options. For example, NO_F00F_HACK could be renamed to a positive FOOF_HACK option. sys/i386/conf/DEFAULT would contain 'options F00F_HACK' and if people wanted to disable it they could use 'nooptions F00F_HACK' in their custom config. This would also make it easier to turn things on by default w/o requiring that a kernel option get renamed to NO_FOO. So, for example, with DEVFS, turning it on by default could have been done by just adding 'options DEVFS' to sys/conf/DEFAULT rather than having to rename it to 'NO_DEVFS'. Another thing this might help with is allowing for smaller kernels w/o bloating normal config files. For example, in 5.x we turned USER_LDT on by default by just removing the option altogether. We could have simply moved USER_LDT to sys/i386/conf/DEFAULT and folks that wanted to build a smaller kernel and didn't need USER_LDT could use 'nooptions USER_LDT'. Just an idea. Another nice thing, btw, might be to add a sys/conf/GENERIC that the MD GENERIC's could include that would include common things like 'ident', 'FFS', 'INET', etc. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Wed Sep 24 16:37:11 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BB5116A4B3; Wed, 24 Sep 2003 16:37:11 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E18443FBF; Wed, 24 Sep 2003 16:37:08 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id JAA15687; Thu, 25 Sep 2003 09:36:56 +1000 Date: Thu, 25 Sep 2003 09:35:31 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: John Baldwin In-Reply-To: Message-ID: <20030925092319.H5418@gamplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Sep 2003 23:37:11 -0000 On Wed, 24 Sep 2003, John Baldwin wrote: > Now that we have 'nooptions' and 'nodevice' in kernel config files as well > as the ability to include other config files, I'd like to tweak config(8) > so that it automatically includes sys/conf/DEFAULT (or DEFAULTS) and > sys/${MACHINE}/conf/DEFAULT (S) when generating a kernel config and then > get rid of all the 'NO_*' options. OK with me. Do we actually gave the abiltity to include other config files? It was quite broken last time I tried to use it for anything more complicated than the example in "SMP". When breaking POLA by renaming options, please use a consistent namespace for the new names... > For example, NO_F00F_HACK could be renamed to a positive FOOF_HACK option. > sys/i386/conf/DEFAULT would contain 'options F00F_HACK' and if people > wanted to disable it they could use 'nooptions F00F_HACK' in their custom > config. In a consistent namespace, it would be CPU_F00F_HACK or maybe CPU_PENTIUM1_F00F_HACK. > Another nice thing, btw, might be to add a sys/conf/GENERIC that the > MD GENERIC's could include that would include common things like > 'ident', 'FFS', 'INET', etc. Too much of this would make it harder to see where things are, especially if there are things toggled back and forth. Something like "make LINT" would be needed to see the final set of directives. Similarly for Makefiles generated by config. It has become hard to temporarily change options by editing the Makefile, since many things are set in .mk files in the source tree. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Sep 26 09:22:01 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC44116A4B3; Fri, 26 Sep 2003 09:22:01 -0700 (PDT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F61C4401E; Fri, 26 Sep 2003 09:22:01 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc12) with ESMTP id <2003092616220001400qkg7ke>; Fri, 26 Sep 2003 16:22:00 +0000 Received: by garple.migus.org (Postfix, from userid 80) id 45DAA8FC30; Fri, 26 Sep 2003 12:22:00 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Fri, 26 Sep 2003 12:22:00 -0400 (EDT) Message-ID: <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> In-Reply-To: <20030925092319.H5418@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org> Date: Fri, 26 Sep 2003 12:22:00 -0400 (EDT) From: "Adam C. Migus" To: "Bruce Evans" User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Sep 2003 16:22:02 -0000 Bruce Evans said: > On Wed, 24 Sep 2003, John Baldwin wrote: > >> Now that we have 'nooptions' and 'nodevice' in kernel config files >> as well >> as the abconsistentnclude other config files, I'd like to tweak >> config(8) >> so that it automatically includes sys/conf/DEFAULT (or DEFAULTS) >> and >> sys/${MACHINE}/conf/DEFAULT (S) when generating a kernel config >> and then >> get rid of all the 'NO_*' options. > > OK with me. > > Do we actually gave the abiltity to include other config files? It > was > quite broken last time I tried to use it for anything more > complicated > than the example in "SMP". > > When breaking POLA by renaming options, please use a consistent > namespace > for the new names... > I use the include feature quite a bit, nested in some cases. It works great for me for creating combinations of debug, diskless, mac and smp kernels for example. >> For example, NO_F00F_HACK could be renamed to a positive FOOF_HACK >> option. >> sys/i386/conf/DEFAULT would contain 'options F00F_HACK' and if >> people >> wanted to disable it they could use 'nooptions F00F_HACK' in their >> custom >> config. > > In a consistent namespace, it would be CPU_F00F_HACK or maybe > CPU_PENTIUM1_F00F_HACK. > >> Another nice thing, btw, might be to add a sys/conf/GENERIC that >> the >> MD GENERIC's could include that would include common things like >> 'ident', 'FFS', 'INET', etc. > > Too much of this would make it harder to see where things are, > especially > if there are things toggled back and forth. Something like "make > LINT" > would be needed to see the final set of directives. Similarly for > Makefiles generated by config. It has become hard to temporarily > change > options by editing the Makefile, since many things are set in .mk > files > in the source tree. > > Bruce Could config(8) get an additional option to print the final set of directives after it does it's magic? It could even be extended to print how/where it got them, if desired, perhaps... -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Fri Sep 26 15:16:04 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B14F716A4B3; Fri, 26 Sep 2003 15:16:04 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBC2F4400E; Fri, 26 Sep 2003 15:16:00 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id IAA31314; Sat, 27 Sep 2003 08:15:51 +1000 Date: Sat, 27 Sep 2003 08:14:30 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: "Adam C. Migus" In-Reply-To: <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> Message-ID: <20030927080420.N18558@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org> <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org cc: John Baldwin Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Sep 2003 22:16:04 -0000 On Fri, 26 Sep 2003, Adam C. Migus wrote: > Bruce Evans said: > > Do we actually gave the abiltity to include other config files? It > > was > > quite broken last time I tried to use it for anything more > > complicated > > than the example in "SMP". > > I use the include feature quite a bit, nested in some cases. It > works great for me for creating combinations of debug, diskless, mac > and smp kernels for example. My example written last Februry still shows that even simple includes don't work: %%% Script started on Tue Feb 25 14:16:01 2003 ttyp0:bde@besplex:/usr/src/sys/i386/conf> cat FOOBAR include FOO ttyp0:bde@besplex:/usr/src/sys/i386/conf> cat FOO machine i386 cpu I486_CPU ident FOO ttyp0:bde@besplex:/usr/src/sys/i386/conf> config FOOBAR config: FOO:1: syntax error ttyp0:bde@besplex:/usr/src/sys/i386/conf> config FOO Kernel build directory is ../compile/FOO Don't forget to do a ``make depend'' ttyp0:bde@besplex:/usr/src/sys/i386/conf> exit Script done on Tue Feb 25 14:16:23 2003 %%% Similarly with FOOBAR's contents identical with SMP's contents except for including FOO instead of GENERIC. So the bug must be related to the file being included ... adding an empty or comment line to the beginning of FOO works around it. I guess there is an off-by-1 byte or line error switching the input stream. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Sep 27 03:53:06 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B54BD16A4B3 for ; Sat, 27 Sep 2003 03:53:06 -0700 (PDT) Received: from fafoe.narf.at (chello212186121237.14.vie.surfer.at [212.186.121.237]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE6164403D for ; Sat, 27 Sep 2003 03:52:52 -0700 (PDT) (envelope-from stefan@fafoe.narf.at) Received: from wombat.fafoe.narf.at (wombat.fafoe.narf.at [192.168.2.102]) by fafoe.narf.at (Postfix) with ESMTP id B299F3FA8; Sat, 27 Sep 2003 12:52:44 +0200 (CEST) Received: by wombat.fafoe.narf.at (Postfix, from userid 1001) id 4B22D5E; Sat, 27 Sep 2003 12:52:44 +0200 (CEST) Date: Sat, 27 Sep 2003 12:52:44 +0200 From: Stefan Farfeleder To: Bruce Evans Message-ID: <20030927105241.GG802@wombat.fafoe.narf.at> References: <20030925092319.H5418@gamplex.bde.org> <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="k1lZvvs/B4yU6o8G" Content-Disposition: inline In-Reply-To: <20030927080420.N18558@gamplex.bde.org> User-Agent: Mutt/1.5.4i cc: arch@FreeBSD.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Sep 2003 10:53:06 -0000 --k1lZvvs/B4yU6o8G Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sat, Sep 27, 2003 at 08:14:30AM +1000, Bruce Evans wrote: > Similarly with FOOBAR's contents identical with SMP's contents except > for including FOO instead of GENERIC. So the bug must be related to > the file being included ... adding an empty or comment line to the > beginning of FOO works around it. I guess there is an off-by-1 byte > or line error switching the input stream. The problem is simply that the input stream is switched immediately to the included file after reading the file name and the parser is still waiting for its newline or semicolon from the production Spec -> Config_spec SEMICOLON. Thus the terminal 'machine' on the first line is a syntax error. Stefan --k1lZvvs/B4yU6o8G Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="config.y.diff" Index: src/usr.sbin/config/config.y =================================================================== RCS file: /usr/home/ncvs/src/usr.sbin/config/config.y,v retrieving revision 1.61 diff -u -r1.61 config.y --- src/usr.sbin/config/config.y 6 Jul 2003 02:00:52 -0000 1.61 +++ src/usr.sbin/config/config.y 27 Sep 2003 10:39:13 -0000 @@ -118,6 +118,9 @@ | Config_spec SEMICOLON | + INCLUDE ID SEMICOLON + = { include($2, 0); }; + | SEMICOLON | error SEMICOLON @@ -164,9 +167,7 @@ = { hints = $2; hintmode = 1; - } | - INCLUDE ID - = { include($2, 0); }; + } System_spec: CONFIG System_id System_parameter_list --k1lZvvs/B4yU6o8G-- From owner-freebsd-arch@FreeBSD.ORG Sat Sep 27 07:05:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EE1416A4B3 for ; Sat, 27 Sep 2003 07:05:36 -0700 (PDT) Received: from arginine.spc.org (arginine.spc.org [195.206.69.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 29D6F44014 for ; Sat, 27 Sep 2003 07:05:35 -0700 (PDT) (envelope-from bms@spc.org) Received: from localhost (localhost [127.0.0.1]) by arginine.spc.org (Postfix) with ESMTP id E7BF1654BB; Sat, 27 Sep 2003 15:05:33 +0100 (BST) Received: from arginine.spc.org ([127.0.0.1]) by localhost (arginine.spc.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 28643-02; Sat, 27 Sep 2003 15:05:33 +0100 (BST) Received: from saboteur.dek.spc.org (lardystuffer.demon.co.uk [212.228.40.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by arginine.spc.org (Postfix) with ESMTP id 9483C654B7; Sat, 27 Sep 2003 15:05:31 +0100 (BST) Received: by saboteur.dek.spc.org (Postfix, from userid 1001) id C46DEC9; Sat, 27 Sep 2003 15:05:25 +0100 (BST) Date: Sat, 27 Sep 2003 15:05:25 +0100 From: Bruce M Simpson To: Stefan Farfeleder Message-ID: <20030927140525.GA3261@saboteur.dek.spc.org> References: <20030925092319.H5418@gamplex.bde.org> <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <20030927105241.GG802@wombat.fafoe.narf.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030927105241.GG802@wombat.fafoe.narf.at> cc: arch@FreeBSD.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Sep 2003 14:05:36 -0000 On Sat, Sep 27, 2003 at 12:52:44PM +0200, Stefan Farfeleder wrote: > > Similarly with FOOBAR's contents identical with SMP's contents except > > for including FOO instead of GENERIC. So the bug must be related to > > the file being included ... adding an empty or comment line to the > > beginning of FOO works around it. I guess there is an off-by-1 byte > > or line error switching the input stream. > > The problem is simply that the input stream is switched immediately to > the included file after reading the file name and the parser is still > waiting for its newline or semicolon from the production bde mentioned this problem to me a few months ago when I suggested we MFC the include feature to 4.x. Thanks for tracking it down! BMS From owner-freebsd-arch@FreeBSD.ORG Sat Sep 27 11:50:00 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52A7A16A4B3 for ; Sat, 27 Sep 2003 11:50:00 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id E888F4402B for ; Sat, 27 Sep 2003 11:49:57 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc11) with ESMTP id <2003092718495601300acc3te>; Sat, 27 Sep 2003 18:49:57 +0000 Received: from localhost (localhost [127.0.0.1]) by localhost (Postfix) with ESMTP id 7F0638FC37; Sat, 27 Sep 2003 14:49:56 -0400 (EDT) Received: from garple.migus.org ([127.0.0.1]) by localhost (caster.migus.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 98954-04-2; Sat, 27 Sep 2003 14:49:56 -0400 (EDT) Received: from migus.org (ganyopa.migus.org [192.168.4.2]) by garple.migus.org (Postfix) with ESMTP id 2A5E48FC30; Sat, 27 Sep 2003 14:49:56 -0400 (EDT) Message-ID: <3F75DBD1.70600@migus.org> Date: Sat, 27 Sep 2003 14:49:53 -0400 From: Adam Migus Organization: Migus Dot Org User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Farfeleder References: <20030925092319.H5418@gamplex.bde.org> <49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <20030927105241.GG802@wombat.fafoe.narf.at> In-Reply-To: <20030927105241.GG802@wombat.fafoe.narf.at> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: arch@FreeBSD.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Sep 2003 18:50:00 -0000 Stefan Farfeleder wrote: >On Sat, Sep 27, 2003 at 08:14:30AM +1000, Bruce Evans wrote: > > > >>Similarly with FOOBAR's contents identical with SMP's contents except >>for including FOO instead of GENERIC. So the bug must be related to >>the file being included ... adding an empty or comment line to the >>beginning of FOO works around it. I guess there is an off-by-1 byte >>or line error switching the input stream. >> >> > >The problem is simply that the input stream is switched immediately to >the included file after reading the file name and the parser is still >waiting for its newline or semicolon from the production > >Spec -> Config_spec SEMICOLON. > >Thus the terminal 'machine' on the first line is a syntax error. > >Stefan > > >------------------------------------------------------------------------ > >Index: src/usr.sbin/config/config.y >=================================================================== >RCS file: /usr/home/ncvs/src/usr.sbin/config/config.y,v >retrieving revision 1.61 >diff -u -r1.61 config.y >--- src/usr.sbin/config/config.y 6 Jul 2003 02:00:52 -0000 1.61 >+++ src/usr.sbin/config/config.y 27 Sep 2003 10:39:13 -0000 >@@ -118,6 +118,9 @@ > | > Config_spec SEMICOLON > | >+ INCLUDE ID SEMICOLON >+ = { include($2, 0); }; >+ | > SEMICOLON > | > error SEMICOLON >@@ -164,9 +167,7 @@ > = { > hints = $2; > hintmode = 1; >- } | >- INCLUDE ID >- = { include($2, 0); }; >+ } > > System_spec: > CONFIG System_id System_parameter_list > > >------------------------------------------------------------------------ > Actually, while the error is not simply an "off-by-one" error, this patch is not sufficient to fix the nested case: Below is the output of running config (with or without this patch) on my set of kernels, one of which includes twice on the first line. This language specification, in fact, fails to handle an include, immediately followed by anything other than a blank line or comment. Thus the fix is a little more complicated. root@caster:ttyp1:conf# for _i in D* F* MAC* SMP*; do echo -n "${_i}:"; grep -n ^include "${_i}"; [ `grep -c ^include "${_i}"` -lt 1 ] && echo; /tmp/config "${_i}" >/dev/null 2>&1; [ $? -ne 0 ] && echo "error"; done DISKLESS:1:include GENERIC DISKLESS_FAST:1:include FAST DISKLESS_FAST_ULE:1:include FAST_ULE DISKLESS_MAC:2:include MAC DISKLESS_MAC_ULE:2:include MAC_ULE DISKLESS_SMP:3:include SMP DISKLESS_SMP_FAST:3:include SMP_FAST DISKLESS_SMP_FAST_ULE:3:include SMP_FAST_ULE DISKLESS_SMP_MAC:1:include SMP_MAC error DISKLESS_SMP_MAC_ULE:3:include SMP_MAC_ULE FAST: FAST_ULE: MAC: MAC_ULE: SMP:6:include GENERIC SMP_FAST:6:include FAST SMP_FAST_ULE:6:include FAST_ULE SMP_MAC:1:include MAC SMP_MAC_ULE:2:include MAC_ULE root@caster:ttyp1:conf#