From owner-freebsd-arch@FreeBSD.ORG Wed Mar 31 01:30:07 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 251DE16A4CE; Wed, 31 Mar 2004 01:30:07 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0FD2943D4C; Wed, 31 Mar 2004 01:30:07 -0800 (PST) (envelope-from peter@evilpete.dyndns.org) Received: from fw.wemm.org (canning.wemm.org [192.203.228.65]) by canning.wemm.org (Postfix) with ESMTP id B4EFB2A8D5; Wed, 31 Mar 2004 01:30:06 -0800 (PST) (envelope-from peter@overcee.wemm.org) Received: from overcee.wemm.org (overcee.wemm.org [10.0.0.3]) by fw.wemm.org (Postfix) with ESMTP id 879A8E29D; Wed, 31 Mar 2004 01:30:07 -0800 (PST) (envelope-from peter@overcee.wemm.org) Received: from overcee.wemm.org (localhost [127.0.0.1]) by overcee.wemm.org (8.12.11/8.12.11) with ESMTP id i2V9TIgo038652; Wed, 31 Mar 2004 01:29:18 -0800 (PST) (envelope-from peter@overcee.wemm.org) Received: from localhost (localhost [[UNIX: localhost]]) by overcee.wemm.org (8.12.11/8.12.11/Submit) id i2V9TCne038649; Wed, 31 Mar 2004 01:29:12 -0800 (PST) (envelope-from peter) From: Peter Wemm To: freebsd-arch@freebsd.org Date: Wed, 31 Mar 2004 01:29:12 -0800 User-Agent: KMail/1.6.1 References: <20040328094048.GA40406@phantom.cris.net> <20040330232429.GA65170@phantom.cris.net> In-Reply-To: <20040330232429.GA65170@phantom.cris.net> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403310129.12916.peter@wemm.org> cc: Dag-Erling Sm?rgrav cc: Alexey Zelkin cc: arch@freebsd.org Subject: Re: CFD: XMLification of NOTES X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2004 09:30:07 -0000 On Tuesday 30 March 2004 03:24 pm, Alexey Zelkin wrote: > On Tue, Mar 30, 2004 at 05:05:24PM +0200, Dag-Erling Sm?rgrav wrote: > > Alexey Zelkin writes: > > > This weekend I have got some spare time (due to illness) and in > > > order to get some rest from current tasks decided to pass thru > > > old TODO file. Most interesting task got my attention was old > > > item related to XMLification NOTES. > > > > > > Important advantage of this step (IMO) is to provide possibility > > > to check dependency/conflicts of kernel configuration file to end > > > users. [...] [...] > > Dependencies between loadable drivers and subsystems are documented > > in the source code (MODULE_DEPEND), though that information is not > > used at compile time. Dependencies that involve non-loadable > > drivers and subsystems are not documented anywhere (except > > sometimes in manual pages). It would not be hard to modify > > config(8) to obtain that information from e.g. > > src/sys/conf/depend*. These are important points. The runtime dependency information is encoded in the source in order to allow for free-form packaging of objects that self-describe their dependencies. We've learned the hard way that having information duplicated in multiple places leads to things getting out of sync. For what its worth, I personally find raw XML to be visually offensive. Machine readability ease doesn't make up for the developer pain in trying to edit/maintain it in this form. I personally would avoid it like the plague. (And thats an interesting question to ask.. Who are the people that you're expecting to maintain the XML format files?) As a side note (but related), I've just finished the first part of a refurbished module loader. I had to do this for amd64, but it has benefits elsewhere. The main change is switching from 'cc -shared' format .ko files to 'ld -r' (relocatable) .ko files. The big consequence of this is that it becomes trivial to build a kernel out of .ko files. (Or at least embed .ko files into the kernel). The new module loader stuff allows this sort of thing: hammer# cd /boot/kernel hammer# ld -r -o usbstuff.ko u*.ko hammer# ls -l usbstuff.ko -rw-r--r-- 1 root wheel 662202 Mar 31 01:12 usbstuff.ko You can then kldload the usbstuff.ko file and get all of the internal modules at once (I can't give an example of that part because I just paniced the machine :-] ). Anyway, you can't do this with the present .ko format. To take full advantage of this flexibility, I'd planned to do a completely new config(8) that used an extended files.*/options.* file set. I wrote group of named.conf-style parsers for it a few years ago but never did anything with them. config(8) was replaced by a couple of standalone parsers that converted the human-readable tables into something that was easy for perl (it was in the tree at the time) scripts to process and generate the kernel and Module makefiles on the fly. What I was working on was building everything into .ka files and assembling the kernel and modules according to the packing list in your config file. [No Bruce, it isn't April 1st here yet] GENERIC included a list of things to go in the static kernel and the generic modules list. All the double compiling went away. I had a huge amount of work to do though. The second part of the work I have planned is to enable the kernel to have dependencies. eg: link the kernel without either the traditional atpc nor the acpi code and do some sort of HAL layer and select the best one at loader(8) time. (It doesn't have to do it this way, but its an example of what could be done. It makes the acpi startup code a lot less painful.) This requires a bit of care and thought though. But this leads to interesting questions.. How do you specify run time dependencies vs static kernel link dependencies? Suppose if_foo.ko has symbol reference dependencies on miibus.ko. There is no reason why you should be prevented from embedding if_foo.ko into the static kernel and loading miibus.ko at boot time to satisfy the static kernel's symbol dependencies. miibus is a bad example, because it is a poorly behaved newbus citizen and makes symbol references that should be done with method calls instead. We have runtime dependencies that do not involve symbols. newbus device stacks do not make promiscuous symbol references to each other, but can only function when all the components are present. For example, you can have a device driver with pci, isa, eisa, cardbus and pccard attachments, all in the kernel. It does *not* need eisa, cardbus, pccard and pci to be present in order to use it on an isa-only system. You could concievably load eisa *after* the driver example above, and it all automagically works. If anybody breaks this, I'll personally hunt them down and shoot them. Anyway, perhaps the thing for me to do is find the parsers and scripts that I wrote a few years ago and freshen them up for some demos. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5