From owner-freebsd-arch Sun Mar 25 2: 1:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from nebula.cybercable.fr (d217.dhcp212-126.cybercable.fr [212.198.126.217]) by hub.freebsd.org (Postfix) with ESMTP id 9089437B71B for ; Sun, 25 Mar 2001 02:01:22 -0800 (PST) (envelope-from mux@qualys.com) Received: (from mux@localhost) by nebula.cybercable.fr (8.11.3/8.11.3) id f2PA0k413461 for arch@FreeBSD.org; Sun, 25 Mar 2001 12:00:46 +0200 (CEST) (envelope-from mux) Date: Sun, 25 Mar 2001 12:00:46 +0200 From: Maxime Henrion To: arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325120046.F777@nebula.cybercable.fr> References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325101657.A36335@ringworld.oblivion.bg>; from roam@orbitel.bg on Sun, Mar 25, 2001 at 10:16:57AM +0300 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Pentchev wrote: > Yeah, blame it all on newfangled means of communication :P > > With all due respect, I think my patch would work better in the case > when one wants to disable all modules except for a select few; yours > would include a whole lot of NO_KMOD_*, which, given the sheer number > of buildable modules (which is a good thing), would sum up to more than > half of /etc/make.conf ;) > > G'luck, > Peter My point is that more often we want to disable some modules and not specify a list of modules which would be scary for beginners and would lead to errors. Maxime -- Don't be fooled by cheap finnish imitations ; BSD is the One True Code Key fingerprint = F9B6 1D5A 4963 331C 88FC CA6A AB50 1EF2 8CBE 99D6 Public Key : http://www.epita.fr/~henrio_m/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 2:16:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 3186937B718 for ; Sun, 25 Mar 2001 02:16:07 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 2384 invoked by uid 1000); 25 Mar 2001 10:15:08 -0000 Date: Sun, 25 Mar 2001 13:15:08 +0300 From: Peter Pentchev To: Maxime Henrion Cc: arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325131508.B472@ringworld.oblivion.bg> Mail-Followup-To: Maxime Henrion , arch@FreeBSD.org References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325120046.F777@nebula.cybercable.fr>; from mux@qualys.com on Sun, Mar 25, 2001 at 12:00:46PM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 12:00:46PM +0200, Maxime Henrion wrote: > Peter Pentchev wrote: > > Yeah, blame it all on newfangled means of communication :P > > > > With all due respect, I think my patch would work better in the case > > when one wants to disable all modules except for a select few; yours > > would include a whole lot of NO_KMOD_*, which, given the sheer number > > of buildable modules (which is a good thing), would sum up to more than > > half of /etc/make.conf ;) > > > > G'luck, > > Peter > > My point is that more often we want to disable some modules and not > specify a list of modules which would be scary for beginners and would > lead to errors. I wonder if the two could be combined... First set SUBDIR's to values determined from MODULES_LIST's, and then do a for loop through the final SUBDIR value, checking for a NO_KMOD_xxx.. But maybe that would be too complicated an interface for something that is going away soon, if committed at all :( G'luck, Peter -- This sentence contradicts itself - or rather - well, no, actually it doesn't! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 2:22:24 2001 Delivered-To: freebsd-arch@freebsd.org Received: from nebula.cybercable.fr (d217.dhcp212-126.cybercable.fr [212.198.126.217]) by hub.freebsd.org (Postfix) with ESMTP id A2DA837B718 for ; Sun, 25 Mar 2001 02:22:21 -0800 (PST) (envelope-from mux@qualys.com) Received: (from mux@localhost) by nebula.cybercable.fr (8.11.3/8.11.3) id f2PALkm13572 for arch@FreeBSD.org; Sun, 25 Mar 2001 12:21:46 +0200 (CEST) (envelope-from mux) Date: Sun, 25 Mar 2001 12:21:46 +0200 From: Maxime Henrion To: arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325122146.G777@nebula.cybercable.fr> References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> <20010325131508.B472@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325131508.B472@ringworld.oblivion.bg>; from roam@orbitel.bg on Sun, Mar 25, 2001 at 01:15:08PM +0300 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Pentchev wrote: > On Sun, Mar 25, 2001 at 12:00:46PM +0200, Maxime Henrion wrote: > > Peter Pentchev wrote: > > > Yeah, blame it all on newfangled means of communication :P > > > > > > With all due respect, I think my patch would work better in the case > > > when one wants to disable all modules except for a select few; yours > > > would include a whole lot of NO_KMOD_*, which, given the sheer number > > > of buildable modules (which is a good thing), would sum up to more than > > > half of /etc/make.conf ;) > > > > > > G'luck, > > > Peter > > > > My point is that more often we want to disable some modules and not > > specify a list of modules which would be scary for beginners and would > > lead to errors. > > I wonder if the two could be combined... First set SUBDIR's to values > determined from MODULES_LIST's, and then do a for loop through the final > SUBDIR value, checking for a NO_KMOD_xxx.. But maybe that would be too > complicated an interface for something that is going away soon, if committed > at all :( Yup, that sounds like overkill :-) Maxime -- Don't be fooled by cheap finnish imitations ; BSD is the One True Code Key fingerprint = F9B6 1D5A 4963 331C 88FC CA6A AB50 1EF2 8CBE 99D6 Public Key : http://www.epita.fr/~henrio_m/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:19: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id E5BFD37B71B for ; Sun, 25 Mar 2001 10:19:02 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f2PIIst59069; Sun, 25 Mar 2001 10:18:54 -0800 (PST) (envelope-from obrien) Date: Sun, 25 Mar 2001 10:18:54 -0800 From: "David O'Brien" To: Peter Pentchev Cc: Maxime Henrion , arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325101854.D58992@dragon.nuxi.com> Reply-To: obrien@FreeBSD.org References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325101657.A36335@ringworld.oblivion.bg>; from roam@orbitel.bg on Sun, Mar 25, 2001 at 10:16:57AM +0300 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 10:16:57AM +0300, Peter Pentchev wrote: > With all due respect, I think my patch would work better in the case > when one wants to disable all modules except for a select few; yours > would include a whole lot of NO_KMOD_*, which, given the sheer number > of buildable modules (which is a good thing), would sum up to more than > half of /etc/make.conf ;) I agree. I plan on committing this to RELENG_4 post-freeze, unless someone says otherwise. -- -- David (obrien@FreeBSD.org) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:21: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id 2CD7837B718 for ; Sun, 25 Mar 2001 10:20:58 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f2PIKs559083; Sun, 25 Mar 2001 10:20:54 -0800 (PST) (envelope-from obrien) Date: Sun, 25 Mar 2001 10:20:53 -0800 From: "David O'Brien" To: Maxime Henrion Cc: arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325102053.E58992@dragon.nuxi.com> Reply-To: obrien@FreeBSD.org References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325120046.F777@nebula.cybercable.fr>; from mux@qualys.com on Sun, Mar 25, 2001 at 12:00:46PM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 12:00:46PM +0200, Maxime Henrion wrote: > My point is that more often we want to disable some modules and not > specify a list of modules which would be scary for beginners and would > lead to errors. I disagree. I think more people would want to specify the ones they want -- just like in the kernel config, you list what you want, not what you don't want (ok, you'll find a very,very few examples doing this). In my kernel I tell it I have a `xl' network card. I want to say the same for modules building, not that I don't have fxp, ed, fe, dc, etc.... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:22:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 53D4737B719 for ; Sun, 25 Mar 2001 10:22:45 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 5048 invoked by uid 1000); 25 Mar 2001 18:21:45 -0000 Date: Sun, 25 Mar 2001 21:21:45 +0300 From: Peter Pentchev To: David O'Brien Cc: Maxime Henrion , arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325212145.C3241@ringworld.oblivion.bg> Mail-Followup-To: David O'Brien , Maxime Henrion , arch@FreeBSD.org References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325101854.D58992@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325101854.D58992@dragon.nuxi.com>; from obrien@FreeBSD.org on Sun, Mar 25, 2001 at 10:18:54AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 10:18:54AM -0800, David O'Brien wrote: > On Sun, Mar 25, 2001 at 10:16:57AM +0300, Peter Pentchev wrote: > > With all due respect, I think my patch would work better in the case > > when one wants to disable all modules except for a select few; yours > > would include a whole lot of NO_KMOD_*, which, given the sheer number > > of buildable modules (which is a good thing), would sum up to more than > > half of /etc/make.conf ;) > > I agree. I plan on committing this to RELENG_4 post-freeze, unless > someone says otherwise. Actually Maxime and I had a talk on IRC; we both came to the same idea, a NO_KMOD_LIST, which lists all the modules you don't want to build. He'll probably post a NO_KMOD_LIST patch RSN. G'luck, Peter -- .siht ekil ti gnidaer eb d'uoy ,werbeH ni erew ecnetnes siht fI To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:24:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id DEEFC37B71D for ; Sun, 25 Mar 2001 10:24:33 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 5071 invoked by uid 1000); 25 Mar 2001 18:23:34 -0000 Date: Sun, 25 Mar 2001 21:23:34 +0300 From: Peter Pentchev To: David O'Brien Cc: Maxime Henrion , arch@FreeBSD.org Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325212334.D3241@ringworld.oblivion.bg> Mail-Followup-To: David O'Brien , Maxime Henrion , arch@FreeBSD.org References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> <20010325102053.E58992@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325102053.E58992@dragon.nuxi.com>; from obrien@FreeBSD.org on Sun, Mar 25, 2001 at 10:20:53AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 10:20:53AM -0800, David O'Brien wrote: > On Sun, Mar 25, 2001 at 12:00:46PM +0200, Maxime Henrion wrote: > > My point is that more often we want to disable some modules and not > > specify a list of modules which would be scary for beginners and would > > lead to errors. > > I disagree. I think more people would want to specify the ones they want > -- just like in the kernel config, you list what you want, not what you > don't want (ok, you'll find a very,very few examples doing this). > > In my kernel I tell it I have a `xl' network card. I want to say the > same for modules building, not that I don't have fxp, ed, fe, dc, etc.... I think so too :) However, Maxime also has another point - keeping a sample MODULES_LIST in /etc/defaults.make.conf means that every time a new module is added, someone has to remember to add it there, too. This is not too hard a task, but well ;) More people should voice their opinions and arguments on that, I think.. G'luck, Peter -- The rest of this sentence is written in Thailand, on To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:33:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 27C9937B719; Sun, 25 Mar 2001 10:33:43 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2PIXZ973706; Sun, 25 Mar 2001 11:33:36 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103251833.f2PIXZ973706@harmony.village.org> To: Peter Pentchev Subject: Re: Building only a specified list of kernel modules Cc: "David O'Brien" , Maxime Henrion , arch@FreeBSD.ORG In-reply-to: Your message of "Sun, 25 Mar 2001 21:23:34 +0300." <20010325212334.D3241@ringworld.oblivion.bg> References: <20010325212334.D3241@ringworld.oblivion.bg> <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> <20010325102053.E58992@dragon.nuxi.com> Date: Sun, 25 Mar 2001 11:33:35 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010325212334.D3241@ringworld.oblivion.bg> Peter Pentchev writes: : More people should voice their opinions and arguments on that, I think.. I personally still like the patch I just posted to -hackers. All this MODULE_PC98_ALT_META_COKEBOTTLE_LIST is way overkill. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:40:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id CF78E37B71B for ; Sun, 25 Mar 2001 10:40:16 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2PIeE973790; Sun, 25 Mar 2001 11:40:14 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103251840.f2PIeE973790@harmony.village.org> To: Peter Pentchev Subject: Re: Building only a specified list of kernel modules Cc: arch@FreeBSD.ORG In-reply-to: Your message of "Sat, 24 Mar 2001 21:11:40 +0200." <20010324211140.C4304@ringworld.oblivion.bg> References: <20010324211140.C4304@ringworld.oblivion.bg> Date: Sun, 25 Mar 2001 11:40:14 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I object to this patch. It is needlessly complicated. It adds too many icky knobs to the tree, created duplicate places where you'd need lists, etc. Index: Makefile =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/modules/Makefile,v retrieving revision 1.171 diff -u -r1.171 Makefile --- Makefile 2001/03/09 20:10:30 1.171 +++ Makefile 2001/03/19 19:17:28 @@ -30,4 +30,8 @@ SUBDIR+=osf1 .endif +.if defined(MODULES_OVERRIDE) +SUBDIR=${MODULES_OVERRIDE} +.endif + .include Should be all that's needed. You can list all the directories you want in there, or trees of directories. No need to go and get extra fancy with this. Peter is working on a better scheme for -current and this is minimally invasive in -stable. People would just add a makeoptions MODULES_OVERRIDE="pcic sound/pcm sound/drivers/ex137x" to their kernel config files and be done with it. While this doesn't provide the NO_KMOD_xxx functionality, I don't think that's needed. People are much much more likely to compile just what they want and nothing else than they are to compile the defaults with one or two things trimmed out. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 10:48: 3 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 0E6EF37B719 for ; Sun, 25 Mar 2001 10:47:59 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 5526 invoked by uid 1000); 25 Mar 2001 18:46:58 -0000 Date: Sun, 25 Mar 2001 21:46:58 +0300 From: Peter Pentchev To: Warner Losh Cc: arch@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325214658.F3241@ringworld.oblivion.bg> Mail-Followup-To: Warner Losh , arch@FreeBSD.ORG References: <20010324211140.C4304@ringworld.oblivion.bg> <200103251840.f2PIeE973790@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103251840.f2PIeE973790@harmony.village.org>; from imp@harmony.village.org on Sun, Mar 25, 2001 at 11:40:14AM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 11:40:14AM -0700, Warner Losh wrote: > I object to this patch. It is needlessly complicated. It adds too > many icky knobs to the tree, created duplicate places where you'd need > lists, etc. With your patch, if I want any kind of sound support in my kernel, I still have to compile all the sound drivers. Same for syscons, same for netgraph, same for any upcoming 'second-level domain' directory. Me no like :) G'luck, Peter -- If the meanings of 'true' and 'false' were switched, then this sentence wouldn't be false. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 11:15:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 5C7E137B71B for ; Sun, 25 Mar 2001 11:15:10 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2PJF0980349; Sun, 25 Mar 2001 12:15:00 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103251915.f2PJF0980349@harmony.village.org> To: Peter Pentchev Subject: Re: Building only a specified list of kernel modules Cc: arch@FreeBSD.ORG In-reply-to: Your message of "Sun, 25 Mar 2001 21:46:58 +0300." <20010325214658.F3241@ringworld.oblivion.bg> References: <20010325214658.F3241@ringworld.oblivion.bg> <20010324211140.C4304@ringworld.oblivion.bg> <200103251840.f2PIeE973790@harmony.village.org> Date: Sun, 25 Mar 2001 12:15:00 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010325214658.F3241@ringworld.oblivion.bg> Peter Pentchev writes: : On Sun, Mar 25, 2001 at 11:40:14AM -0700, Warner Losh wrote: : > I object to this patch. It is needlessly complicated. It adds too : > many icky knobs to the tree, created duplicate places where you'd need : > lists, etc. : : With your patch, if I want any kind of sound support in my kernel, : I still have to compile all the sound drivers. Same for syscons, : same for netgraph, same for any upcoming 'second-level domain' directory. : Me no like :) You are mistaken. I've succuessfully just built one sound driver + the pcm base with using makeoptions MODULES_OVERRIDE="sound/pcm sound/driver/es137x" So your objection is more one of ignorance of what can be done with the patches rather than a failing in the patches. Note, I had to patch the Makefile.${MACHINE_ARCH} as well, so the complete patch looks more like: Index: modules/Makefile =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/modules/Makefile,v retrieving revision 1.171 diff -u -r1.171 Makefile --- modules/Makefile 2001/03/09 20:10:30 1.171 +++ modules/Makefile 2001/03/19 19:17:28 @@ -30,4 +30,8 @@ SUBDIR+=osf1 .endif +.if defined(MODULES_OVERRIDE) +SUBDIR=${MODULES_OVERRIDE} +.endif + .include Index: conf/Makefile.alpha =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/conf/Makefile.alpha,v retrieving revision 1.94 diff -u -r1.94 Makefile.alpha --- conf/Makefile.alpha 2001/03/12 07:47:09 1.94 +++ conf/Makefile.alpha 2001/03/25 19:12:35 @@ -324,6 +324,9 @@ .endif MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR} +.if defined(MODULES_OVERRIDE) +MKMODULESENV+= MODULES_OVERRIDE="${MODULES_OVERRIDE}" +.endif modules: @mkdir -p ${.OBJDIR}/modules Index: conf/Makefile.i386 =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/conf/Makefile.i386,v retrieving revision 1.227 diff -u -r1.227 Makefile.i386 --- conf/Makefile.i386 2001/03/12 07:47:09 1.227 +++ conf/Makefile.i386 2001/03/25 19:11:53 @@ -284,6 +284,9 @@ .endif MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR} +.if defined(MODULES_OVERRIDE) +MKMODULESENV+= MODULES_OVERRIDE="${MODULES_OVERRIDE}" +.endif modules: @mkdir -p ${.OBJDIR}/modules Index: conf/Makefile.ia64 =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/conf/Makefile.ia64,v retrieving revision 1.16 diff -u -r1.16 Makefile.ia64 --- conf/Makefile.ia64 2001/03/12 07:47:09 1.16 +++ conf/Makefile.ia64 2001/03/25 19:12:12 @@ -288,6 +288,9 @@ .endif MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR} +.if defined(MODULES_OVERRIDE) +MKMODULESENV+= MODULES_OVERRIDE="${MODULES_OVERRIDE}" +.endif modules: @mkdir -p ${.OBJDIR}/modules Index: conf/Makefile.pc98 =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/sys/conf/Makefile.pc98,v retrieving revision 1.126 diff -u -r1.126 Makefile.pc98 --- conf/Makefile.pc98 2001/03/12 07:47:09 1.126 +++ conf/Makefile.pc98 2001/03/25 19:12:02 @@ -287,6 +287,9 @@ .endif MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR} +.if defined(MODULES_OVERRIDE) +MKMODULESENV+= MODULES_OVERRIDE="${MODULES_OVERRIDE}" +.endif MKMODULESENV+= MACHINE=pc98 modules: Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 15:18:44 2001 Delivered-To: freebsd-arch@freebsd.org Received: from molly.straylight.com (molly.straylight.com [209.68.199.242]) by hub.freebsd.org (Postfix) with ESMTP id 057F037B71A for ; Sun, 25 Mar 2001 15:18:43 -0800 (PST) (envelope-from jonathan@graehl.org) Received: from dickie (case.straylight.com [209.68.199.244]) by molly.straylight.com (8.11.0/8.10.0) with SMTP id f2PNIZE15264 for ; Sun, 25 Mar 2001 15:18:35 -0800 From: "Jonathan Graehl" To: "freebsd-Arch" Subject: configuration files, XML, Mac OS X release Date: Sun, 25 Mar 2001 15:18:00 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Apparently ( http://maccentral.macworld.com/news/0103/24.primer.shtml ) Mac OS X ships with the traditional set of Unix services (although they intentionally don't have a shell or probably a lot of the other standard programs we're used to). They use XML configuration files to present a uniform graphical configuration for their daemons. Does anyone have a Mac with which to check it out? Is source available to their entire userspace system, or just the OS itself? If Mac OS X has done configuration files right, perhaps we could copy their approach. If not, we can learn from their mistakes. -- Jonathan Graehl http://jonathan.graehl.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 17: 5:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id DE3C137B718 for ; Sun, 25 Mar 2001 17:05:46 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2Q15D027957; Sun, 25 Mar 2001 17:05:13 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: jonathan@graehl.org Cc: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML, Mac OS X release In-Reply-To: References: X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010325170513Z.jkh@osd.bsdi.com> Date: Sun, 25 Mar 2001 17:05:13 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 6 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG The project has a Mac (eatmorepie.freebsd.org) running the OS X release candidate, though it appears to be down at the moment (I'll fix that). Someone can certainly log into it at some point and look this stuff over. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 17:14:35 2001 Delivered-To: freebsd-arch@freebsd.org Received: from klapaucius.zer0.org (klapaucius.zer0.org [204.152.186.45]) by hub.freebsd.org (Postfix) with ESMTP id DE2E337B71B for ; Sun, 25 Mar 2001 17:14:33 -0800 (PST) (envelope-from gsutter@zer0.org) Received: by klapaucius.zer0.org (Postfix, from userid 1001) id AC649239A93; Sun, 25 Mar 2001 17:14:33 -0800 (PST) Date: Sun, 25 Mar 2001 17:14:33 -0800 From: Gregory Sutter To: Jordan Hubbard Cc: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML, Mac OS X release Message-ID: <20010325171433.E77952@klapaucius.zer0.org> References: <20010325170513Z.jkh@osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325170513Z.jkh@osd.bsdi.com>; from jkh@osd.bsdi.com on Sun, Mar 25, 2001 at 05:05:13PM -0800 Organization: Zer0 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 2001-03-25 17:05 -0800, Jordan Hubbard wrote: > The project has a Mac (eatmorepie.freebsd.org) running the OS X > release candidate, though it appears to be down at the moment (I'll > fix that). Someone can certainly log into it at some point and > look this stuff over. I have a copy of the released OS X that I will donate to the project if desired. I'm not sure it will be necessary, though, as Apple's remote upgrade utility is supposed to be real hot stuff, able to upgrade nearly any part of the OS. Greg -- Gregory S. Sutter The best way to accelerate Windows mailto:gsutter@zer0.org is at 9.8 m/s^2. http://www.zer0.org/~gsutter/ hkp://wwwkeys.pgp.net/0x845DFEDD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 18: 7:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail-100baset.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 5863037B719 for ; Sun, 25 Mar 2001 18:07:45 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id VAA81104; Sun, 25 Mar 2001 21:07:34 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: Date: Sun, 25 Mar 2001 21:07:33 -0500 To: "Jonathan Graehl" , "freebsd-Arch" From: Garance A Drosihn Subject: Re: configuration files, XML, Mac OS X release Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 3:18 PM -0800 3/25/01, Jonathan Graehl wrote: >Apparently ( ... ) Mac OS X ships with the traditional set >of Unix services (although they intentionally don't have a >shell or probably a lot of the other standard programs we're >used to). It comes with /bin/sh, /bin/csh, /bin/tcsh, and /bin/zsh. However, MacOS 10 applications (end-user, GUI-type) are not supposed to depend on that being there. The earlier versions also included bash, but the released version does not have it. >They use XML configuration files to present a uniform graphical >configuration for their daemons. > >Does anyone have a Mac with which to check it out? Is source >available to their entire userspace system, or just the OS >itself? Some of the configuration-related sources might not be available. I think Darwin (the open-source OS layer) might do some of the system-configuration things differently than MacOS 10. >If Mac OS X has done configuration files right, perhaps we >could copy their approach. If not, we can learn from their >mistakes. I like some of the things they did with a user-level "defaults" database, to get away from environment variables. (there's a unix command called 'defaults', at least in MacOS 10). -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 18:57:16 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id D45FB37B718; Sun, 25 Mar 2001 18:57:13 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id SAA03510; Sun, 25 Mar 2001 18:57:06 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda03508; Sun Mar 25 18:57:02 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2Q2uuB53752; Sun, 25 Mar 2001 18:56:56 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpde53750; Sun Mar 25 18:56:32 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2Q2uRY01728; Sun, 25 Mar 2001 18:56:27 -0800 (PST) Message-Id: <200103260256.f2Q2uRY01728@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdfc1724; Sun Mar 25 18:56:09 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Peter Pentchev Cc: "David O'Brien" , arch@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules In-reply-to: Your message of "Sun, 25 Mar 2001 10:20:13 +0300." <20010325102013.B36335@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 25 Mar 2001 18:56:08 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010325102013.B36335@ringworld.oblivion.bg>, Peter Pentchev writes : > On Sat, Mar 24, 2001 at 03:06:04PM -0800, David O'Brien wrote: > > On Sat, Mar 24, 2001 at 09:11:40PM +0200, Peter Pentchev wrote: > > > Any severe objections to the attached patch? (and no, I still have not > > > taken the time to put my ICBM address into freebsd.committers.markers, > > > so any objections will have to be in the form of e-mail ;) > > > > I like this patch for RELENG_4 which will not get any of Peter's new and > > improved config(8) work. > > Yes, IMHO RELENG_4 systems could benefit from this, especially those > for which root fs space is at a premium. And slow machines. I can remember when a kernel build would take 20 minutes on my P120 at home. Now it takes 50 minutes. (I suppose I should upgrade. Maybe this fall). Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 19:16:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id E52E037B719 for ; Sun, 25 Mar 2001 19:16:29 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2Q3InZ01309; Sun, 25 Mar 2001 19:19:08 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3ABEB519.CA9F1029@integratus.com> Date: Sun, 25 Mar 2001 19:18:49 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Jordan Hubbard Cc: jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML, Mac OS X release References: <20010325170513Z.jkh@osd.bsdi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jordan Hubbard wrote: > > The project has a Mac (eatmorepie.freebsd.org) running the OS X > release candidate, though it appears to be down at the moment (I'll > fix that). Someone can certainly log into it at some point and > look this stuff over. I took a look at the release during the big pre release party at Apple. It looks like the only XML DTD they have in the system is a property list DTD designed to allow them to store groups of key/value pairs as the backing store for application configuration data. I remain willing to help integrate a lightweight BSD license XML based configuration management system into FreeBSD. Does anybody really want such a beast? -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 19:54:40 2001 Delivered-To: freebsd-arch@freebsd.org Received: from rbn-gw.bgtu.debryansk.ru (rbn-gw.bgtu.debryansk.ru [62.76.89.2]) by hub.freebsd.org (Postfix) with ESMTP id 28D8237B718 for ; Sun, 25 Mar 2001 19:54:09 -0800 (PST) (envelope-from kapr@acm.org) Received: from server.bitmcnit.bryansk.su (root@bitmcnit.bryansk.su [192.168.121.2]) by rbn-gw.bgtu.debryansk.ru (8.11.2/8.11.2) with ESMTP id f2Q3qj317274; Mon, 26 Mar 2001 07:52:45 +0400 Received: (from uucp@localhost) by server.bitmcnit.bryansk.su (8.9.3/8.9.3) with UUCP id HAA07189; Mon, 26 Mar 2001 07:46:05 +0400 Received: (from alex@localhost) by kapran.bitmcnit.bryansk.su (8.11.3/8.11.3) id f2PJjPS00780; Sun, 25 Mar 2001 23:45:25 +0400 (MSD) (envelope-from kapr@acm.org) X-Authentication-Warning: kapran.bitmcnit.bryansk.su: alex set sender to kapr@acm.org using -f Date: Sun, 25 Mar 2001 23:45:25 +0400 From: Alex Kapranoff To: Maxime Henrion Cc: arch@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules Message-ID: <20010325234525.B484@kapran.bitmcnit.bryansk.su> References: <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010325120046.F777@nebula.cybercable.fr>; from mux@qualys.com on Sun, Mar 25, 2001 at 12:00:46PM +0200 X-Operating-System: FreeBSD 5.0-CURRENT i386 Organization: Internal Mongolia Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 12:00:46PM +0200, Maxime Henrion wrote: > Peter Pentchev wrote: > > Yeah, blame it all on newfangled means of communication :P > > > > With all due respect, I think my patch would work better in the case > > when one wants to disable all modules except for a select few; yours > > would include a whole lot of NO_KMOD_*, which, given the sheer number > > of buildable modules (which is a good thing), would sum up to more than > > half of /etc/make.conf ;) > > > > G'luck, > > Peter > > My point is that more often we want to disable some modules and not > specify a list of modules which would be scary for beginners and would > lead to errors. Hm? You usually don't have a box with all the hundred NICS supported by FreeBSD EXCEPT FOR ONE, but instead a box with one or several NICS so that all other if_* modules waste your '/'. Same about snd_* drivers. It seems to be more practical to have a list of needed modules. I have a patch almost identical to Peter's in my tree for about a month and it speeds up kernel builds by a factor of 2 or more. -- Alex Kapranoff, Voice: +7(0832)791845 We've lived 2014 hours in the brand new millenium... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 25 20:36: 2 2001 Delivered-To: freebsd-arch@freebsd.org Received: from outblaze12.outblaze.com (outblaze12.outblaze.com [209.249.164.196]) by hub.freebsd.org (Postfix) with SMTP id 46D2537B719 for ; Sun, 25 Mar 2001 20:36:00 -0800 (PST) (envelope-from weichong78@bsdmail.com) Received: (qmail 30078 invoked by uid 1001); 26 Mar 2001 04:36:00 -0000 Message-ID: <20010326043600.30077.qmail@bsdmail.com> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Mailer: MIME-tools 4.104 (Entity 4.117) From: "Tan Wei Chong" To: arch@freebsd.org Date: Mon, 26 Mar 2001 12:35:59 +0800 Subject: kernel programming newbie Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I used freebsd for quite some time, however is considered a NEWBIE in terms of Unix/BSD kernel programming and would like to get involve in it. For Linux kernel and driver, I can find books but not for FreeBSD. Can anyone kindly point to me some direction (sites, doc, books etc.) that can get me started in either kernel or driver programming in FreeBSD, or more specifically, how do i write a driver in FreeBSD? Thank you Truly, Tan Wei Chong -- Get your free email from http://www.bsdmail.com Powered by Outblaze To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 0:14:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from areilly.bpc-users.org (CPE-144-132-234-126.nsw.bigpond.net.au [144.132.234.126]) by hub.freebsd.org (Postfix) with SMTP id 0500337B718 for ; Mon, 26 Mar 2001 00:14:44 -0800 (PST) (envelope-from areilly@bigpond.net.au) Received: (qmail 76499 invoked by uid 1000); 26 Mar 2001 08:14:44 -0000 From: "Andrew Reilly" Date: Mon, 26 Mar 2001 18:14:43 +1000 To: Jack Rusher Cc: Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML, Mac OS X release Message-ID: <20010326181443.A75840@gurney.reilly.home> References: <20010325170513Z.jkh@osd.bsdi.com> <3ABEB519.CA9F1029@integratus.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3ABEB519.CA9F1029@integratus.com>; from jar@integratus.com on Sun, Mar 25, 2001 at 07:18:49PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 07:18:49PM -0800, Jack Rusher wrote: > Jordan Hubbard wrote: > > > > The project has a Mac (eatmorepie.freebsd.org) running the OS X > > release candidate, though it appears to be down at the moment (I'll > > fix that). Someone can certainly log into it at some point and > > look this stuff over. > > I took a look at the release during the big pre release party at > Apple. It looks like the only XML DTD they have in the system is a > property list DTD designed to allow them to store groups of key/value > pairs as the backing store for application configuration data. > > I remain willing to help integrate a lightweight BSD license XML based > configuration management system into FreeBSD. Does anybody really want > such a beast? I really like the idea of a uniform and configurator-friendly config management system. I'm not crazy about XML, but my opinion on the subject hardly matters. However, it occurred to me a while back that there is no need to do this all at once, and probably many good reasons not to. I'd suggest that an approach might be: 1) Dermine a C API for access to the (XML?) config files. 2) Build a library that implements that API. 3) Convert the ad-hoc config file mechanisms in each application to use the library as well or instead. It sounds as though you have (1) and (2) sorted out. (3) is the biggie, but as I mentioned above: it doesn't have to happen all at once. None of the existing applications share config formats, or rely on common mechanisms, so changing them one at a time will have the positive effect of allowing for the necessary user education process to be gradual. It also gives more room for gradual refinement of the API (1) to accommodate different functionality. Once there's a config API and implementation, it should be an easy (or easier) matter to replace or modify the library to include other forms of config store, such as LDAP databases or netinfo servers or whatever. Note too that (at least with a text (XML) style config store) all of this can be done without the pre-existence of a universal configurator tool, which history has shown to be a difficult and complicated beast. -- Andrew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 1: 9:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ska.bsn (CPE-144-132-209-248.nsw.bigpond.net.au [144.132.209.248]) by hub.freebsd.org (Postfix) with ESMTP id BAA4037B71A for ; Mon, 26 Mar 2001 01:09:31 -0800 (PST) (envelope-from atrn@zeta.org.au) Received: from juju.bsn (juju.bsn [192.168.1.5]) by ska.bsn (8.9.3/8.9.3) with ESMTP id TAA19210; Mon, 26 Mar 2001 19:09:29 +1000 (EST) (envelope-from andy@ska.bsn) Received: (from andy@localhost) by juju.bsn (8.11.1/8.9.3) id f2Q99S836903; Mon, 26 Mar 2001 19:09:28 +1000 (EST) (envelope-from andy) Message-Id: <200103260909.f2Q99S836903@juju.bsn> Date: Mon, 26 Mar 2001 19:09:28 +1000 (EST) From: Andy Newman Reply-To: atrn@zeta.org.au Subject: Re: configuration files, XML, Mac OS X release To: Garance A Drosihn Cc: Jonathan Graehl , freebsd-Arch In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; CHARSET=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Garance A Drosihn wrote: > I like some of the things they did with a user-level "defaults" > database, to get away from environment variables. (there's a > unix command called 'defaults', at least in MacOS 10). Wow, real innovation for once. It used to take two commands in NeXTSTEP. -- Andy (who BTW is going out to buy a Mac tomorrow :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 1:41:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from kwanon.research.canon.com.au (kwanon.research.canon.com.au [203.12.172.254]) by hub.freebsd.org (Postfix) with ESMTP id D16EA37B71B for ; Mon, 26 Mar 2001 01:41:29 -0800 (PST) (envelope-from iain@research.canon.com.au) Received: from bellmann.research.canon.com.au (bellmann.research.canon.com.au [10.5.0.3]) by kwanon.research.canon.com.au (Postfix) with ESMTP id D2C34A4B14; Mon, 26 Mar 2001 09:54:40 +0000 (UTC) Received: from elph.research.canon.com.au (elph.research.canon.com.au [203.12.174.253]) by bellmann.research.canon.com.au (Postfix) with ESMTP id 7C7718B10; Mon, 26 Mar 2001 19:30:45 +1000 (EST) Received: from blow.research.canon.com.au (blow.research.canon.com.au [10.8.1.4]) by elph.research.canon.com.au (Postfix) with ESMTP id B12C53B0; Mon, 26 Mar 2001 19:41:18 +1000 (EST) Date: Mon, 26 Mar 2001 19:41:26 +1000 (EST) From: Iain Templeton To: Andy Newman Cc: Garance A Drosihn , Jonathan Graehl , freebsd-Arch Subject: Re: configuration files, XML, Mac OS X release In-Reply-To: <200103260909.f2Q99S836903@juju.bsn> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 26 Mar 2001, Andy Newman wrote: > Garance A Drosihn wrote: > > I like some of the things they did with a user-level "defaults" > > database, to get away from environment variables. (there's a > > unix command called 'defaults', at least in MacOS 10). > Isn't that to some degree what login.conf can do for you? I know you can set environmental variables there. Or is it rather that apps look in defaults, or some other semantic difference? Not having used a NeXT (and they won't let us plug them in here :-) > Wow, real innovation for once. It used to take two commands in NeXTSTEP. > > -- > Andy (who BTW is going out to buy a Mac tomorrow :) > A friend of mine with a iBook once was going to show me DPx of MacOS X, but he couldn't remember the command to boot it :-) Iain To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 1:56:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail-100baset.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 8AAA837B718 for ; Mon, 26 Mar 2001 01:56:48 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id EAA26574; Mon, 26 Mar 2001 04:56:43 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: Date: Mon, 26 Mar 2001 04:56:42 -0500 To: Iain Templeton From: Garance A Drosihn Subject: Re: configuration files, XML, Mac OS X release Cc: freebsd-arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 7:41 PM +1000 3/26/01, Iain Templeton wrote: >On Mon, 26 Mar 2001, Andy Newman wrote: > >> Garance A Drosihn wrote: > > > I like some of the things they did with a user-level > > > "defaults" database, to get away from environment > > > variables. (there's a unix command called 'defaults', > > > at least in MacOS 10). > > >Isn't that to some degree what login.conf can do for you? >I know you can set environmental variables there. Or is >it rather that apps look in defaults, or some other >semantic difference? Applications query the same "defaults database" for their preferences. So you could type a 'defaults' command in one window, and the application will see that the next time it checks (probably the next time the application is started). No need to log out and back in. [although I'm not sure how much discussion of MacOS 10 we'd want to get into on the freebsd-arch list...] -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 8:47:28 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 41C3B37B719 for ; Mon, 26 Mar 2001 08:47:24 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2QGo1Z01836; Mon, 26 Mar 2001 08:50:01 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3ABF7339.B197AF34@integratus.com> Date: Mon, 26 Mar 2001 08:50:01 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Andrew Reilly Cc: Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? References: <20010325170513Z.jkh@osd.bsdi.com> <3ABEB519.CA9F1029@integratus.com> <20010326181443.A75840@gurney.reilly.home> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Andrew Reilly wrote: > > I really like the idea of a uniform and configurator-friendly > config management system. I'm not crazy about XML, but my > opinion on the subject hardly matters. XML certainly isn't the universal savior that it's made out to be, but it is a decent language for specifying other languages within constraints that more or less match what we are talking about. My feeling is that if a technology is widely adopted and good enough to get the job done, I'd rather use it than make up "yet another standard." > at once. None of the existing applications share config > formats, or rely on common mechanisms, so changing them one at a > time will have the positive effect of allowing for the necessary > user education process to be gradual. It also gives more room I agree with you completely. However, my biggest concern with such a project isn't that the technology will be dauntingly difficult, but rather that people won't want it. There is a sort of tension in this community between developing new technology and trying to make sure that we are as much like the Ghost of Unix past as possible. I am concerned that changing configuration file formats that have been static for twenty years might deeply offend some of the user (and developer) base. > easy (or easier) matter to replace or modify the library to > include other forms of config store, such as LDAP databases or > netinfo servers or whatever. Sure. PAM-like modules to select which ass end you want your configuration to use. > all of this can be done without the pre-existence of a universal > configurator tool, which history has shown to be a difficult and > complicated beast. I think this gets a little bit easier when you have DTDs to represent constraints. We may want to go so far as to add Schema parsing so that we can have real value range information embedded in the format definitions. -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 9: 9:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id E676037B718 for ; Mon, 26 Mar 2001 09:09:49 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2QH9Q937241; Mon, 26 Mar 2001 10:09:26 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103261709.f2QH9Q937241@harmony.village.org> To: Alex Kapranoff Subject: Re: Building only a specified list of kernel modules Cc: Maxime Henrion , arch@FreeBSD.ORG In-reply-to: Your message of "Sun, 25 Mar 2001 23:45:25 +0400." <20010325234525.B484@kapran.bitmcnit.bryansk.su> References: <20010325234525.B484@kapran.bitmcnit.bryansk.su> <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> Date: Mon, 26 Mar 2001 10:09:26 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010325234525.B484@kapran.bitmcnit.bryansk.su> Alex Kapranoff writes: : Hm? You usually don't have a box with all the hundred NICS supported by : FreeBSD EXCEPT FOR ONE, but instead a box with one or several NICS so : that all other if_* modules waste your '/'. Same about snd_* drivers. It : seems to be more practical to have a list of needed modules. I have a : patch almost identical to Peter's in my tree for about a month and it : speeds up kernel builds by a factor of 2 or more. Alex, did you see my simplified patches? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 10: 7: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id A66D937B719 for ; Mon, 26 Mar 2001 10:06:55 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id KAA06684; Mon, 26 Mar 2001 10:05:11 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda06682; Mon Mar 26 10:05:10 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2QI54p58683; Mon, 26 Mar 2001 10:05:04 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdD58667; Mon Mar 26 10:04:53 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2QI4rR06049; Mon, 26 Mar 2001 10:04:53 -0800 (PST) Message-Id: <200103261804.f2QI4rR06049@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdar6037; Mon Mar 26 10:03:59 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Jack Rusher Cc: Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-reply-to: Your message of "Mon, 26 Mar 2001 08:50:01 PST." <3ABF7339.B197AF34@integratus.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 26 Mar 2001 10:03:59 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <3ABF7339.B197AF34@integratus.com>, Jack Rusher writes:> Andrew Reilly wrote: > > > > I really like the idea of a uniform and configurator-friendly > > config management system. I'm not crazy about XML, but my > > opinion on the subject hardly matters. > > XML certainly isn't the universal savior that it's made out to be, but > it is a decent language for specifying other languages within > constraints that more or less match what we are talking about. My > feeling is that if a technology is widely adopted and good enough to get > the job done, I'd rather use it than make up "yet another standard." > > > at once. None of the existing applications share config > > formats, or rely on common mechanisms, so changing them one at a > > time will have the positive effect of allowing for the necessary > > user education process to be gradual. It also gives more room > > I agree with you completely. However, my biggest concern with such a > project isn't that the technology will be dauntingly difficult, but > rather that people won't want it. There is a sort of tension in this > community between developing new technology and trying to make sure that > we are as much like the Ghost of Unix past as possible. I am concerned > that changing configuration file formats that have been static for > twenty years might deeply offend some of the user (and developer) base. I think it has less to do with offending people than having an O/S that is compatible with the other UNIX O/S's out there. For example, syslog.conf, services, protocols, inetd.conf, to name a few can be maintained and copied to FreeBSD, Linux (excepting inetd.conf), Solaris, Tru64-UNIX, AIX, DG-UX, and HP/UX, just to name a few. I for one would stop using FreeBSD if I lost the capability to leverage my time by copying, merging, or editing config files between the various platforms I support -- in other words if the 20% of the systems I maintain require 80% of my time to maintain them I will look for other solutions to reduce my workload. There are enough issues of platform incompatibility between the various UNIX platforms out there without going out of our way to create new incompatibilities. An example of copying, merging and editing is my "wrap" script I've used for years to implement TCP/Wrappers on various platforms: #!/usr/bin/awk -f BEGIN {print"## Wrapped by OSG wrap script.\n##"} $1 !~ /^#/ && $6 != "internal" && $6 !~ /tcpd/ && $6 ~ /sbin/ && $7 !~ /identd/ {print "## " $0; print $1 "\t" $2 "\t" $3 "\t" $4 "\t" $5 "\t/usr/local/etc/tcpd\ t" $7 "\t" $8 " " $9} $1 !~ /^#/ && $6 != "internal" && $6 !~ /tcpd/ && $6 !~ /sbin/ && $7 !~ /identd/ {print "## " $0; print $1 "\t" $2 "\t" $3 "\t" $4 "\t" $5 "\t/usr/local/etc/tcpd\ t" $6 "\t" $8 " " $9} $1 != "time" && $6 == "internal" {print "## " $0} $1 == "time" {print $0} $1 ~ /^#/ || $6 ~ /tcpd/ || $7 ~ /identd/ {print $0} For example, changing inetd.conf's format would mean that I could use this script everywhere except for FreeBSD (and RH Linux as RH uses xinetd). (Our policy has been FreeBSD for servers and Linux or FreeBSD for our desktops). In regards to FreeBSD-only config files like login.conf or login.access, I don't care what format they are in as long as it's intelligent. Rather than stand in the way of progress, having the new config file format adopted by most of the major vendors would go a long way to mitigate many fears people have about a new super-config file format. Another way we can mitigate these concerns is to support two formats until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, DG-UX) catch up. How we go about addressing these concerns doesn't matter at this point. (I do have some implementation ideas but I am purposely not discussing them as implementation details should not be discussed until our policy has been decided.) What does matter is that we do address the compatibility issues before making any sort of decision. Don't read into this that I am offended that the config file format may change. I do dislike having to vi a file on one platform only to have to vi a similar file on another platform when all I'm trying to do is implement a policy change across the 150+ systems. If I was running a FreeBSD-only shop, I would not be concerned with this issue. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 10:36:16 2001 Delivered-To: freebsd-arch@freebsd.org Received: from molly.straylight.com (molly.straylight.com [209.68.199.242]) by hub.freebsd.org (Postfix) with ESMTP id 0B6FD37B719 for ; Mon, 26 Mar 2001 10:36:14 -0800 (PST) (envelope-from jonathan@graehl.org) Received: from dickie (case.straylight.com [209.68.199.244]) by molly.straylight.com (8.11.0/8.10.0) with SMTP id f2QIa5E22326; Mon, 26 Mar 2001 10:36:05 -0800 From: "Jonathan Graehl" To: "Garance A Drosihn" Cc: "freebsd-Arch" Subject: RE: configuration files, XML, Mac OS X release Date: Mon, 26 Mar 2001 10:35:27 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-reply-to: Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Applications query the same "defaults database" for their > preferences. So you could type a 'defaults' command in > one window, and the application will see that the next > time it checks (probably the next time the application > is started). No need to log out and back in. ... like the Windows Registry ;) (duck) -Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 12:19:35 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id B170937B71B for ; Mon, 26 Mar 2001 12:19:30 -0800 (PST) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id GN5Y6MKC; Mon, 26 Mar 2001 15:19:35 -0500 Reply-To: Randell Jesup To: Cy Schubert - ITSD Open Systems Group Cc: Jack Rusher , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? References: <200103261804.f2QI4rR06049@cwsys.cwsent.com> From: Randell Jesup Date: 26 Mar 2001 15:21:23 -0500 In-Reply-To: Cy Schubert - ITSD Open Systems Group's message of "Mon, 26 Mar 2001 10:03:59 -0800" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group writes: >I think it has less to do with offending people than having an O/S that >is compatible with the other UNIX O/S's out there. For example, >syslog.conf, services, protocols, inetd.conf, to name a few can be >maintained and copied to FreeBSD, Linux (excepting inetd.conf), >Solaris, Tru64-UNIX, AIX, DG-UX, and HP/UX, just to name a few. I for >one would stop using FreeBSD if I lost the capability to leverage my >time by copying, merging, or editing config files between the various >platforms I support -- in other words if the 20% of the systems I >maintain require 80% of my time to maintain them I will look for other >solutions to reduce my workload. There are enough issues of platform >incompatibility between the various UNIX platforms out there without >going out of our way to create new incompatibilities. This is a very good point. >In regards to FreeBSD-only config files like login.conf or >login.access, I don't care what format they are in as long as it's >intelligent. Which makes them a good place to start. >Rather than stand in the way of progress, having the new config file >format adopted by most of the major vendors would go a long way to >mitigate many fears people have about a new super-config file format. >Another way we can mitigate these concerns is to support two formats >until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, >DG-UX) catch up. How we go about addressing these concerns doesn't >matter at this point. (I do have some implementation ideas but I am >purposely not discussing them as implementation details should not be >discussed until our policy has been decided.) What does matter is that >we do address the compatibility issues before making any sort of >decision. Certainly. There are some possible tricks for dealing with the compatibility issues, especially if we can have some sort of programmatic conversion (especially if it can work both ways). This is a little like how we handle /etc/passwd, but with two-way automatic conversions. For example, if you changed /etc/all_network_stuff.xml, the system may go and rewrite /etc/resolv.conf; and if you modify /etc/resolv.conf, the system might mirror the change into /etc/all_network_stuff.xml. There may be other solutions, such as allowing a configuration file to hold either old-style data or new-style XML; which allows the admin to decide whether they want old or new style on a file-by-file basis. The cost is supporting both parsers in the service using the file. We've recently moved to XML to hold all our configuration data which is used to set up large number of config files in the system (network, various applications like apache, etc). Stuff like: abuse: root bin: root daemon: root [ SNIP ] This greatly simplifies setting up hundreds of machines and updating their configurations; and makes a unified character-gui configuration interface (that reads/modifies this config file) FAR simpler and easier to deal with. The backend that installs these changes becomes quite simple. Just an example. -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 12:25:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail-100baset.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 4F7CB37B718 for ; Mon, 26 Mar 2001 12:25:57 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id PAA45568; Mon, 26 Mar 2001 15:25:51 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: Date: Mon, 26 Mar 2001 15:25:49 -0500 To: "Jonathan Graehl" From: Garance A Drosihn Subject: RE: configuration files, XML, Mac OS X release Cc: "freebsd-Arch" Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 10:35 AM -0800 3/26/01, Jonathan Graehl wrote: > > Applications query the same "defaults database" for their >> preferences. So you could type a 'defaults' command in >> one window, and the application will see that the next >> time it checks (probably the next time the application >> is started). No need to log out and back in. > >... like the Windows Registry ;) (duck) No. More like the defaults database on nextstep, which was running about five years before Windows95 saw the light of day. It's always amusing when people claim MacOS 10 is stealing something from Windows, when those very things existed in NeXTSTEP before Windows was shipping. But now we have digressed... -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 12:36:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from molly.straylight.com (molly.straylight.com [209.68.199.242]) by hub.freebsd.org (Postfix) with ESMTP id F1E9837B71A for ; Mon, 26 Mar 2001 12:36:07 -0800 (PST) (envelope-from jonathan@graehl.org) Received: from dickie (case.straylight.com [209.68.199.244]) by molly.straylight.com (8.11.0/8.10.0) with SMTP id f2QKa2E23132; Mon, 26 Mar 2001 12:36:02 -0800 From: "Jonathan Graehl" To: "Garance A Drosihn" Cc: "freebsd-Arch" Subject: RE: configuration files, XML, Mac OS X release Date: Mon, 26 Mar 2001 12:35:26 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-reply-to: Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > At 10:35 AM -0800 3/26/01, Jonathan Graehl wrote: > > > Applications query the same "defaults database" for their > >> preferences. So you could type a 'defaults' command in > >> one window, and the application will see that the next > >> time it checks (probably the next time the application > >> is started). No need to log out and back in. > > > >... like the Windows Registry ;) (duck) > > No. More like the defaults database on nextstep, which was > running about five years before Windows95 saw the light of > day. > > It's always amusing when people claim MacOS 10 is stealing > something from Windows, when those very things existed in > NeXTSTEP before Windows was shipping. > > But now we have digressed... I'm asking, since I'm familiar with Windows but not NeXTSTEP, if it is the same concept, or, if not, what the differences are. My point was not at all "Windows was first" - experience has shown that to be unlikely. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 19:23:30 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 938B937B718 for ; Mon, 26 Mar 2001 19:23:24 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2R3OlZ02133; Mon, 26 Mar 2001 19:24:48 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC007FF.1D9F60A3@integratus.com> Date: Mon, 26 Mar 2001 19:24:47 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Cy Schubert - ITSD Open Systems Group Cc: Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? References: <200103261804.f2QI4rR06049@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group wrote: > > solutions to reduce my workload. There are enough issues of platform > incompatibility between the various UNIX platforms out there without > going out of our way to create new incompatibilities. This is the primary reason for the offense. :-) "I don't want to learn another way of doing things, I want this OS to behave like my other POSIX OSes, etc." These are all good reasons not to want to do this sort of thing. On the other hand, these are exactly the reasons that Rob Pike is mostly right when he says that OS work is stagnant (ok, he says it's dead). If you're going to innovate, you're going to break compatibility. That's the nature of the beast. Unfortunately, it means that only a non-UNIX like Plan9 or (dare I say it?) Linux will be able to make fundamental change without pissing people off. I guess in the Linux case, it's just that they don't mind pissing people off. ;-) The caveat to the above is that people are accepting of good ideas that don't change the Old Way(tm). I would cite kqueue() as a nice idea that doesn't port to anyone else's playground. It doesn't scare anybody because it doesn't displace any of the old ways of doing things. Unfortunately, some things can't be done that way. > In regards to FreeBSD-only config files like login.conf or > login.access, I don't care what format they are in as long as it's > intelligent. Which makes them a great place to start. I was also thinking of writing up a little PAM module that does authentication against an XML version of the passwd/shadow files. We are lucky enough to have loadable module support for that subsystem, so it's a nice place to implement a demo. > Rather than stand in the way of progress, having the new config file > format adopted by most of the major vendors would go a long way to > mitigate many fears people have about a new super-config file format. We would obviously want everybody to join us in using the new configuration format/tools. I do think somebody needs to belly up to the bar and order the first drink, though. > Another way we can mitigate these concerns is to support two formats > until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, This strikes me as hard for the subsystems that don't have an API in place that governs how they access files. What are some of your ideas of how you would do this? > Don't read into this that I am offended that the config file format may I don't take it that way at all. This is really good feedback from someone who is thinking carefully about the problem. Thanks, -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 20: 4: 2 2001 Delivered-To: freebsd-arch@freebsd.org Received: from rbn-gw.bgtu.debryansk.ru (rbn-gw.bgtu.debryansk.ru [62.76.89.2]) by hub.freebsd.org (Postfix) with ESMTP id A5AA637B719 for ; Mon, 26 Mar 2001 20:03:59 -0800 (PST) (envelope-from kapr@acm.org) Received: from server.bitmcnit.bryansk.su (root@bitmcnit.bryansk.su [192.168.121.2]) by rbn-gw.bgtu.debryansk.ru (8.11.2/8.11.2) with ESMTP id f2R42t326449; Tue, 27 Mar 2001 08:02:55 +0400 Received: (from uucp@localhost) by server.bitmcnit.bryansk.su (8.9.3/8.9.3) with UUCP id IAA26334; Tue, 27 Mar 2001 08:00:20 +0400 Received: (from alex@localhost) by kapran.bitmcnit.bryansk.su (8.11.3/8.11.3) id f2QJ5N205855; Mon, 26 Mar 2001 23:05:23 +0400 (MSD) (envelope-from kapr@acm.org) X-Authentication-Warning: kapran.bitmcnit.bryansk.su: alex set sender to kapr@acm.org using -f Date: Mon, 26 Mar 2001 23:05:22 +0400 From: Alex Kapranoff To: Warner Losh Cc: Maxime Henrion , arch@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules Message-ID: <20010326230522.A5834@kapran.bitmcnit.bryansk.su> References: <20010325234525.B484@kapran.bitmcnit.bryansk.su> <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> <20010325234525.B484@kapran.bitmcnit.bryansk.su> <200103261709.f2QH9Q937241@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103261709.f2QH9Q937241@harmony.village.org>; from imp@harmony.village.org on Mon, Mar 26, 2001 at 10:09:26AM -0700 X-Operating-System: FreeBSD 5.0-CURRENT i386 Organization: Internal Mongolia Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Mar 26, 2001 at 10:09:26AM -0700, Warner Losh wrote: > In message <20010325234525.B484@kapran.bitmcnit.bryansk.su> Alex Kapranoff writes: > : Hm? You usually don't have a box with all the hundred NICS supported by > : FreeBSD EXCEPT FOR ONE, but instead a box with one or several NICS so > : that all other if_* modules waste your '/'. Same about snd_* drivers. It > : seems to be more practical to have a list of needed modules. I have a > : patch almost identical to Peter's in my tree for about a month and it > : speeds up kernel builds by a factor of 2 or more. > > Alex, > did you see my simplified patches? Yep. You see, I'm writing this using kernel built with your patch applied. Thanks, Warner! We all should go and do our homework on make-fu :) -- Alex Kapranoff, Voice: +7(0832)791845 We've lived 12 weeks in the brand new millenium... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 20:11:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 496FB37B718 for ; Mon, 26 Mar 2001 20:11:43 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2R4BV941756; Mon, 26 Mar 2001 21:11:32 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103270411.f2R4BV941756@harmony.village.org> To: Alex Kapranoff Subject: Re: Building only a specified list of kernel modules Cc: Maxime Henrion , arch@FreeBSD.ORG In-reply-to: Your message of "Mon, 26 Mar 2001 23:05:22 +0400." <20010326230522.A5834@kapran.bitmcnit.bryansk.su> References: <20010326230522.A5834@kapran.bitmcnit.bryansk.su> <20010325234525.B484@kapran.bitmcnit.bryansk.su> <20010324211140.C4304@ringworld.oblivion.bg> <20010324205005.D777@nebula.cybercable.fr> <20010325101657.A36335@ringworld.oblivion.bg> <20010325120046.F777@nebula.cybercable.fr> <20010325234525.B484@kapran.bitmcnit.bryansk.su> <200103261709.f2QH9Q937241@harmony.village.org> Date: Mon, 26 Mar 2001 21:11:31 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010326230522.A5834@kapran.bitmcnit.bryansk.su> Alex Kapranoff writes: : applied. Thanks, Warner! We all should go and do our homework on make-fu :) I've done some evil things with make. At least this isn't one of them :-) Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 20:43:30 2001 Delivered-To: freebsd-arch@freebsd.org Received: from katin.codeconcepts.com (smtz3-038.dfw.tx.bbnow.net [24.219.83.38]) by hub.freebsd.org (Postfix) with ESMTP id 007BC37B718 for ; Mon, 26 Mar 2001 20:43:27 -0800 (PST) (envelope-from greg@codeconcepts.com) Received: from katin.codeconcepts.com (localhost.codeconcepts.com [127.0.0.1]) by katin.codeconcepts.com (8.11.1/8.11.1) with ESMTP id f2R4eRr88969; Mon, 26 Mar 2001 22:40:27 -0600 (CST) (envelope-from greg@katin.codeconcepts.com) Message-Id: <200103270440.f2R4eRr88969@katin.codeconcepts.com> To: Jack Rusher Cc: Cy Schubert - ITSD Open Systems Group , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? Date: Mon, 26 Mar 2001 22:40:06 -0600 From: greg Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Rather than stand in the way of progress, having the new config file > > format adopted by most of the major vendors would go a long way to > > mitigate many fears people have about a new super-config file format. > > We would obviously want everybody to join us in using the new > configuration format/tools. I do think somebody needs to belly up to > the bar and order the first drink, though. > > > Another way we can mitigate these concerns is to support two formats > > until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, > > This strikes me as hard for the subsystems that don't have an API in > place that governs how they access files. What are some of your ideas > of how you would do this? I apologize for not having followed this thread closely enough, but it seems to reak of AIX/ODM. If for nothing other than the sake of completeness, the merits and shortcomings of IBM's solution should be taken into account. At the very least it would shed some light onto the sheer magnitude of such an undertaking. Greg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 20:55:52 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id E6CF237B718 for ; Mon, 26 Mar 2001 20:55:49 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id UAA08375; Mon, 26 Mar 2001 20:55:48 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08373; Mon Mar 26 20:55:33 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2R4tRE63037; Mon, 26 Mar 2001 20:55:27 -0800 (PST) Message-Id: <200103270455.f2R4tRE63037@passer.osg.gov.bc.ca> Received: from localhost.osg.gov.bc.ca(127.0.0.1), claiming to be "passer.osg.gov.bc.ca" via SMTP by localhost.osg.gov.bc.ca, id smtpdU63028; Mon Mar 26 20:55:07 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group X-Sender: cyschubert To: greg Cc: Jack Rusher , Cy Schubert - ITSD Open Systems Group , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-reply-to: Your message of "Mon, 26 Mar 2001 22:40:06 CST." <200103270440.f2R4eRr88969@katin.codeconcepts.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 26 Mar 2001 20:55:07 -0800 From: Cy Schubert Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200103270440.f2R4eRr88969@katin.codeconcepts.com>, greg writes: > > > > Rather than stand in the way of progress, having the new config file > > > format adopted by most of the major vendors would go a long way to > > > mitigate many fears people have about a new super-config file format. > > > > We would obviously want everybody to join us in using the new > > configuration format/tools. I do think somebody needs to belly up to > > the bar and order the first drink, though. > > > > > Another way we can mitigate these concerns is to support two formats > > > until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, > > > > This strikes me as hard for the subsystems that don't have an API in > > place that governs how they access files. What are some of your ideas > > of how you would do this? > > > I apologize for not having followed this thread closely enough, but > it seems to reak of AIX/ODM. If for nothing other than the sake of > completeness, the merits and shortcomings of IBM's solution should be > taken into account. At the very least it would shed some light onto > the sheer magnitude of such an undertaking. The AIX Team here does not have much good to say about ODM. This is just anecdotal from sharing coffee with them. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 21: 9:54 2001 Delivered-To: freebsd-arch@freebsd.org Received: from katin.codeconcepts.com (smtz3-038.dfw.tx.bbnow.net [24.219.83.38]) by hub.freebsd.org (Postfix) with ESMTP id 3D25337B718 for ; Mon, 26 Mar 2001 21:09:52 -0800 (PST) (envelope-from greg@codeconcepts.com) Received: from katin.codeconcepts.com (localhost.codeconcepts.com [127.0.0.1]) by katin.codeconcepts.com (8.11.1/8.11.1) with ESMTP id f2R58Br89159; Mon, 26 Mar 2001 23:08:11 -0600 (CST) (envelope-from greg@katin.codeconcepts.com) Message-Id: <200103270508.f2R58Br89159@katin.codeconcepts.com> To: Cy Schubert - ITSD Open Systems Group Cc: Jack Rusher , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? Date: Mon, 26 Mar 2001 23:08:11 -0600 From: greg Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > In message <200103270440.f2R4eRr88969@katin.codeconcepts.com>, greg > writes: > > > > > > Rather than stand in the way of progress, having the new config file > > > > format adopted by most of the major vendors would go a long way to > > > > mitigate many fears people have about a new super-config file format. > > > > > > We would obviously want everybody to join us in using the new > > > configuration format/tools. I do think somebody needs to belly up to > > > the bar and order the first drink, though. > > > > > > > Another way we can mitigate these concerns is to support two formats > > > > until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, > > > > > > This strikes me as hard for the subsystems that don't have an API in > > > place that governs how they access files. What are some of your ideas > > > of how you would do this? > > > > > > I apologize for not having followed this thread closely enough, but > > it seems to reak of AIX/ODM. If for nothing other than the sake of > > completeness, the merits and shortcomings of IBM's solution should be > > taken into account. At the very least it would shed some light onto > > the sheer magnitude of such an undertaking. > > The AIX Team here does not have much good to say about ODM. This is > just anecdotal from sharing coffee with them. Exactly. But until we can characterize precisely what "not much good to say" really means it's nothing more than FUD. The fact of the matter is AIX has "been there, done that" on, well, pick an initiative. You could certainly argue the completeness/correctness of their solutions, but hey, isn't that what this thread is all about? Greg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 23: 9:43 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id ED11937B719 for ; Mon, 26 Mar 2001 23:09:38 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id XAA08662; Mon, 26 Mar 2001 23:08:52 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08657; Mon Mar 26 23:08:35 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2R786o63811; Mon, 26 Mar 2001 23:08:06 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdB63807; Mon Mar 26 23:07:45 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2R77SR08693; Mon, 26 Mar 2001 23:07:28 -0800 (PST) Message-Id: <200103270707.f2R77SR08693@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdjd8679; Mon Mar 26 23:07:06 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Jack Rusher Cc: Cy Schubert - ITSD Open Systems Group , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-reply-to: Your message of "Mon, 26 Mar 2001 19:24:47 PST." <3AC007FF.1D9F60A3@integratus.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 26 Mar 2001 23:07:06 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This thread so far has discussed some general ideas and generally beat around the bush. What specifically are you proposing? What would specifically be changed, in which way will it specifically be changed, and what will be done to mitigate the effect of the proposed changes to heterogeneous environments like mine that need to leverage the time of staff across various platform types? Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC In message <3AC007FF.1D9F60A3@integratus.com>, Jack Rusher writes: > Cy Schubert - ITSD Open Systems Group wrote: > > > > solutions to reduce my workload. There are enough issues of platform > > incompatibility between the various UNIX platforms out there without > > going out of our way to create new incompatibilities. > > This is the primary reason for the offense. :-) "I don't want to > learn another way of doing things, I want this OS to behave like my > other POSIX OSes, etc." These are all good reasons not to want to do > this sort of thing. On the other hand, these are exactly the reasons > that Rob Pike is mostly right when he says that OS work is stagnant (ok, > he says it's dead). > > If you're going to innovate, you're going to break compatibility. > That's the nature of the beast. Unfortunately, it means that only a > non-UNIX like Plan9 or (dare I say it?) Linux will be able to make > fundamental change without pissing people off. I guess in the Linux > case, it's just that they don't mind pissing people off. ;-) > > The caveat to the above is that people are accepting of good ideas > that don't change the Old Way(tm). I would cite kqueue() as a nice idea > that doesn't port to anyone else's playground. It doesn't scare anybody > because it doesn't displace any of the old ways of doing things. > Unfortunately, some things can't be done that way. > > > In regards to FreeBSD-only config files like login.conf or > > login.access, I don't care what format they are in as long as it's > > intelligent. > > Which makes them a great place to start. I was also thinking of > writing up a little PAM module that does authentication against an XML > version of the passwd/shadow files. We are lucky enough to have > loadable module support for that subsystem, so it's a nice place to > implement a demo. > > > Rather than stand in the way of progress, having the new config file > > format adopted by most of the major vendors would go a long way to > > mitigate many fears people have about a new super-config file format. > > We would obviously want everybody to join us in using the new > configuration format/tools. I do think somebody needs to belly up to > the bar and order the first drink, though. > > > Another way we can mitigate these concerns is to support two formats > > until the other popular UNIX systems (Solaris, AIX, Tru64-UNIX, HP/UX, > > This strikes me as hard for the subsystems that don't have an API in > place that governs how they access files. What are some of your ideas > of how you would do this? > > > Don't read into this that I am offended that the config file format may > > I don't take it that way at all. This is really good feedback from > someone who is thinking carefully about the problem. > > > Thanks, > > -- > Jack Rusher, Senior Engineer | mailto:jar@integratus.com > Integratus, Inc. | http://www.integratus.com > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Mar 26 23:42:29 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 50A2E37B719 for ; Mon, 26 Mar 2001 23:42:25 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2R7fT034079; Mon, 26 Mar 2001 23:41:29 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: Cy.Schubert@uumail.gov.bc.ca Cc: jar@integratus.com, areilly@bigpond.net.au, jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-Reply-To: <200103270707.f2R77SR08693@cwsys.cwsent.com> References: <3AC007FF.1D9F60A3@integratus.com> <200103270707.f2R77SR08693@cwsys.cwsent.com> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010326234129J.jkh@osd.bsdi.com> Date: Mon, 26 Mar 2001 23:41:29 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 44 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I think the point which has been emerging from this discussion is that one can't really win in this game. In scenario A, you have an /etc/foo.conf and all the Unix die-hards ("That's *always* been in /etc/foo.conf!") and heterogeneous environment folks ("I have 437 different Unix variants here and 391 of them use /etc/foo.conf") are happy. Everyone else, who wonder just why the heck it's called /etc/foo.conf and why the heck its syntax looks like some bad late-70's ASCII terminal control character set (answer: "It is!"), is unhappy. In scenario B, you have /var/db/localhost/config.pgsql with an complete documentation set for interface programmers and set of access utilities for everyone else which totally front-ends the configuration process so that Joe User doesn't even have to know where the data is kept or what format it's in. All the non-Unix and not-so-techy people are happy. The Unix die-hards and heterogeneous folks are hunched over a toilet bowl being explosively sick and screaming for their Solaris or SCO installation media. In scenario C, you have the abstract configuration database AND you have /etc/foo.conf with some sort of fancy two-way auto-merging thingamajiggy which propagates any change made from one data representation to the other. The joe users are still happy because they can use the configuration database thingy, though they still sometimes stumble over /etc/foo.conf and wonder just why the heck it's there. The Unix die-hards, on the other hand, are highly skeptical about viability of this whole thing and proceed to prove their point by getting themselves wrapped around the axle numerous times while attempting to use both interfaces at once or getting into odd race conditions between the two interfaces. Between that and occasionally managing to turn their systems to spam by having the wrong edit back-propagate to the correct copy, well, pretty soon they're calling your scenario C solution a grossly over-engineered and highly dysfunctional hack. "Bring back scenario A!", they will cry. To put it somewhat more succinctly, I think we're all pretty wedded to our ways and only accept change when it's a significant order of magnitude improvement beyond what we currently have or it doesn't conflict with any existing scenario. Thus Unix slowly accretes functionality to itself without changing shape too much. :) - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 0:19:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from areilly.bpc-users.org (CPE-144-132-234-126.nsw.bigpond.net.au [144.132.234.126]) by hub.freebsd.org (Postfix) with SMTP id EE95A37B718 for ; Tue, 27 Mar 2001 00:19:43 -0800 (PST) (envelope-from andrew@areilly.bpc-users.org) Received: (qmail 90109 invoked from network); 27 Mar 2001 08:19:42 -0000 Received: from localhost (HELO gurney.reilly.home) (andrew@127.0.0.1) by localhost with SMTP; 27 Mar 2001 08:19:42 -0000 Date: Tue, 27 Mar 2001 18:19:41 +1000 (EST) From: Andrew Reilly Subject: Re: configuration files, XML? To: jkh@osd.bsdi.com Cc: Cy.Schubert@uumail.gov.bc.ca, jar@integratus.com, jonathan@graehl.org, freebsd-arch@FreeBSD.ORG In-Reply-To: <20010326234129J.jkh@osd.bsdi.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Message-Id: <20010327081943.EE95A37B718@hub.freebsd.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 26 Mar, Jordan Hubbard wrote: > To put it somewhat more succinctly, I think we're all pretty wedded to > our ways and only accept change when it's a significant order of > magnitude improvement beyond what we currently have or it doesn't > conflict with any existing scenario. Thus Unix slowly accretes > functionality to itself without changing shape too much. :) OK. I'm probably not alone in thinking that that's a bit of a limiting position, but I understand the arguments, and they're good. The compatability issue is big and important, and makes things significantly harder, but not impossible. It ought to still be possible to define a libconfig API that contains a set of access mechanisms that provide the functionality required by the standard tools, if not necessarily the form. Trouble is that the initial, default implementation has to be clever/mangled enough that it can use the _existing_ gamut of /etc/foo.conf files and their formats as the repository, rather than getting the immediate bennefit of a grand unified format. For that much of the process, libconfig is likely to be a read-only API in most cases, with man/vi still the preferred write API. At the very worst, it would consist of grabbing the existing config-file reading code from each application in turn, and whacking it so that the parsed results conform to the uniform in-memory structure, which the application is taught to use. Some time later, or even in parallel, the alternate libconfig(s) can be designed and tested, so that a pam-style switch is what is necessary to decide which config engine to use on a particular machine. As more write-friendly config storage formats are developed, the libconfig API can be extend to include the write access mechanisms required by configurators. These would just be ENOTIMP stubs in the "compatability" libconfig.so. One step at a time is the strategy. The trick seems to be finding small enough steps, and pointing them in the right direction. -- Andrew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 0:44: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 4706E37B718 for ; Tue, 27 Mar 2001 00:44:06 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2R8hH034336; Tue, 27 Mar 2001 00:43:17 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: areilly@bigpond.net.au Cc: Cy.Schubert@uumail.gov.bc.ca, jar@integratus.com, jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-Reply-To: <20010327081943.EE95A37B718@hub.freebsd.org> References: <20010326234129J.jkh@osd.bsdi.com> <20010327081943.EE95A37B718@hub.freebsd.org> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010327004317J.jkh@osd.bsdi.com> Date: Tue, 27 Mar 2001 00:43:17 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 6 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Well, to be fair, I could actually see a very incremental approach like this working, but there would still have to be a significant upside other than "it's cleaner" just to get it accepted. Perhaps if you also provided a nice configuration editor or something. :-) - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 1:43:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id 8879E37B718 for ; Tue, 27 Mar 2001 01:43:50 -0800 (PST) (envelope-from bp@butya.kz) Received: by relay.butya.kz (Postfix, from userid 1000) id D981D28675; Tue, 27 Mar 2001 16:43:45 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id BFB5228638; Tue, 27 Mar 2001 16:43:45 +0700 (ALMST) Date: Tue, 27 Mar 2001 16:43:45 +0700 (ALMST) From: Boris Popov To: Andrew Reilly Cc: jkh@osd.bsdi.com, Cy.Schubert@uumail.gov.bc.ca, jar@integratus.com, jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-Reply-To: <20010327081943.EE95A37B718@hub.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 27 Mar 2001, Andrew Reilly wrote: > It ought to still be possible to define a libconfig API that > contains a set of access mechanisms that provide the functionality > required by the standard tools, if not necessarily the form. > Trouble is that the initial, default implementation has to be > clever/mangled enough that it can use the _existing_ gamut of > /etc/foo.conf files and their formats as the repository, rather > than getting the immediate bennefit of a grand unified format. Yes, this sounds like a right way to go. Initially libconfig will contain many hacks (or just switches like h = cfg_open("/etc/blah.conf", CFGT_SHELL_VARS)) to handle different formats of /etc/* files. It should be possible to select for old or new config type for individual standard files. All new programs (including ports) may support only new format(s). This way old unix gurus will be happy with old-style files, and newbies may choose a new style (assuming that there will be a nice utility with all sorts of buttons, lists etc.) > For that much of the process, libconfig is likely to be a > read-only API in most cases, with man/vi still the preferred write > API. At the very worst, it would consist of grabbing the existing > config-file reading code from each application in turn, and > whacking it so that the parsed results conform to the uniform > in-memory structure, which the application is taught to use. There is already some standards like DOM. And the primary task for configng will be design of libconfig API which should be flexible enough to deal with different types of configuration files. -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 1:45:58 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dt051n37.san.rr.com (dt051n37.san.rr.com [204.210.32.55]) by hub.freebsd.org (Postfix) with ESMTP id 1701137B719 for ; Tue, 27 Mar 2001 01:45:56 -0800 (PST) (envelope-from DougB@DougBarton.net) Received: from DougBarton.net (Studded@master [10.0.0.2]) by dt051n37.san.rr.com (8.9.3/8.9.3) with ESMTP id BAA78644 for ; Tue, 27 Mar 2001 01:45:55 -0800 (PST) (envelope-from DougB@DougBarton.net) Message-ID: <3AC06153.EEBF632E@DougBarton.net> Date: Tue, 27 Mar 2001 01:45:55 -0800 From: Doug Barton Organization: Triborough Bridge & Tunnel Authority X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files References: <20010326234129J.jkh@osd.bsdi.com> <20010327081943.EE95A37B718@hub.freebsd.org> <20010327004317J.jkh@osd.bsdi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Can someone please, just for fun, define the actual _problem_ you're trying to solve. All I've seen so far are solutions [sic], but no real good definition of the goals you're trying to achieve. Without this, continuing to talk about solutions is futile. Doug -- Perhaps the greatest damage the American system of education has done to its children is to teach them that their opinions are relevant simply because they are their opinions. Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 2:11: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3885837B719 for ; Tue, 27 Mar 2001 02:11:02 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2RAB0F24789; Tue, 27 Mar 2001 02:11:00 -0800 (PST) Date: Tue, 27 Mar 2001 02:11:00 -0800 From: Alfred Perlstein To: Doug Barton Cc: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files Message-ID: <20010327021100.V9431@fw.wintelcom.net> References: <20010326234129J.jkh@osd.bsdi.com> <20010327081943.EE95A37B718@hub.freebsd.org> <20010327004317J.jkh@osd.bsdi.com> <3AC06153.EEBF632E@DougBarton.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3AC06153.EEBF632E@DougBarton.net>; from DougB@DougBarton.net on Tue, Mar 27, 2001 at 01:45:55AM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Doug Barton [010327 01:46] wrote: > Can someone please, just for fun, define the actual _problem_ you're trying > to solve. All I've seen so far are solutions [sic], but no real good > definition of the goals you're trying to achieve. Without this, continuing > to talk about solutions is futile. And thus the point of XML is realized... A solution for a problem that doesn't exist. :) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 9:55: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 6DBD037B719 for ; Tue, 27 Mar 2001 09:55:01 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2RHvxZ03254; Tue, 27 Mar 2001 09:57:59 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC0D4A7.CF09E4A3@integratus.com> Date: Tue, 27 Mar 2001 09:57:59 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: "freebsd-arch@FreeBSD.ORG" Subject: Re: configuration files, XML? References: <200103270707.f2R77SR08693@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group wrote: > > around the bush. What specifically are you proposing? What would This is a mass reply. I will now address the following: o What's the problem? Unix file formats have traditionally been created in an ad hoc fashion in whatever format the author of that subsystem felt like. This leads to a seemingly random collection of position dependent, tagged, and line oriented file formats. The authors of the subsystems in question were likely to repeat the same set of parser errors with every subsystem. Steve Johnson's lex & yacc have made common input handling mistakes less frequent, but not everyone uses that technology. And, even if they DO use lex/yacc, it's so flexible that configuration file formats still remain random. Unix file formats don't contain self descriptive metadata (some people say "holographic labels"). This context information is really helpful in preventing people from making silly mistakes whilst editing these files. There is no constraint language for most configuration files. This makes it hard to write programs that can edit these files. You would essentially need to write one custom program for each format to make this work. This sucks for several reasons: sysinstall is harder to write & maintain, a "system configurator" is a big pain in the ass, and writing a netconfigd that allows machines to be reconfigured from the network is an artificially harder problem. There are more, but that seems like enough. o How does this relate to AIX? The point of this little project isn't to produce IBM's SMIT for BSD. Someone can write a SMIT if they feel that it'll make the end user experience more pleasant. The real point is that you've got a platform on which to write such a beast, or anything else that programmatically mucks with system configuration. I'm sure Son of Sysinstall would like this functionality. o What do you want to do about it? What I would like to see is a set of constraint files that describe the syntax of configuration files on the system, a consistent "style" for these file formats, and an API to access a library that knows how to deal with the underlying files. I would suggest that the library support loadable file format modules and that a hacked up constraint language that's able to express current file formats is the first module we write. After we have that much done, some enterprising soul could go around and retrofit this configuration file library into existing applications and subsystems. Note: at this point no user has experienced a user interface change. After we have all of this done, new applications and subsystems can code to the new API and avoid the common mistakes mentioned above & we can start experimenting with different back ends (XML, whatever) using an /etc/conf.conf approach. If we pull off the library correctly, we get conversion betwixt file formats pretty cheaply. Thanks, -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 10:47:27 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 71DE737B718 for ; Tue, 27 Mar 2001 10:47:24 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id LAA35840; Tue, 27 Mar 2001 11:47:19 -0700 Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp10.phx.gblx.net, id smtpd5Fvtia; Tue Mar 27 11:47:14 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id LAA12668; Tue, 27 Mar 2001 11:47:01 -0700 (MST) From: Terry Lambert Message-Id: <200103271847.LAA12668@usr05.primenet.com> Subject: Re: configuration files, XML? To: rjesup@wgate.com Date: Tue, 27 Mar 2001 18:46:45 +0000 (GMT) Cc: Cy.Schubert@uumail.gov.bc.ca (Cy Schubert - ITSD Open Systems Group), jar@integratus.com (Jack Rusher), areilly@bigpond.net.au (Andrew Reilly), jkh@osd.bsdi.com (Jordan Hubbard), jonathan@graehl.org, freebsd-arch@FreeBSD.ORG In-Reply-To: from "Randell Jesup" at Mar 26, 2001 03:21:23 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I'm going to be shot, but... > >I think it has less to do with offending people than having an O/S that > >is compatible with the other UNIX O/S's out there. For example, > >syslog.conf, services, protocols, inetd.conf, to name a few can be > >maintained and copied to FreeBSD, Linux (excepting inetd.conf), > > This is a very good point. And the rc files. BSD is gratuitously different. > Certainly. There are some possible tricks for dealing with > the compatibility issues, especially if we can have some sort of > programmatic conversion (especially if it can work both ways). > This is a little like how we handle /etc/passwd, but with two-way > automatic conversions. We could call the conversion utilities "niload" and "nidump"... (I hated them on NeXTStep; I think that caching configuration data is a harmful practice). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 11:21:12 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id D077337B719 for ; Tue, 27 Mar 2001 11:21:10 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2RJKn036683; Tue, 27 Mar 2001 11:20:50 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: DougB@DougBarton.net Cc: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files In-Reply-To: <3AC06153.EEBF632E@DougBarton.net> References: <20010327081943.EE95A37B718@hub.freebsd.org> <20010327004317J.jkh@osd.bsdi.com> <3AC06153.EEBF632E@DougBarton.net> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010327112049F.jkh@osd.bsdi.com> Date: Tue, 27 Mar 2001 11:20:49 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 18 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Can someone please, just for fun, define the actual _problem_ you're trying > to solve. All I've seen so far are solutions [sic], but no real good > definition of the goals you're trying to achieve. Without this, continuing > to talk about solutions is futile. I think the "problem" is pretty obvious, Doug. We have a whole bunch of system and application configuration data living in /etc and a few other places. Almost every configuration file has its own unique format and set of rules about how you're supposed to edit it or what utility (foo_mkdb) you're supposed to run after editing it so that its backing database, if it has one, is updated. It is, in short, a highly ad-hoc system which has evolved over time rather than benefitting from a single coherent design strategy. Somehow retrofitting a single coherent design strategy into this mess without pissing off all the legacy people *is* the goal people are discussing here. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 11:26: 1 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id C2B1F37B719 for ; Tue, 27 Mar 2001 11:25:59 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2RJPc036704; Tue, 27 Mar 2001 11:25:38 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: bright@wintelcom.net Cc: DougB@DougBarton.net, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files In-Reply-To: <20010327021100.V9431@fw.wintelcom.net> References: <20010327004317J.jkh@osd.bsdi.com> <3AC06153.EEBF632E@DougBarton.net> <20010327021100.V9431@fw.wintelcom.net> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010327112538N.jkh@osd.bsdi.com> Date: Tue, 27 Mar 2001 11:25:38 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 13 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > And thus the point of XML is realized... > A solution for a problem that doesn't exist. :) And that's a statement made more for amusement value than a representation of actual fact. XML is a solution for a lot of problems and it's already been widely deployed on a lot of them. It's not XML's fault that people have attempted to use it for things it is _not_ well suited for ("when all you have is a hammer...") and whether or not it's suitable for the stuff currently under discussion is less relevent than whether or not people can agree on ANY representational format that isn't exactly the same as what we're using now. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 12:14:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 7F7F637B719 for ; Tue, 27 Mar 2001 12:14:36 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id NAA12980; Tue, 27 Mar 2001 13:14:34 -0700 Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp10.phx.gblx.net, id smtpdp3A0qa; Tue Mar 27 13:14:27 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id NAA14374; Tue, 27 Mar 2001 13:14:25 -0700 (MST) From: Terry Lambert Message-Id: <200103272014.NAA14374@usr05.primenet.com> Subject: Re: configuration files To: jkh@osd.bsdi.com (Jordan Hubbard) Date: Tue, 27 Mar 2001 20:14:24 +0000 (GMT) Cc: bright@wintelcom.net, DougB@DougBarton.net, freebsd-arch@FreeBSD.ORG In-Reply-To: <20010327112538N.jkh@osd.bsdi.com> from "Jordan Hubbard" at Mar 27, 2001 11:25:38 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > And thus the point of XML is realized... > > A solution for a problem that doesn't exist. :) > > And that's a statement made more for amusement value than a > representation of actual fact. XML is a solution for a lot of > problems and it's already been widely deployed on a lot of them. It's > not XML's fault that people have attempted to use it for things it is > _not_ well suited for ("when all you have is a hammer...") and whether > or not it's suitable for the stuff currently under discussion is less > relevent than whether or not people can agree on ANY representational > format that isn't exactly the same as what we're using now. A real problem with XML is that there is very little in the way of schema standardization out there (the one big exception is EDI of financial data, which was standardized almost the day XML came out). There's also the problem of divorcing object data from any standardized accessor/mutator functions, which pretty much damages any reasonable ability to do generalized triggers to do things like regenerate sendmail.cf files when configuration data is changed in the configuration base. It's somewhat of a nasty problem, if you want to do this a minimal number of times, given a large number of changes whose groupings are designed by a UI person's idea of what's "logical", rather than the data entry screens being in third normal form. For that reason, it's generally much more useful to have a protocol gating access to your data model, rather than just a raw data model sitting around somewhere, with no way to demark transactions into "do these changes atomically, and generate the new sendmail.cf file only once, please". What this basically means is that it's great, if you are doing code that you don't expect to interoperate with anyone elses code, and less great otherwise. Or to rip of an old saying "The wonderful thing about XML schema standards are that there's so many to choose from". The primary reason I see it being used in places like IBM is that it can tunnel RPC calls and other data over HTTP, which people tend to let through firewalls. In other words, it is capable of routing around anal retentive security types, who live in deathly fear of FTP and DNS. IMO, XML was practically invented just to get around IBM network security. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 13: 6:52 2001 Delivered-To: freebsd-arch@freebsd.org Received: from molly.straylight.com (molly.straylight.com [209.68.199.242]) by hub.freebsd.org (Postfix) with ESMTP id D335B37B719 for ; Tue, 27 Mar 2001 13:06:48 -0800 (PST) (envelope-from jonathan@graehl.org) Received: from dickie (case.straylight.com [209.68.199.244]) by molly.straylight.com (8.11.0/8.10.0) with SMTP id f2RL6eE32280; Tue, 27 Mar 2001 13:06:40 -0800 From: "Jonathan Graehl" To: "Terry Lambert" Cc: "freebsd-Arch" Subject: RE: configuration files Date: Tue, 27 Mar 2001 13:06:14 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <200103272014.NAA14374@usr05.primenet.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry said: > A real problem with XML is that there is very little in the > way of schema standardization out there (the one big exception > is EDI of financial data, which was standardized almost the > day XML came out). This is not a problem with the XML spec; what would you change about it that would solve this problem? I believe, for example, that standardization of XML DTDs for product descriptions of components (chips, fasteners, etc.) has been moving forward. > There's also the problem of divorcing object data from any > standardized accessor/mutator functions, which pretty much > damages any reasonable ability to do generalized triggers to > do things like regenerate sendmail.cf files when configuration > data is changed in the configuration base. It's somewhat of > a nasty problem, if you want to do this a minimal number of > times, given a large number of changes whose groupings are > designed by a UI person's idea of what's "logical", rather > than the data entry screens being in third normal form. I don't see why the DTD couldn't allow specification of actions to perform when modifying document subtrees. If the XML file were the primary source of configuration information for my program, I would not want any accessor/mutator functions at all; I would simply provide validation constraints and construct what I need from the configuration file on startup, or when signaled. > For that reason, it's generally much more useful to have a > protocol gating access to your data model, rather than just a > raw data model sitting around somewhere, with no way to demark > transactions into "do these changes atomically, and generate > the new sendmail.cf file only once, please". Suggestions? I agree that a uniform protocol for making configuration changes active is as important as a uniform format for describing them (although the action to make the changes active could itself be described in the schema). I would think the editor would signal the running daemon to reconfigure itself from the source data, after initiating any post-modification steps indicated in the XML configuration file. > What this basically means is that it's great, if you are doing > code that you don't expect to interoperate with anyone elses > code, and less great otherwise. XML gives you a data interchange format, not an actual protocol. Shared XML schemas can be extended while still allowing for base interoperability. > The primary reason I see it being used in places like IBM is > that it can tunnel RPC calls and other data over HTTP, which > people tend to let through firewalls. In other words, it is > capable of routing around anal retentive security types, who > live in deathly fear of FTP and DNS. IMO, XML was practically > invented just to get around IBM network security. I don't understand: why is XML necessary to tunnel your application-specific messages over HTTP? If I wanted to bypass IBM network security for my application, I could roll my own data interchange format that happens to look like HTTP requests/replies involving the usual port. XML-RPC-HTTP facilities may have been designed to bypass crude network filtering, but XML was not. > > Terry Lambert > terry@lambert.org > -- Jonathan Graehl http://jonathan.graehl.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 16:49:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 841EE37B718 for ; Tue, 27 Mar 2001 16:49:50 -0800 (PST) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id RAA11400; Tue, 27 Mar 2001 17:42:58 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp02.primenet.com, id smtpdAAATqaqpw; Tue Mar 27 17:42:54 2001 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id RAA26184; Tue, 27 Mar 2001 17:49:44 -0700 (MST) From: Terry Lambert Message-Id: <200103280049.RAA26184@usr01.primenet.com> Subject: Re: configuration files To: jonathan@graehl.org (Jonathan Graehl) Date: Wed, 28 Mar 2001 00:49:43 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), freebsd-arch@FreeBSD.ORG (freebsd-Arch) In-Reply-To: from "Jonathan Graehl" at Mar 27, 2001 01:06:14 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > A real problem with XML is that there is very little in the > > way of schema standardization out there (the one big exception > > is EDI of financial data, which was standardized almost the > > day XML came out). > > This is not a problem with the XML spec; what would you change about it that > would solve this problem? Require that schema be published whenever data is published, so that someone who wants to interpret the data has the necessary schema at hand to do the job. As it stands, it's permissable for programs to have implied contracts about schema, which means they will only interoperate with each other, unless they are reverse engineered.. > I believe, for example, that standardization of XML > DTDs for product descriptions of components (chips, fasteners, etc.) has been > moving forward. That's really the whole "B2B Exchange" fallacy. I'm convinced that that is really going nowhere. It effectively requires that fastener suppliers be willing to publish data on an exchange, where the primary determining factor between vendors is the lowest commodity price. There are now two companies that I'm aware of who are trying to "value add" to this model, by using additional information, such as delivery dates, order bundling, and so on. But as things currently stand, you would have to be a fool, if you wre a supplier who participated in one of these exchanges, where the primary factor will end up being price. > I don't see why the DTD couldn't allow specification of actions to > perform when modifying document subtrees. The question is not whether it could, but whether you can rely on it doing so. > If the XML file were > the primary source of configuration information for my program, I > would not want any accessor/mutator functions at all; I would > simply provide validation constraints and construct what I need > from the configuration file on startup, or when signaled. This gets back to the cached configuration data problem. For the sendmail example, the host name might change, and sendmail would not be aware of the change, and would therefore answer connection requests with incorrect data. Similarly, the need to HUP a bind serving 10,000 domains takes the bind out of service for the reload period (DNSUPDAT can help this, but only if you hack bind to permit zone creation/deletion via DNSUPDAT). The problem with this approach is that you are, in effect, making your server unreliable, in exchange for making the changes take effect more quickly. Really, you are interested in hard and soft service interdependencies, which are very difficult to express in XML, and which are equally difficult to express in such a way as to ensure ordering, atomicity, and idempotence for transactions. At Whistle/IBM GSB, there were serious interaction and QOS issues that derived from the "restartd" architecture. It was a good idea, which lacked in complexity of execution. It was in the process of being redesigned -- with the configuration store fronted by a single protocol point. I didn't entirely agree with the design, but it was a far sight better than the predecessor, and solved many issues. Link management in the InterJet is what I would call a data interface, rather than a procedural/functional interface. While everything was under our control, per se, so we avoided the many issues surround use of data interfaces and schema synchronization, you have only to look as far as "ps", "netstat", and "libkvm" for validation of the idea that data interfaces are inherently untrustworthy. In 20/20 hindsight, there should have been a link management daemon, which sat on a link management device, and decided, based on who was asking, whether the policy currently in effect should permit or prohibit the request, and attempts to connect a socket (which, if permitted, would result in a demand link up) should be administratively denied in the kernel, and return an error indicating administrative denial. It is simply not acceptable to have to take on the maintenance burden of teching every open source package on the planet how to do its own link management, without a centralized policy enforcement mechanism. By the same token, a centralize policy enforcement mechanism is needed for UNIX configuration data as well. Vanilla XML, without a protocol, API, or other procedural access point to provide subschema enforcement of your validation constraints suffers many of the same weaknesses we fought.. > > For that reason, it's generally much more useful to have a > > protocol gating access to your data model, rather than just a > > raw data model sitting around somewhere, with no way to demark > > transactions into "do these changes atomically, and generate > > the new sendmail.cf file only once, please". > > Suggestions? I agree that a uniform protocol for making configuration changes > active is as important as a uniform format for describing them (although the > action to make the changes active could itself be described in the schema). I > would think the editor would signal the running daemon to reconfigure itself > from the source data, after initiating any post-modification steps indicated > in the XML configuration file. ACAP is one option. My personal preference would be LTAP, if the LDAP server modifications for asynchronous notification could be pried free from Lucent. LDAP would work, if a simple transaction extension were added (nesting would not be required, if data were properly normalized). The back end data store for the protocol server could be XML (or ROT-13 binary files; it doesn't matter). I have to say I'm violently opposed to signalling as a catch-all; I understand that it's the standard means of informing a daemon of a configuration change, but an overly simplistic, low granularity approach is exactly what must be avoided. As another example from the InterJet, it ran a split horizon DNS. When the link came up, it changed the IP address in the external DNS; in most cases there was no reason for this: in the case of a static IP address, the address would never change. In the case of a dynamic IP address, the address would change to the point of it being useless to run a DNS on the external port. If this had to run, the most correct approach would be to use DNSUPDAT. Because the external mail server (which was used for fingerd, connection up-based SMTP trigger, and (later), ODMR dependended on external DNS, when external DNS had to be restarted to change the IP address, so did mail and other services dependent upon DNS services. But sendmail did not care about _any_ DNS changes: it only cared about A record (canonical server name) changes. But the system was insufficiently granular to distinguish different types of data changes in DNS, and so customers suffered the consequences of their mail server being hit over the head with a brickbat, and therefore not being up at precisely the time that the ISP mail server was attempting to contact them. > > What this basically means is that it's great, if you are doing > > code that you don't expect to interoperate with anyone elses > > code, and less great otherwise. > > XML gives you a data interchange format, not an actual protocol. Shared XML > schemas can be extended while still allowing for base interoperability. I'm (I guess) known for the statement "standard plus extensions is not standard". I think that deviations from standards render code practically useless. > > The primary reason I see it being used in places like IBM is > > that it can tunnel RPC calls and other data over HTTP, which > > people tend to let through firewalls. In other words, it is > > capable of routing around anal retentive security types, who > > live in deathly fear of FTP and DNS. IMO, XML was practically > > invented just to get around IBM network security. > > I don't understand: why is XML necessary to tunnel your application-specific > messages over HTTP? If I wanted to bypass IBM network security for my > application, I could roll my own data interchange format that happens to look > like HTTP requests/replies involving the usual port. XML-RPC-HTTP facilities > may have been designed to bypass crude network filtering, but XML was not. I was being facetious. I don't see XML being a panacea. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 17:30:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from molly.straylight.com (molly.straylight.com [209.68.199.242]) by hub.freebsd.org (Postfix) with ESMTP id A7E7B37B719 for ; Tue, 27 Mar 2001 17:30:21 -0800 (PST) (envelope-from jonathan@graehl.org) Received: from dickie (case.straylight.com [209.68.199.244]) by molly.straylight.com (8.11.0/8.10.0) with SMTP id f2S1UEE01539; Tue, 27 Mar 2001 17:30:14 -0800 From: "Jonathan Graehl" To: "Terry Lambert" Cc: "freebsd-Arch" Subject: RE: configuration files Date: Tue, 27 Mar 2001 17:29:50 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <200103280049.RAA26184@usr01.primenet.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > If the XML file were > > the primary source of configuration information for my program, I > > would not want any accessor/mutator functions at all; I would > > simply provide validation constraints and construct what I need > > from the configuration file on startup, or when signaled. > > This gets back to the cached configuration data problem. For > the sendmail example, the host name might change, and sendmail > would not be aware of the change, and would therefore answer > connection requests with incorrect data. > > Similarly, the need to HUP a bind serving 10,000 domains takes > the bind out of service for the reload period (DNSUPDAT can help > this, but only if you hack bind to permit zone creation/deletion > via DNSUPDAT). > > The problem with this approach is that you are, in effect, making > your server unreliable, in exchange for making the changes take > effect more quickly. Excellent point - I did not consider the need to incrementally (atomically) update huge configuration stores while the server still acts on the old configuration. > It is simply not acceptable to have to take on the maintenance burden > of teching every open source package on the planet how to do its own > link management, without a centralized policy enforcement mechanism. I agree. Similarly, we can't expect every package to spontaneously come up with a common configuration file format ;) > By the same token, a centralize policy enforcement mechanism is > needed for UNIX configuration data as well. > > Vanilla XML, without a protocol, API, or other procedural access > point to provide subschema enforcement of your validation > constraints suffers many of the same weaknesses we fought.. I agree; vanilla XML simply replaces many arbitrary data formats with a single flexible format (with an explicit DTD). Having XML configuration files would be wonderful, but you are right, there are other problems to solve as well. > ACAP is one option. My personal preference would be LTAP, if the > LDAP server modifications for asynchronous notification could be > pried free from Lucent. LDAP would work, if a simple transaction > extension were added (nesting would not be required, if data were > properly normalized). The back end data store for the protocol > server could be XML (or ROT-13 binary files; it doesn't matter). First I've heard of the two ... http://asg.web.cmu.edu/acap/ ACAP RFC: ftp://ftp.isi.edu/in-notes/rfc2244.txt (looks like a nice idea; web page seems to be neglected) http://www-db.research.bell-labs.com/project/ltap/ (patent filed?) LTAP looks like a gateway to a backend that lacks triggers, where you access the backend through the gateway and you can request notifications when certain accesses are made by others? Do you like their protocol, or just the idea? > I have to say I'm violently opposed to signaling as a catch-all; > I understand that it's the standard means of informing a daemon > of a configuration change, but an overly simplistic, low granularity > approach is exactly what must be avoided. Well, like it or not, it is the easiest way to do things. A basic configuration file meta-format and signaling mechanism would still be good. I agree that more fine-grained stuff for updating in-use configuration data should be developed, and that such a beast might as well have nothing to do with XML (except that an XML language could describe the interfaces/data types :). In cases where static configuration files make sense, I would like those configuration files to be XML (or some other unified meta-format which can be programmatically operated on without case-specific coding). > > > XML gives you a data interchange format, not an actual protocol. Shared XML > > schemas can be extended while still allowing for base interoperability. > > I'm (I guess) known for the statement "standard plus extensions > is not standard". I think that deviations from standards render > code practically useless. The "extension" I was referring to was that of augmenting a DTD such that it still has enough in common with the "standard DTD" that programs will still be able to get the standard information out of your document. I was not referring to an extension to XML itself. A protocol for passing around bits of XML data is outside the scope of the XML standard itself. I didn't realize that people passed around XML documents without publicizing their DTDs. That sucks. I'm not sure a requirement to include the DTD to be "standards compliant" would prevent people from being proprietary if they still want to. -- Jonathan Graehl http://jonathan.graehl.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 27 20:48:21 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 8802337B719 for ; Tue, 27 Mar 2001 20:48:18 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id UAA13999; Tue, 27 Mar 2001 20:46:13 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda13997; Tue Mar 27 20:46:05 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2S4jun72509; Tue, 27 Mar 2001 20:45:56 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdp72505; Tue Mar 27 20:45:26 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2S4j5S14065; Tue, 27 Mar 2001 20:45:05 -0800 (PST) Message-Id: <200103280445.f2S4j5S14065@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdU14058; Tue Mar 27 20:44:52 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Terry Lambert Cc: rjesup@wgate.com, Cy.Schubert@uumail.gov.bc.ca (Cy Schubert - ITSD Open Systems\ Group), jar@integratus.com (Jack Rusher), areilly@bigpond.net.au (Andrew Reilly), jkh@osd.bsdi.com (Jordan Hubbard), jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? In-reply-to: Your message of "Tue, 27 Mar 2001 18:46:45 GMT." <200103271847.LAA12668@usr05.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 27 Mar 2001 20:44:52 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200103271847.LAA12668@usr05.primenet.com>, Terry Lambert writes: > I'm going to be shot, but... > > > >I think it has less to do with offending people than having an O/S that > > >is compatible with the other UNIX O/S's out there. For example, > > >syslog.conf, services, protocols, inetd.conf, to name a few can be > > >maintained and copied to FreeBSD, Linux (excepting inetd.conf), > > > > This is a very good point. > > And the rc files. BSD is gratuitously different. This is going to sound like sacrilege and inflammatory, especially to a BSD crowd, however most of what is out there, Solaris, AIX, DG-UX, Tru64-UNIX (in a weird sort of way), just to name a couple, try to conform the the SYSV standard. Not that I'm proposing that we do this now but it might be a good idea to have two init's, one for BSD and the other for SYSV compatibility or to have our init be able to optionally behave SYSV-like through some flag or #ifdef. I'm not proposing scrapping our init or its default behaviour. I'm not just talking about rc files but run levels and init's management of process creation & termination. However I would be happy with just the rc file structure, only because it helps me leverage my time better across platforms. /etc/rc.d might be a good place to start. This is something I've been thinking a lot about over the last couple of years and I've been pondering implementing this idea using a port initially and subsequently presenting it to this list for discussion someday. Of course if the FreeBSD community doesn't want it, it would be a total waste of time. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 0:55:38 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id 79A0C37B71A for ; Wed, 28 Mar 2001 00:55:36 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f2S8tVX40826; Wed, 28 Mar 2001 00:55:31 -0800 (PST) (envelope-from obrien) Date: Wed, 28 Mar 2001 00:55:31 -0800 From: "David O'Brien" To: Warner Losh Cc: arch@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules Message-ID: <20010328005531.D18676@dragon.nuxi.com> Reply-To: obrien@FreeBSD.ORG References: <20010325214658.F3241@ringworld.oblivion.bg> <20010324211140.C4304@ringworld.oblivion.bg> <200103251840.f2PIeE973790@harmony.village.org> <20010325214658.F3241@ringworld.oblivion.bg> <200103251915.f2PJF0980349@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103251915.f2PJF0980349@harmony.village.org>; from imp@harmony.village.org on Sun, Mar 25, 2001 at 12:15:00PM -0700 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Mar 25, 2001 at 12:15:00PM -0700, Warner Losh wrote: > Note, I had to patch the Makefile.${MACHINE_ARCH} as well, so the > complete patch looks more like: Commit [to -current]!! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 3:33:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from nothing-going-on.demon.co.uk (unknown [194.128.198.234]) by hub.freebsd.org (Postfix) with ESMTP id A328537B71E for ; Wed, 28 Mar 2001 03:33:53 -0800 (PST) (envelope-from nik@nothing-going-on.demon.co.uk) Received: (from nik@localhost) by nothing-going-on.demon.co.uk (8.11.1/8.11.1) id f2SB2R847341; Wed, 28 Mar 2001 12:02:27 +0100 (BST) (envelope-from nik) Date: Wed, 28 Mar 2001 12:02:27 +0100 From: Nik Clayton To: Cy Schubert - ITSD Open Systems Group Cc: Terry Lambert , rjesup@wgate.com, Jack Rusher , Andrew Reilly , Jordan Hubbard , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? Message-ID: <20010328120227.B46989@canyon.nothing-going-on.org> References: <200103271847.LAA12668@usr05.primenet.com> <200103280445.f2S4j5S14065@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="cmJC7u66zC7hs+87" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103280445.f2S4j5S14065@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Tue, Mar 27, 2001 at 08:44:52PM -0800 Organization: FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --cmJC7u66zC7hs+87 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 27, 2001 at 08:44:52PM -0800, Cy Schubert - ITSD Open Systems G= roup wrote: > This is something I've been thinking a lot about over the last couple=20 > of years and I've been pondering implementing this idea using a port=20 > initially and subsequently presenting it to this list for discussion=20 > someday. Of course if the FreeBSD community doesn't want it, it would=20 > be a total waste of time. I think doing this as a port would be an excellent idea. The "FreeBSD community" is a wide and diverse group, and while a segment of that community might say "Go away, we don't want to know", I'm sure other segments will be interested. By making this work a port you make it *much* easier for people to experiment with your work, without any of the obstacles you would face if you wanted to get it integrated in to the system immediately. N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ FreeBSD Documentation Project http://www.freebsd.org/docproj/ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- --cmJC7u66zC7hs+87 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (FreeBSD) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjrBxMIACgkQk6gHZCw343Uj3gCfWlQ89+r0D+LuR1jt6uaTN2OF UA4AnREpqYtneWgutxpJNAw3OGLv0jVH =r4m0 -----END PGP SIGNATURE----- --cmJC7u66zC7hs+87-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 4:44:14 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id AE05137B71C for ; Wed, 28 Mar 2001 04:44:07 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id FAA06603; Wed, 28 Mar 2001 05:42:33 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp01.primenet.com, id smtpdAAA6xaWOm; Wed Mar 28 05:42:14 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id FAA03936; Wed, 28 Mar 2001 05:43:43 -0700 (MST) From: Terry Lambert Message-Id: <200103281243.FAA03936@usr05.primenet.com> Subject: Re: configuration files To: jonathan@graehl.org (Jonathan Graehl) Date: Wed, 28 Mar 2001 12:43:43 +0000 (GMT) Cc: freebsd-arch@freebsd.org In-Reply-To: from "Jonathan Graehl" at Mar 27, 2001 05:29:50 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > The problem with this approach is that you are, in effect, making > > your server unreliable, in exchange for making the changes take > > effect more quickly. > > Excellent point - I did not consider the need to incrementally (atomically) > update huge configuration stores while the server still acts on the old > configuration. It's my biggest peeve. 8-). > > It is simply not acceptable to have to take on the maintenance burden > > of teching every open source package on the planet how to do its own > > link management, without a centralized policy enforcement mechanism. > > I agree. Similarly, we can't expect every package to spontaneously > come up with a common configuration file format ;) I think Apache would accept the change quickly. BIND would probably accept it also, if people could keep their fingers limited to the configuration data itself, and not muck with the BIND code proper. Representing better than two dimensional hierarchical relationships requires metadata. The sendmail.cf file is an excellent example of of a confiugration file containing complex relationships, formed out of rules segments, with those segments indexed and activated several ways, including implicitly, or as a result of metadata causing rules to be invoked by particular mailer definitions, which are in turn themselves invoked by routing rules. I'd really hate to have it be my job to XML-ify sendmail configs; I guess I could do it, if I were paid enough for the task, and it wasn't going to turn into an ongoing nightmare, where I was chained to the desk as maintainer for the rest of eternity. Sort of like unclogging sewers: someone has to do it, but it's not a job people line up around the block to do, unless there is serious money on the table. It would probably be easier to arrange a "takeover" mechanism as a result of getting a HUP, where the open connections complete processing, and the open listen sockets, if they are still relevent, get taken over once a forked copy of the daemon, entirely up on the new configuration, is ready to start processing requests. Part of the whole socket thing kind of breaks down at this point, since part of the cached state we need to avoid is that sockets are bound to IP addresses, instead of interfaces, so if you change the IP address, you need to rebind (whereas if you were to bind to an interface instead, the IP address could bounce all over the map, and you really wouldn't care). > > ACAP is one option. My personal preference would be LTAP, if the > > LDAP server modifications for asynchronous notification could be > > pried free from Lucent. LDAP would work, if a simple transaction > > extension were added (nesting would not be required, if data were > > properly normalized). The back end data store for the protocol > > server could be XML (or ROT-13 binary files; it doesn't matter). > > First I've heard of the two ... > > http://asg.web.cmu.edu/acap/ > ACAP RFC: ftp://ftp.isi.edu/in-notes/rfc2244.txt > (looks like a nice idea; web page seems to be neglected) > > http://www-db.research.bell-labs.com/project/ltap/ > (patent filed?) > LTAP looks like a gateway to a backend that lacks triggers, where > you access the backend through the gateway and you can request > notifications when certain accesses are made by others? Do you > like their protocol, or just the idea? The idea. The protocol lacks acknowledgement of the asynchronus update notifications, which would permit listeners to "veto" a change (this would have to be at a transaction level, not a simple datum level). For example, if you had a machine with two network cards, you really don't want to permit both cards to have the same network address. Yes, this is a contrived example, since the real answer to that problem is to use IPv6 stateless autoconfiguration (or IPv4 stateless autoconfiguration into link.local and then NAT). > > I have to say I'm violently opposed to signaling as a catch-all; > > I understand that it's the standard means of informing a daemon > > of a configuration change, but an overly simplistic, low granularity > > approach is exactly what must be avoided. > > Well, like it or not, it is the easiest way to do things. A basic > configuration file meta-format and signaling mechanism would still > be good. I agree that more fine-grained stuff for updating in-use > configuration data should be developed, and that such a beast might > as well have nothing to do with XML (except that an XML language > could describe the interfaces/data types :). Actually, we already have the "easiest" approach: do nothing. It requires no additional work. I like the idea of "right" vs. "expedient", which has traditionally been a hallmark of BSD engineering. > In cases where static configuration files make sense, I would like > those configuration files to be XML (or some other unified > meta-format which can be programmatically operated on without > case-specific coding). I definitely agree; so long as there is sufficient metadata involved to allow a UI component (text, graphical, libdialog, whatever), it vastly simplifies adding UI to any configuration in that format. I also think that XML would be near ideal, in terms of back up and restore of configuration information. Which brings up the next issue, if we can agree to the metadata requirement (which implies interfaces that act on the data based on the content of the metadata): arbitrary hierarchical grouping of data. It's a real common problem that people want to solve these days... "how do I have the minimum set of machine and/or site specific configuration information, and how do I share as much of the configuration data as possible between multiple machines?". Hierarchy is necessary to distinguish machine, cluster, site, colocation facility, enterprise, and global configurations. On top of that, there is the concept of "role". In other words, I may have a set of machines, some of whom have the role "DNS servers", some with the role "web servers", some with the role "mail servers", etc.. > > I'm (I guess) known for the statement "standard plus extensions > > is not standard". I think that deviations from standards render > > code practically useless. > > The "extension" I was referring to was that of augmenting a DTD such that it > still has enough in common with the "standard DTD" that programs will still be > able to get the standard information out of your document. I was not referring > to an extension to XML itself. A protocol for passing around bits of XML data > is outside the scope of the XML standard itself. I'm still very leery of this; historically, SNMP has been very problematic because of just this approach. You would have to build some brakes into the process, so that they could be stomped on with force, should things start to go to hell. There are plenty of SNMP standards in RFC form from the IETF, but everyone either ignores them ("We're Cisco: we ARE the standard"), or tries to follow them, only to find that they think they need "just a tweak" so that they can "add value", or, worse "achieve market differentiation" (code for "we don't run from their config data, but they can run from ours, so you should buy ours"). > I didn't realize that people passed around XML documents without publicizing > their DTDs. That sucks. I'm not sure a requirement to include the DTD to be > "standards compliant" would prevent people from being proprietary if they still > want to. It's a common practice, actually. Mostly it has to do with how easy it is to set up private interfaces, and then not document them because your not proud of them, or because they would give hooks into your internals, and you haven't really protected them from arbitrary interfacing ("Always set 3, or it will core dump the server"). As the saying goes, "never attribute to malice...". Not that there isn't malice out there, it's just not, IMO, what's driving non-interoperability and the non-publication of DTD's. It's pretty much why Diamond didn't used to tell you how to program their RAMDAC's to get X windows running: they changed the PAL code and ROM data at the same time, but they never provided a standard layout or meand of identifying the RAMDAC data in the ROM. As a result, it wasn't possible for software to find out, from an arbitrary video card, how to program modes not covered by the VGA/SVGA/XVGA specifications (which themselves assumed INT 10 calls with interrupts disable to do a lot of the work). Because it was possible to fry the hardware with bad data, they would have had to reveal all of their RAMDAC settings (which were in the ROM data: they were just opaque to everything but the ROM code that hooked the INT 10 BIOS), AND all of the information necessary to identify their cards. At which point, they could be easily reverse engineered by clone card manufacturers. In other words, an initial engineering mistake, which led to their hardware being practically unusable outside the environements they anticipated. Diamond finally relented on documentation, when people started reverse engineering to build drivers anyway, and they have since been more careful about their assumptions, and are now generally regarded as being friendly. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 5:29:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dire.bris.ac.uk (dire.bris.ac.uk [137.222.10.60]) by hub.freebsd.org (Postfix) with ESMTP id 43DF837B71F for ; Wed, 28 Mar 2001 05:29:18 -0800 (PST) (envelope-from Jan.Grant@bristol.ac.uk) Received: from mail.ilrt.bris.ac.uk by dire.bris.ac.uk with SMTP-PRIV with ESMTP; Wed, 28 Mar 2001 14:29:04 +0100 Received: from cmjg (helo=localhost) by mail.ilrt.bris.ac.uk with local-esmtp (Exim 3.16 #1) id 14iFzA-0003QB-00; Wed, 28 Mar 2001 14:27:40 +0100 Date: Wed, 28 Mar 2001 14:27:40 +0100 (BST) From: Jan Grant To: Jack Rusher Cc: "freebsd-arch@FreeBSD.ORG" Subject: Re: configuration files, XML? In-Reply-To: <3AC0D4A7.CF09E4A3@integratus.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 27 Mar 2001, Jack Rusher wrote: > Cy Schubert - ITSD Open Systems Group wrote: > > > > around the bush. What specifically are you proposing? What would > > This is a mass reply. I will now address the following: > > o What's the problem? > > Unix file formats have traditionally been created in an ad hoc > fashion in whatever format the author of that subsystem felt like. This > leads to a seemingly random collection of position dependent, tagged, > and line oriented file formats. > o What do you want to do about it? > > What I would like to see is a set of constraint files that describe > the syntax of configuration files on the system, a consistent "style" > for these file formats, and an API to access a library that knows how to > deal with the underlying files. I would suggest that the library > support loadable file format modules and that a hacked up constraint > language that's able to express current file formats is the first module > we write. After we have that much done, some enterprising soul could go > around and retrofit this configuration file library into existing > applications and subsystems. There's a reasonable 'half-way house': use XML to define config file formats and validators (the current crop of tools available is good for this) - use XSLT to process these into the formats that external applications require. Barring the usual problems of keeping source and output in sync, this gives you a (standard) semi-declarative way of describing the relationship between XML and .conf file. You can also process and validate at the same time: see schematron (don't have a URL to hand at the moment) for example. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 8: 7:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id AA47C37B725; Wed, 28 Mar 2001 08:07:16 -0800 (PST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.11.1/8.11.1) with ESMTP id f2SG7G956166; Wed, 28 Mar 2001 09:07:16 -0700 (MST) (envelope-from imp@harmony.village.org) Message-Id: <200103281607.f2SG7G956166@harmony.village.org> To: obrien@FreeBSD.ORG Subject: Re: Building only a specified list of kernel modules Cc: arch@FreeBSD.ORG In-reply-to: Your message of "Wed, 28 Mar 2001 00:55:31 PST." <20010328005531.D18676@dragon.nuxi.com> References: <20010328005531.D18676@dragon.nuxi.com> <20010325214658.F3241@ringworld.oblivion.bg> <20010324211140.C4304@ringworld.oblivion.bg> <200103251840.f2PIeE973790@harmony.village.org> <20010325214658.F3241@ringworld.oblivion.bg> <200103251915.f2PJF0980349@harmony.village.org> Date: Wed, 28 Mar 2001 09:07:16 -0700 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010328005531.D18676@dragon.nuxi.com> "David O'Brien" writes: : On Sun, Mar 25, 2001 at 12:15:00PM -0700, Warner Losh wrote: : > Note, I had to patch the Makefile.${MACHINE_ARCH} as well, so the : > complete patch looks more like: : : Commit [to -current]!! OK. I'll do it later today or tomorrow unless someone yells loudly. I know Peter is working on a better solution, but in the interrum this will be good. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 8:39: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id DF97C37B71A for ; Wed, 28 Mar 2001 08:39:05 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2SGg8Z04031; Wed, 28 Mar 2001 08:42:09 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC21460.B5E83111@integratus.com> Date: Wed, 28 Mar 2001 08:42:08 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Jan Grant Cc: "freebsd-arch@FreeBSD.ORG" Subject: Re: configuration files, XML? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jan Grant wrote: > > Barring the usual problems of keeping source and output in sync, this > gives you a (standard) semi-declarative way of describing the > relationship between XML and .conf file. That's certainly a way to do it, and it's an interesting compromise. My primary objection is that I don't really like the idea of moving all of our configuration data to a system that expresses one of my least favorite properties of NIS; the configure/make cycle. If we can do it, I would sure like to make a more complete change. -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 9: 2:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 862E037B71E for ; Wed, 28 Mar 2001 09:02:14 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2SH5JZ04173; Wed, 28 Mar 2001 09:05:19 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC219CF.B1716422@integratus.com> Date: Wed, 28 Mar 2001 09:05:19 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: "freebsd-arch@FreeBSD.ORG" Subject: Re: configuration files References: <200103281243.FAA03936@usr05.primenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry Lambert wrote: > > > Excellent point - I did not consider the need to incrementally (atomically) > > update huge configuration stores while the server still acts on the old > > configuration. > > It's my biggest peeve. 8-). I have always wondered whether it would be worth my while to create a VMS style versioning file system (everyone who remembers DEL *.*;* raise your hand) to hold configuration files. It seems like a natural mechanism for this (although they didn't use it for that in VMS). > I think Apache would accept the change quickly. They have been slowly changing their file format to something XML like for ages. I'm surprised they haven't made the cut over yet. > BIND would probably accept it also, if people could keep their This would, of course, be harder. It would probably be easier to change everything BUT the main configuration file in BIND9; they have hooks for zone file storage now. > It would probably be easier to arrange a "takeover" mechanism as a > result of getting a HUP, where the open connections complete I appreciate the direction you're taking with this, and I feel your pain, but this is (as far as I can tell) a much harder problem than the one I am chasing right now. The HUP mechanism is obviously a holdover from another time. Hell, the whole signal handling mechanism is pretty whacked. If UNIX had been designed to support long lived servers, rather than short lived timesharing work, we would have a fourth default file descriptor called "stdcontrol" for passing out of band commands. As it is, everyone has to fake it in their own special way. Bummmer. > I definitely agree; so long as there is sufficient metadata > involved to allow a UI component (text, graphical, libdialog, > whatever), it vastly simplifies adding UI to any configuration > in that format. > > [...] > > Hierarchy is necessary to distinguish machine, cluster, site, > colocation facility, enterprise, and global configurations. On > top of that, there is the concept of "role". In other words, I > may have a set of machines, some of whom have the role "DNS > servers", some with the role "web servers", some with the role > "mail servers", etc.. These are the sorts of things that I would to see become possible. I think we need to build some cleaner infrastructure that doesn't immediately grant us any operational benefit in order to open the door to this sort of improvement. -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 13:51:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 6225A37B722 for ; Wed, 28 Mar 2001 13:51:39 -0800 (PST) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id OAA00165; Wed, 28 Mar 2001 14:48:09 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp03.primenet.com, id smtpdAAAVRa4ra; Wed Mar 28 14:48:07 2001 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id OAA12651; Wed, 28 Mar 2001 14:51:30 -0700 (MST) From: Terry Lambert Message-Id: <200103282151.OAA12651@usr02.primenet.com> Subject: Re: configuration files To: jar@integratus.com (Jack Rusher) Date: Wed, 28 Mar 2001 21:51:30 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), freebsd-arch@FreeBSD.ORG (freebsd-arch@FreeBSD.ORG) In-Reply-To: <3AC219CF.B1716422@integratus.com> from "Jack Rusher" at Mar 28, 2001 09:05:19 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > > Excellent point - I did not consider the need to incrementally (atomically) > > > update huge configuration stores while the server still acts on the old > > > configuration. > > > > It's my biggest peeve. 8-). > > I have always wondered whether it would be worth my while to create a > VMS style versioning file system (everyone who remembers DEL *.*;* raise > your hand) to hold configuration files. It seems like a natural > mechanism for this (although they didn't use it for that in VMS). VMS versioning really depends on doing globbing in the kernel instead of in user space. Despite the fact that globbing in the kernel makes it so you can pass back only the data matching your search criterian, instead of all the data available, after which you iterate through it again in user space, UNIX people really, really tend to hate kernel globbing as a concept. Without kernel globbing, you have to split up the namespace, and hack both the FS (namespace folding) and the user land tools (display versioning data from a seperate metadata request as a result of being given an option to do so). The result is that everything except special tools only see the most recent version of a file. It ends up being really, really hackish, to the point of not being worth the effort. > > I think Apache would accept the change quickly. > > They have been slowly changing their file format to something XML like > for ages. I'm surprised they haven't made the cut over yet. Yeah; I admit, Apache was an easy call. > > BIND would probably accept it also, if people could keep their > > This would, of course, be harder. It would probably be easier to > change everything BUT the main configuration file in BIND9; they have > hooks for zone file storage now. I admit that that's messay, bit it's workable. You could pretend that you had multiple bifurcation points, and use XML metadata as if the file were a directory, and you were accesseing files by bifurcation point. It's kind of the opposite of the namespace folding you have to do to make versions work in UFS. > > It would probably be easier to arrange a "takeover" mechanism as a > > result of getting a HUP, where the open connections complete > > I appreciate the direction you're taking with this, and I feel your > pain, but this is (as far as I can tell) a much harder problem than the > one I am chasing right now. The HUP mechanism is obviously a holdover > from another time. Hell, the whole signal handling mechanism is pretty > whacked. If UNIX had been designed to support long lived servers, rather > than short lived timesharing work, we would have a fourth default file > descriptor called "stdcontrol" for passing out of band commands. As it > is, everyone has to fake it in their own special way. Bummmer. Uh... kevent? (Ducks). > > I definitely agree; so long as there is sufficient metadata > > involved to allow a UI component (text, graphical, libdialog, > > whatever), it vastly simplifies adding UI to any configuration > > in that format. > > > > [...] > > > > Hierarchy is necessary to distinguish machine, cluster, site, > > colocation facility, enterprise, and global configurations. On > > top of that, there is the concept of "role". In other words, I > > may have a set of machines, some of whom have the role "DNS > > servers", some with the role "web servers", some with the role > > "mail servers", etc.. > > These are the sorts of things that I would to see become possible. I > think we need to build some cleaner infrastructure that doesn't > immediately grant us any operational benefit in order to open the door > to this sort of improvement. People hit me with tomatoes when I say things like that... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 18: 6: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 92A1337B71A for ; Wed, 28 Mar 2001 18:05:56 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2T292Z04978; Wed, 28 Mar 2001 18:09:03 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC2993E.1D3392AE@integratus.com> Date: Wed, 28 Mar 2001 18:09:02 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: "freebsd-arch@FreeBSD.ORG" Subject: Re: configuration files References: <200103282151.OAA12651@usr02.primenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry Lambert wrote: > > VMS versioning really depends on doing globbing in the kernel > instead of in user space. To get the exact VMS behavior, yes. My answer was to get in the way of namei() and parse the ;XX on the end of the file name. This is much like the approach I later found Elephant using to do the same thing (only they used times, rather than version numbers). You don't get wildcard operations against multiple versions, but you do get versioned back ups and easy roll forward after file close. It worked all right under SunOS 4.x, but I've never gotten around to reimplementing it against a modern BSD with the full complement of real features. > People hit me with tomatoes when I say things like that... I may yet be branded the village idiot and burned at the stake. We'll see. :-) -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 28 18:42:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from booyaa.hq.netapp.com (nat-198-95-226-227.netapp.com [198.95.226.227]) by hub.freebsd.org (Postfix) with ESMTP id 3EB7237B71F for ; Wed, 28 Mar 2001 18:42:55 -0800 (PST) (envelope-from dtm@foobox.net) Received: (from dtm@localhost) by booyaa.hq.netapp.com (8.11.3/8.11.3) id f2T2ge728326; Wed, 28 Mar 2001 18:42:40 -0800 (PST) (envelope-from dtm@foobox.net) X-Authentication-Warning: booyaa.hq.netapp.com: dtm set sender to dtm@foobox.net using -f To: Jordan Hubbard Cc: DougB@DougBarton.net, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files References: <20010327081943.EE95A37B718@hub.freebsd.org> <20010327004317J.jkh@osd.bsdi.com> <3AC06153.EEBF632E@DougBarton.net> <20010327112049F.jkh@osd.bsdi.com> From: Duane T Mun Date: 28 Mar 2001 18:42:40 -0800 In-Reply-To: <20010327112049F.jkh@osd.bsdi.com> Message-ID: Lines: 83 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> "JH" == Jordan Hubbard writes: JH> We have a whole bunch of system and application configuration JH> data living in /etc and a few other places. Almost every JH> configuration file has its own unique format and set of rules JH> about how you're supposed to edit it or what utility JH> (foo_mkdb) you're supposed to run after editing it so that its JH> backing database, if it has one, is updated. Ever taken a look at cfengine (http://www.iu.hioslo.no/cfengine/)? Its a system configuration tool that uses classes to define what gets done. So, lets say I don't like my root account to use csh(1), and prefer sh(1). -------------------------------------------------------------------------- editfiles: freebsd:: { ${CFTESTDIR}/etc/master.passwd ReplaceAll '/root:/bin/.sh$' With '/root:/bin/sh' DefineClasses 'rebuild_passwd' } shellcommands: freebsd.rebuild_passwd.postprocess:: "/usr/sbin/pwd_mkdb -p -d ${CFTESTDIR}/etc ${CFTESTDIR}/etc/master.passw d" -------------------------------------------------------------------------- I would use _editfiles_ to modify /etc/master.passwd. _ReplaceAll_ is similar to `sed '1,$s/.../.../g'`. Then a new class is defined _rebuild_passwd_. When _shellcommands_ is executed, it tests to see if all three classes (freebsd, rebuild_passwd, and postprocess) are defined. If so, then /usr/sbin/pwd_mkdb is run. The next time cfengine is run, the _Replace_All_ would not execute because there's no match, and the class _rebuild_passwd_ would not be defined, so the _shellcommands_ stuff shown will also not run. BTW, the ${CFTESTDIR} is an environment variable that is passed to cfengine when I'm doing testing. Here's a quote from the docs: Cfengine is a tool for setting up and maintaining BSD and System-V-like operating system optionally attached to a TCP/IP network. You can think of cfengine as a very high level language--much higher level than Perl or shell: a single statement can result in many hundreds of operations being performed on multiple hosts. Cfengine is good at performing a lot of common system administration tasks, and allows you to build on its strengths with your own scripts. You can also use it as a netwide front-end for `cron'. Once you have set up cfengine, you'll be free to use your time being like a human being, instead of playing R2-D2 with the system. The main purpose of cfengine is to allow you to create a single, central system configuration which will define how every host on your network should be configured in an intuitive way. An interpreter runs on every host on your network and parses the master file (or file-set); the configuration of each host is checked against this file and then, if you request it, any deviations from the defined configuration are fixed automatically. You do not have to mention every host specifically by name in order to configure them: instead you can refer to the properties which distinguish hosts from one another. Cfengine uses a flexible system of "classes" which helps you to single out a specific group of hosts with a single statement. Its a decent system configuration tool, and doesn't require any changes to the way UNIX-like systems work. -- dtm To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 0: 9:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 0E1AD37B719 for ; Thu, 29 Mar 2001 00:09:53 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id VAA06966 for ; Wed, 28 Mar 2001 21:22:10 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103290522.VAA06966@beastie.mckusick.com> To: arch@freebsd.org Subject: Background Fsck Date: Wed, 28 Mar 2001 21:22:10 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I now have the capability in fsck_ffs to run a background fsck on an active filesystem. The question is how to deploy this capability in the system. I have discussed how best to deploy it with several folks and have put together the following proposal. I am now soliciting input from the larger `arch' audience. The default will be automatic background fsck of filesystems running with soft updates. It will be possible to override this default by adding the letter `F' to the passno in /etc/fstab (e.g., saying `2F' means do it in the second pass before going multiuser). The implementation will be to add a new option, -B, to the front end, fsck. The startup scripts will invoke fsck twice, once at the current location without -B to do all the foreground checks, and once towards the end of the startup script with -B to start the background checks. When called the first time without -B to do the foreground checks, the front end will call each back end (e.g., fsck_ffs, fsck_ifs, etc) without -B. Filesystems that are able and prefer to do background checks will just print out a current summary and note that the check is being deferred (the output will look much like the output from a clean filesystem today). The fsck front end will understand the new `2F' style passno and invoke the back end with -f to force a foreground check. Also, anything listed as passno 1 will always be done in foreground (which principally means that the root will always be done in foreground). It will still be necessary to have the `2F' designation rather than just using a passno of 1 to indicate foreground checking because everything marked pass 1 is done serially. The front end running in foreground will start up as many passno 2's in parallel as it can (using its current algorithm). The second invocation of fsck with -B will go through all the filesystems marked with passno 2 (but not the 2F's) invoking their backend with the -B option. Those that are already clean or do not do background checks will simply return having done nothing. Unlike the foreground varient the filesystems will be done one at a time. It would also be possible to have the children nice'd down to a low priority to reduce their CPU demands (though note that it would not affect their I/O demands). The output would be piped to the logger program rather than being dribbled out on the console. Comments please. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 7:16:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from garm.bart.nl (garm.bart.nl [194.158.170.13]) by hub.freebsd.org (Postfix) with ESMTP id 7135A37B719 for ; Thu, 29 Mar 2001 07:15:59 -0800 (PST) (envelope-from asmodai@wxs.nl) Received: from daemon.chronias.ninth-circle.org (root@cable.ninth-circle.org [195.38.232.6]) by garm.bart.nl (8.10.1/8.10.1) with ESMTP id f2TFFsU54007; Thu, 29 Mar 2001 17:15:54 +0200 (CEST) Received: (from asmodai@localhost) by daemon.chronias.ninth-circle.org (8.11.3/8.11.3) id f2TFFqn07418; Thu, 29 Mar 2001 17:15:53 +0200 (CEST) (envelope-from asmodai) Date: Thu, 29 Mar 2001 17:15:52 +0200 From: Jeroen Ruigrok/Asmodai To: Tan Wei Chong Cc: arch@freebsd.org Subject: Re: kernel programming newbie Message-ID: <20010329171552.J90260@daemon.ninth-circle.org> References: <20010326043600.30077.qmail@bsdmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20010326043600.30077.qmail@bsdmail.com>; from weichong78@bsdmail.com on Mon, Mar 26, 2001 at 12:35:59PM +0800 Organisation: Ninth-Circle Enterprises Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG -On [20010326 07:00], Tan Wei Chong (weichong78@bsdmail.com) wrote: >Can anyone kindly point to me some direction (sites, doc, books etc.) >that can get me started in either kernel or driver programming in >FreeBSD, or more specifically, how do i write a driver in FreeBSD? http://people.freebsd.org/~asmodai/newbus-draft.txt might help. -- Jeroen Ruigrok van der Werven/Asmodai .oUo. asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its best D78D D0AD 244D 1D12 C9CA 7152 035C 1138 546A B867 Conscience is God's presence in man... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 9:16:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.gmx.net (pop.gmx.net [194.221.183.20]) by hub.freebsd.org (Postfix) with SMTP id C56A037B71B for ; Thu, 29 Mar 2001 09:16:51 -0800 (PST) (envelope-from tmoestl@gmx.net) Received: (qmail 16576 invoked by uid 0); 29 Mar 2001 17:16:50 -0000 Received: from p3e9e0443.dip.t-dialin.net (HELO forge.local) (62.158.4.67) by mail.gmx.net (mp002-rz3) with SMTP; 29 Mar 2001 17:16:50 -0000 Received: from tmm by forge.local with local (Exim 3.20 #1) id 14ig2T-00013G-00; Thu, 29 Mar 2001 19:16:49 +0200 Date: Thu, 29 Mar 2001 19:16:49 +0200 From: Thomas Moestl To: freebsd-arch@freebsd.org Cc: Robert Watson Subject: Moving libposix1e into libc Message-ID: <20010329191649.A3998@crow.dom2ip.de> Mail-Followup-To: Thomas Moestl , freebsd-arch@freebsd.org, Robert Watson Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, I'd like to move libposix1e, the library that implements the userland part of the POSIX.1e additions (like ACLs, capabilities, ...) into libc. The reasons for this are: - an increasing number of programs in the base system are going to require it, e.g. all libutil consumers, because setusercontext(3) will start to rely on POSIX.1e context management calls, and base system utilities that will need to be changed to correctly handle ACLs - it implements functions defined in sys/ headers (sys/acl.h, sys/capability.h and some more to come) - other implementations have integrated this into their libc, so it would increase compatability The overhead of this addition should be relatively small (about 50k for a static libc, and ~40k for a shared one). I would like to do the move by creating a posix1e subdirectory under src/lib/libc/ and having libposix1e repocopied there. That would mean that some syscall wrappers would also reside there, but I think that this is justifyable given that the POSIX.1e additions are a work in progress and also functionally belong together. There would also be a need to add libc_r handlers for four functions that deal with file descriptors before the build would be activated. Some parts of the library are not thread safe, but not in a way that would affect other functions (they define types without locks, so variables of those types cannot be shared between multiple threads). Any comments or objections to this? Thanks, - thomas To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 9:47:14 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id 9D03837B71B for ; Thu, 29 Mar 2001 09:47:10 -0800 (PST) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id GN5Y8B1T; Thu, 29 Mar 2001 12:47:08 -0500 Reply-To: Randell Jesup To: Terry Lambert Cc: jar@integratus.com (Jack Rusher), freebsd-arch@FreeBSD.ORG (freebsd-arch@FreeBSD.ORG) Subject: Re: configuration files References: <200103282151.OAA12651@usr02.primenet.com> From: Randell Jesup Date: 29 Mar 2001 12:49:15 -0500 In-Reply-To: Terry Lambert's message of "Wed, 28 Mar 2001 21:51:30 +0000 (GMT)" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry Lambert writes: >> I have always wondered whether it would be worth my while to create a >> VMS style versioning file system (everyone who remembers DEL *.*;* raise >> your hand) to hold configuration files. It seems like a natural >> mechanism for this (although they didn't use it for that in VMS). > >VMS versioning really depends on doing globbing in the kernel >instead of in user space. > >Despite the fact that globbing in the kernel makes it so you >can pass back only the data matching your search criterian, >instead of all the data available, after which you iterate >through it again in user space, UNIX people really, really >tend to hate kernel globbing as a concept. Full globbing in the _kernel_ isn't necessary (though I like it, for a bunch of reasons I'll throw in as a postscript). It does require that the default version of readdir() return only the most recent, and a load of other changes (of course). Much of this can be in user-space in libraries, though. The suggestion above seemed to be a slight variation on the normal VMS idea, in that it was only mentioned for use with configuration files. I.e. no (or few) userland changes, but weird FS semantics (open(), write(), close() on an existing file causes the old file(s) to be renamed - foo.cfg becomes foo.cfg.1, .1 becomes .2, etc). This wouldn't be used to replace UFS/etc, just for config files. (We did something a little similar on the Amiga at one point; we implemented file-change-notification on the RAMdisk only, and at boot copied all config files to the ramdisk (actually ENV:) from permanent storage (ENVARC:). You could reassign ENV: to be on a disk which didn't support notification; you just lost the benefits of live notification.) >Without kernel globbing, you have to split up the namespace, and >hack both the FS (namespace folding) and the user land tools >(display versioning data from a seperate metadata request as a >result of being given an option to do so). The result is that >everything except special tools only see the most recent version >of a file. It ends up being really, really hackish, to the point >of not being worth the effort. That I'll agree with, for all that it's occasionally handy. p.s. Kernel globbing: 1) it reduces A LOT the traffic required across a network (or across the kernel/user interface). 2) it's universal - all programs get the same globbing and it doesn't depend on the shell 3) it makes "mv *.c *.c.old" possible 4) it makes syntaxes to commands that need regexps or regexp characters in arguments simpler, especially for non-hacker users. (I've written shells, and I still end up playing the "Ok, lets add quotes and \'s at semi-random until I get the result I want" game way too often.) 5) it works well as part of a unified command-line-parsing package that all program should use (kinda like getopt and the gnu parsing stuff), which is something that's still very iffy in Unix mostly due to history. Yes, I know there are downsides too, and that it'll never happen in Unix because it's assumed to be the work of the shell by all commands, but I can dream, can't I? (One of the few things I LOVED about Stratus VOS (made by a bunch of ex-Multics hackers) was the command-line parsing utilities, and the display-form capability (plus that every OS and user command used it and that all options were spelled out in english. Of course, this made aliases important - and it supported only simple ones, like "alias cd change_current_directory". Unlike Unix, which tried hard to minimize typing, VOS assumed you liked to type. ;-) -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 16:54: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from thunderer.cnchost.com (thunderer.concentric.net [207.155.252.72]) by hub.freebsd.org (Postfix) with ESMTP id 302F537B71D for ; Thu, 29 Mar 2001 16:54:03 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com ([64.1.14.226]) by thunderer.cnchost.com id TAA27553; Thu, 29 Mar 2001 19:53:57 -0500 (EST) [ConcentricHost SMTP Relay 1.10] Message-ID: <200103300053.TAA27553@thunderer.cnchost.com> To: Kirk McKusick Cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: Your message of "Wed, 28 Mar 2001 21:22:10 PST." <200103290522.VAA06966@beastie.mckusick.com> Date: Thu, 29 Mar 2001 16:53:55 -0800 From: Bakul Shah Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dumb question time. Why would I want to run a background fsck on an active filesystem? One wouldn't mount an unsafe filesystem in the first place. Perhaps you are talking about background garbage collection on an active fs -- blocks and inodes not reachable from the root set of objects (root inode + freelist + superblock?) recovered lazily. If this is really what you have, wouldn't it make sense to call it something else (e.g. fsgc)? On a somewhat related note, I have always wondered if the current fsck algorithm can be significantly improved or if it is about as efficient as it can be (barring any peephole code improvements). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 17:24:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 5596537B71B for ; Thu, 29 Mar 2001 17:24:48 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.3/8.11.2) with ESMTP id f2U1OMg84787; Thu, 29 Mar 2001 17:24:23 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: bakul@bitblocks.com Cc: arch@FreeBSD.ORG Subject: Re: Background Fsck In-Reply-To: <200103300053.TAA27553@thunderer.cnchost.com> References: <200103290522.VAA06966@beastie.mckusick.com> <200103300053.TAA27553@thunderer.cnchost.com> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010329172422V.jkh@osd.bsdi.com> Date: Thu, 29 Mar 2001 17:24:22 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 9 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Dumb question time. Why would I want to run a background > fsck on an active filesystem? One wouldn't mount an unsafe > filesystem in the first place. Perhaps you are talking about That's what the whole snapshot mechanism in softupdates is about. You can fsck and reconcile a filesystem's contents while it's mounted and active. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 20: 1:33 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peorth.iteration.net (peorth.iteration.net [208.190.180.178]) by hub.freebsd.org (Postfix) with ESMTP id 7AEF237B71D for ; Thu, 29 Mar 2001 20:01:30 -0800 (PST) (envelope-from keichii@peorth.iteration.net) Received: by peorth.iteration.net (Postfix, from userid 1001) id 9628B59477; Thu, 29 Mar 2001 22:01:28 -0600 (CST) Date: Thu, 29 Mar 2001 22:01:28 -0600 From: "Michael C . Wu" To: Bakul Shah Cc: Kirk McKusick , arch@freebsd.org Subject: Re: Background Fsck Message-ID: <20010329220128.B21838@peorth.iteration.net> Reply-To: "Michael C . Wu" References: <200103290522.VAA06966@beastie.mckusick.com> <200103300053.TAA27553@thunderer.cnchost.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103300053.TAA27553@thunderer.cnchost.com>; from bakul@bitblocks.com on Thu, Mar 29, 2001 at 04:53:55PM -0800 X-PGP-Fingerprint: 5025 F691 F943 8128 48A8 5025 77CE 29C5 8FA1 2E20 X-PGP-Key-ID: 0x8FA12E20 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I think Kirk would know a thing or two about FFS. ;-) On Thu, Mar 29, 2001 at 04:53:55PM -0800, Bakul Shah scribbled: | Dumb question time. Why would I want to run a background | fsck on an active filesystem? One wouldn't mount an unsafe You want your system to be up as soon as possible. Have you ever tried to fsck even just a 200gb system? | filesystem in the first place. Perhaps you are talking about | background garbage collection on an active fs -- blocks and No, he calls it background fsck because that is what it is. | inodes not reachable from the root set of objects (root inode | + freelist + superblock?) recovered lazily. If this is | really what you have, wouldn't it make sense to call it | something else (e.g. fsgc)? Please at least try to understand what this feature is and does. | On a somewhat related note, I have always wondered if the | current fsck algorithm can be significantly improved or if it | is about as efficient as it can be (barring any peephole code | improvements). This is a significant architecture addition/redesign to reduce fsck time. -- +-----------------------------------------------------------+ | keichii@iteration.net | keichii@freebsd.org | | http://iteration.net/~keichii | Yes, BSD is a conspiracy. | +-----------------------------------------------------------+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 20:12: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 8C2EC37B71E for ; Thu, 29 Mar 2001 20:12:03 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2U4Buc08081; Thu, 29 Mar 2001 20:11:56 -0800 (PST) Date: Thu, 29 Mar 2001 20:11:56 -0800 From: Alfred Perlstein To: "Michael C . Wu" Cc: Bakul Shah , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010329201156.P9431@fw.wintelcom.net> References: <200103290522.VAA06966@beastie.mckusick.com> <200103300053.TAA27553@thunderer.cnchost.com> <20010329220128.B21838@peorth.iteration.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010329220128.B21838@peorth.iteration.net>; from keichii@iteration.net on Thu, Mar 29, 2001 at 10:01:28PM -0600 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Michael C . Wu [010329 20:01] wrote: > I think Kirk would know a thing or two about FFS. ;-) > > On Thu, Mar 29, 2001 at 04:53:55PM -0800, Bakul Shah scribbled: > | Dumb question time. Why would I want to run a background > | fsck on an active filesystem? One wouldn't mount an unsafe > > You want your system to be up as soon as possible. > Have you ever tried to fsck even just a 200gb system? > > | filesystem in the first place. Perhaps you are talking about > | background garbage collection on an active fs -- blocks and > > No, he calls it background fsck because that is what it is. > > | inodes not reachable from the root set of objects (root inode > | + freelist + superblock?) recovered lazily. If this is > | really what you have, wouldn't it make sense to call it > | something else (e.g. fsgc)? > > Please at least try to understand what this feature is and does. > > | On a somewhat related note, I have always wondered if the > | current fsck algorithm can be significantly improved or if it > | is about as efficient as it can be (barring any peephole code > | improvements). > > This is a significant architecture addition/redesign to > reduce fsck time. er, actually Bakul Shah is correct in his questions, you're the one who doesn't seem to understand. :) It is basically a garbage collection that's possible because the disk is "frozen" in the snapshot. As far as speeding up fsck in general, I haven't heard anything, suggestions are welcome. :) And as far as 'fsgc', that might be a good thing to call it, basically put code into 'fsck' so that when argv[0] = "fscg" it does the snapshotting and gc sweep. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 20:20:33 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 8A08A37B71D for ; Thu, 29 Mar 2001 20:20:31 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2U4KUT08398; Thu, 29 Mar 2001 20:20:30 -0800 (PST) Date: Thu, 29 Mar 2001 20:20:30 -0800 From: Alfred Perlstein To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010329202028.S9431@fw.wintelcom.net> References: <200103290522.VAA06966@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103290522.VAA06966@beastie.mckusick.com>; from mckusick@mckusick.com on Wed, Mar 28, 2001 at 09:22:10PM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Kirk McKusick [010329 00:10] wrote: > I now have the capability in fsck_ffs to run a background fsck on an > active filesystem. The question is how to deploy this capability in > the system. I have discussed how best to deploy it with several folks > and have put together the following proposal. I am now soliciting > input from the larger `arch' audience. k, thnx :) > Also, anything listed as passno 1 will always be > done in foreground (which principally means that the root will > always be done in foreground). It will still be necessary to have > the `2F' designation rather than just using a passno of 1 to indicate > foreground checking because everything marked pass 1 is done > serially. Well, even though most common sense says not to make a giant /, people still do this, especially people coming from Linux. It would be good to be able to background check / if at all possible. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 21:15: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by hub.freebsd.org (Postfix) with ESMTP id 5D6D137B71E for ; Thu, 29 Mar 2001 21:15:03 -0800 (PST) (envelope-from brdavis@odin.ac.hmc.edu) Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.11.0/8.11.0) id f2U5Erk24486; Thu, 29 Mar 2001 21:14:53 -0800 Date: Thu, 29 Mar 2001 21:14:53 -0800 From: Brooks Davis To: Alfred Perlstein Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010329211453.B21650@Odin.AC.HMC.Edu> References: <200103290522.VAA06966@beastie.mckusick.com> <20010329202028.S9431@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="JP+T4n/bALQSJXh8" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010329202028.S9431@fw.wintelcom.net>; from bright@wintelcom.net on Thu, Mar 29, 2001 at 08:20:30PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --JP+T4n/bALQSJXh8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 29, 2001 at 08:20:30PM -0800, Alfred Perlstein wrote: > Well, even though most common sense says not to make a giant /, > people still do this, especially people coming from Linux. >=20 > It would be good to be able to background check / if at all possible. Wouldn't just moving it to pass 2 take care of that? -- Brooks --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --JP+T4n/bALQSJXh8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6xBZMXY6L6fI4GtQRAtOZAJ0U5WRfRmJLrneday29zppJCKN8UwCguTD/ Sg7ecfoAzotgElmQIB08Hyc= =GwN/ -----END PGP SIGNATURE----- --JP+T4n/bALQSJXh8-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 21:30:33 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3943A37B71F for ; Thu, 29 Mar 2001 21:30:31 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2U5UTl09991; Thu, 29 Mar 2001 21:30:29 -0800 (PST) Date: Thu, 29 Mar 2001 21:30:29 -0800 From: Alfred Perlstein To: Brooks Davis Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010329213029.T9431@fw.wintelcom.net> References: <200103290522.VAA06966@beastie.mckusick.com> <20010329202028.S9431@fw.wintelcom.net> <20010329211453.B21650@Odin.AC.HMC.Edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010329211453.B21650@Odin.AC.HMC.Edu>; from brooks@one-eyed-alien.net on Thu, Mar 29, 2001 at 09:14:53PM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Brooks Davis [010329 21:14] wrote: > On Thu, Mar 29, 2001 at 08:20:30PM -0800, Alfred Perlstein wrote: > > Well, even though most common sense says not to make a giant /, > > people still do this, especially people coming from Linux. > > > > It would be good to be able to background check / if at all possible. > > Wouldn't just moving it to pass 2 take care of that? I don't know. I'm saying that if there's something in here that makes doing on / not possible because of Kirk's preference (even though it's technically possible) I hope he reconsiders. If it's not technically possible then we'll live with it. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 21:47:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 2C63237B718 for ; Thu, 29 Mar 2001 21:47:35 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id VAA09139; Thu, 29 Mar 2001 21:47:32 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103300547.VAA09139@beastie.mckusick.com> To: Alfred Perlstein Subject: Re: Background Fsck Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 29 Mar 2001 21:30:29 PST." <20010329213029.T9431@fw.wintelcom.net> Date: Thu, 29 Mar 2001 21:47:32 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Date: Thu, 29 Mar 2001 21:30:29 -0800 From: Alfred Perlstein To: Brooks Davis Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: Background Fsck * Brooks Davis [010329 21:14] wrote: > On Thu, Mar 29, 2001 at 08:20:30PM -0800, Alfred Perlstein wrote: > > Well, even though most common sense says not to make a giant /, > > people still do this, especially people coming from Linux. > > > > It would be good to be able to background check if at all possible. > > Wouldn't just moving it to pass 2 take care of that? I don't know. I'm saying that if there's something in here that makes doing on / not possible because of Kirk's preference (even though it's technically possible) I hope he reconsiders. If it's not technically possible then we'll live with it. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. Background checking the root is possible. The only reason to exclude it was because people are not yet comfortable with entrusting their root to a background check. Since most roots are small, the foreground fsck time is minimal. It would be possible to make background chacking of the root happen by default and have the administrator add the `F' flag to the passno if they wanted to force a background check. Kirk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 21:49: 9 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 613AF37B722 for ; Thu, 29 Mar 2001 21:49:04 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.2/8.11.2) with ESMTP id f2U5mkG52392; Thu, 29 Mar 2001 21:48:46 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200103290522.VAA06966@beastie.mckusick.com> Date: Thu, 29 Mar 2001 21:48:28 -0800 (PST) From: John Baldwin To: Kirk McKusick Subject: RE: Background Fsck Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 29-Mar-01 Kirk McKusick wrote: > I now have the capability in fsck_ffs to run a background fsck on an > active filesystem. The question is how to deploy this capability in > the system. I have discussed how best to deploy it with several folks > and have put together the following proposal. I am now soliciting > input from the larger `arch' audience. > > The default will be automatic background fsck of filesystems > running with soft updates. It will be possible to override this > default by adding the letter `F' to the passno in /etc/fstab > (e.g., saying `2F' means do it in the second pass before going > multiuser). Uppercase or either case? [ snip ] > Comments please. > > Kirk McKusick Sounds ok to me. :) -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 21:58:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 926A437B718 for ; Thu, 29 Mar 2001 21:58:13 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id VAA09201; Thu, 29 Mar 2001 21:58:11 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103300558.VAA09201@beastie.mckusick.com> To: Bakul Shah Subject: Re: Background Fsck Cc: arch@freebsd.org In-Reply-To: Your message of "Thu, 29 Mar 2001 16:53:55 PST." <200103300053.TAA27553@thunderer.cnchost.com> Date: Thu, 29 Mar 2001 21:58:11 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG To: Kirk McKusick cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: Your message of "Wed, 28 Mar 2001 21:22:10 PST." <200103290522.VAA06966@beastie.mckusick.com> Date: Thu, 29 Mar 2001 16:53:55 -0800 From: Bakul Shah Dumb question time. Why would I want to run a background fsck on an active filesystem? One wouldn't mount an unsafe filesystem in the first place. Background fsck can only be run on filesystems that are using soft updates. It is safe to mount a soft updates filesystem after a crash because the only inconsistencies will be lost block and inodes. So, in a sense you are mouting a dirty filesystem, albeit one on which it is safe to operate. Perhaps you are talking about background garbage collection on an active fs -- blocks and inodes not reachable from the root set of objects (root inode + freelist + superblock?) recovered lazily. If this is really what you have, wouldn't it make sense to call it something else (e.g. fsgc)? I agree that background fsck is doing garbage collection, but then one can argue that traditional fsck is doing garbage collection too. It might have been reasonable to call the original program fsgc, but it was called fsck. The background fsck is doing exactly the same set of fixes that the traditional fsck does. The only difference is that it has to do system calls to update the bitmaps rather than writing them directly. So, changing the name seems unnecessarily confusing to me. On a somewhat related note, I have always wondered if the current fsck algorithm can be significantly improved or if it is about as efficient as it can be (barring any peephole code improvements). Many improvements have been made to fsck over the years. Through there are undoubtedly more that could be made, there are no big easy improvements left. Kirk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 22: 2:14 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id C134437B71B; Thu, 29 Mar 2001 22:02:11 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id WAA09248; Thu, 29 Mar 2001 22:02:10 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103300602.WAA09248@beastie.mckusick.com> To: John Baldwin Subject: Re: Background Fsck Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 29 Mar 2001 21:48:28 PST." Date: Thu, 29 Mar 2001 22:02:10 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Date: Thu, 29 Mar 2001 21:48:28 -0800 (PST) From: John Baldwin To: Kirk McKusick Subject: RE: Background Fsck Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG On 29-Mar-01 Kirk McKusick wrote: > I now have the capability in fsck_ffs to run a background fsck on an > active filesystem. The question is how to deploy this capability in > the system. I have discussed how best to deploy it with several folks > and have put together the following proposal. I am now soliciting > input from the larger `arch' audience. > > The default will be automatic background fsck of filesystems > running with soft updates. It will be possible to override this > default by adding the letter `F' to the passno in /etc/fstab > (e.g., saying `2F' means do it in the second pass before going > multiuser). Uppercase or either case? My plan was to make it upper case only. Do you feel that it should be one or the other or case insensitive? Unix does not generally do case insensitive. [ snip ] > Comments please. > > Kirk McKusick Sounds ok to me. :) -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 22:27:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (awfulhak.demon.co.uk [194.222.196.252]) by hub.freebsd.org (Postfix) with ESMTP id A3E3C37B718 for ; Thu, 29 Mar 2001 22:27:31 -0800 (PST) (envelope-from brian@Awfulhak.org) Received: from hak.lan.Awfulhak.org (root@hak.lan.Awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.3/8.11.3) with ESMTP id f2U6RI800484; Fri, 30 Mar 2001 07:27:18 +0100 (BST) (envelope-from brian@lan.Awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.3/8.11.3) with ESMTP id f2U6Vq042781; Fri, 30 Mar 2001 07:31:52 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200103300631.f2U6Vq042781@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Kirk McKusick Cc: Alfred Perlstein , arch@FreeBSD.ORG, brian@Awfulhak.org Subject: Re: Background Fsck In-Reply-To: Message from Kirk McKusick of "Thu, 29 Mar 2001 21:47:32 -0800." <200103300547.VAA09139@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 30 Mar 2001 07:31:52 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Date: Thu, 29 Mar 2001 21:30:29 -0800 > From: Alfred Perlstein > To: Brooks Davis > Cc: Kirk McKusick , arch@FreeBSD.ORG > Subject: Re: Background Fsck > > * Brooks Davis [010329 21:14] wrote: > > On Thu, Mar 29, 2001 at 08:20:30PM -0800, Alfred Perlstein wrote: > > > Well, even though most common sense says not to make a giant /, > > > people still do this, especially people coming from Linux. > > > > > > It would be good to be able to background check if at all possible. > > > > Wouldn't just moving it to pass 2 take care of that? > > I don't know. > > I'm saying that if there's something in here that makes doing on > / not possible because of Kirk's preference (even though it's > technically possible) I hope he reconsiders. > > If it's not technically possible then we'll live with it. > > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > Instead of asking why a piece of software is using "1970s technology," > start asking why software is ignoring 30 years of accumulated wisdom. > > Background checking the root is possible. The only reason to exclude > it was because people are not yet comfortable with entrusting their > root to a background check. Since most roots are small, the foreground > fsck time is minimal. It would be possible to make background chacking > of the root happen by default and have the administrator add the `F' > flag to the passno if they wanted to force a background check. ^^^^ fore > Kirk -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 29 22:34:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id EBF0E37B71B for ; Thu, 29 Mar 2001 22:34:13 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2U6bKJ14180; Thu, 29 Mar 2001 22:37:26 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC4299F.EAAE27C6@integratus.com> Date: Thu, 29 Mar 2001 22:37:19 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103300547.VAA09139@beastie.mckusick.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk McKusick wrote: > > fsck time is minimal. It would be possible to make background chacking > of the root happen by default and have the administrator add the `F' > flag to the passno if they wanted to force a background check. Which I will do on all of my personal machines immediately after this is committed. This is a wonderful piece of work. I am very excited about adding fast reboots to the lovely consistency guarantees of FFS/soft updates. This eliminates my primary need for a journaled file system under FreeBSD. Nice one. Thanks, -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 4:52:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id F0E1A37B71B for ; Fri, 30 Mar 2001 04:52:45 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id FAA15865; Fri, 30 Mar 2001 05:49:20 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp03.primenet.com, id smtpdAAAPZai8E; Fri Mar 30 05:49:12 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id FAA06540; Fri, 30 Mar 2001 05:52:34 -0700 (MST) From: Terry Lambert Message-Id: <200103301252.FAA06540@usr05.primenet.com> Subject: Re: Background Fsck To: mckusick@mckusick.com (Kirk McKusick) Date: Fri, 30 Mar 2001 12:52:29 +0000 (GMT) Cc: arch@FreeBSD.ORG In-Reply-To: <200103290522.VAA06966@beastie.mckusick.com> from "Kirk McKusick" at Mar 28, 2001 09:22:10 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I have a question avout the safety of this approach: You don't seem to be able to distinguish between: 1) Hardware crash without data coruption - e.g. power failure 2) Hardware crash with data corruption - e.g. disk/controller/memory failure 3) Software crash without data corruption - e.q. resource availability failure, or panic as a result of coding error 4) Software crash with data corruption - e.g. a panic resulting from kernel data becoming corrupt, with an unknown interval preceeding the crash in which some of these structures might have had FS data in them, or a such crash in the FS code path itself, where the data corruption was a primary effect instead of a side effect It seems to me tha background checking is only safe in cases 1 and 3, and (the current California power grid reliability not withstanding), that these cases are not provably the statistically most common cases. The reason Whistle did not do this work earlier was that we were unable to address this concern adequately without non-volatile RAM to store the failure reason and the disk write cache status. Since panic reasons are mathematically indistinguishable in the limit, were were also unable to address differentiating 3 and 4, without placing the FS and I/O subsystem into a seperate protection domain. Even doing this, we would only gain some statistical protection against #4, which means the only value which we could add was to case #1, were we to invest in the additional hardware. In other words, it was not speed of fsck which drove Whistle to soft updates. My question is this: how were you able to address these issues in your implementation? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 4:59: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp1.libero.it (smtp1.libero.it [193.70.192.51]) by hub.freebsd.org (Postfix) with ESMTP id 28EA437B71F for ; Fri, 30 Mar 2001 04:59:04 -0800 (PST) (envelope-from litos2001@libero.it) Received: from libero.it (193.70.192.61) by smtp1.libero.it (5.5.025) id 3AB48C81002B4329 for arch@FreeBSD.ORG; Fri, 30 Mar 2001 14:59:03 +0200 Date: Fri, 30 Mar 2001 15:00:51 +0200 Message-Id: Subject: Startup scripts a la NetBSD MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable From: "litos2001@libero.it" To: arch@FreeBSD.ORG X-XaM3-API-Version: 1.1.9.1.36 X-SenderIP: 192.106.117.97 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi all, I'd like to know if there are any plans to use the new startup system used in NetBSD 1.5 (/etc/rc.d/*) ... I searched in the archives but I didn't find anything, if you discussed before, sorry! :) Thanks! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 5:10:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id A8AE837B71B for ; Fri, 30 Mar 2001 05:10:35 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id GAA22925; Fri, 30 Mar 2001 06:08:56 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp01.primenet.com, id smtpdAAARca4SS; Fri Mar 30 06:08:48 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id GAA06817; Fri, 30 Mar 2001 06:10:15 -0700 (MST) From: Terry Lambert Message-Id: <200103301310.GAA06817@usr05.primenet.com> Subject: Re: configuration files To: rjesup@wgate.com Date: Fri, 30 Mar 2001 13:10:10 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jar@integratus.com (Jack Rusher), freebsd-arch@FreeBSD.ORG (freebsd-arch@FreeBSD.ORG) In-Reply-To: from "Randell Jesup" at Mar 29, 2001 12:49:15 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Full globbing in the _kernel_ isn't necessary (though I like it, > for a bunch of reasons I'll throw in as a postscript). > > It does require that the default version of readdir() return > only the most recent, and a load of other changes (of course). Much of > this can be in user-space in libraries, though. This can't be done on NFS mounted volumes. The "load of other changes" would include wrapping all path-using system calls, with incidental (complex) changes required to the threading libraries, since they would require orivate versions of these calls in both the libc_r and the third-paty-wraps-open-syscall cases. It is much better to simply turn files into directories, and then build a stacking layer that represents a view on them as "data file with highest version number is returned as if it were the file itself, on references to the directories path component name". Access to the directory contents (versions, in this case, but ACLs, extended attributes, etc.) would require a namespace escape be used. This is the "namespace folding" approach I alluded to earlier; it has the characteristic that it provides 100% application backward compatability to applications not using the escape, and file rewrites can result in versioning (but two stage commits, which is where you wanted it in the first place, all have to be taught about the change). > The suggestion above seemed to be a slight variation on the normal > VMS idea, in that it was only mentioned for use with configuration files. > I.e. no (or few) userland changes, but weird FS semantics (open(), write(), > close() on an existing file causes the old file(s) to be renamed - foo.cfg > becomes foo.cfg.1, .1 becomes .2, etc). This wouldn't be used to replace > UFS/etc, just for config files. (We did something a little similar on the > Amiga at one point; we implemented file-change-notification on the RAMdisk > only, and at boot copied all config files to the ramdisk (actually ENV:) > from permanent storage (ENVARC:). You could reassign ENV: to be on a disk > which didn't support notification; you just lost the benefits of > live notification.) Metadata incursion is preferrable to the file name component incursion (VMS actually used a semicolon seperator, not a period, and restricted the user to not using semicolons, and only using one period in file names, both of which are unacceptable for I18N/L10N reasons). > p.s. Kernel globbing: > 1) it reduces A LOT the traffic required across a network (or > across the kernel/user interface). > 2) it's universal - all programs get the same globbing and it > doesn't depend on the shell > 3) it makes "mv *.c *.c.old" possible I miss this the most. You can achieve the same effect by making "mv" a shell built-in, however, but that doesn't fix rename(2). > 4) it makes syntaxes to commands that need regexps or regexp > characters in arguments simpler, especially for non-hacker > users. (I've written shells, and I still end up playing the > "Ok, lets add quotes and \'s at semi-random until I get the > result I want" game way too often.) > 5) it works well as part of a unified command-line-parsing > package that all program should use (kinda like getopt > and the gnu parsing stuff), which is something that's still > very iffy in Unix mostly due to history. CLDs had their uses; among other things, it was easy to generate "almost complete" documentation programmatically. > Yes, I know there are downsides too, and that it'll never happen in Unix > because it's assumed to be the work of the shell by all commands, but I > can dream, can't I? ... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 5:21:22 2001 Delivered-To: freebsd-arch@freebsd.org Received: from blackhelicopters.org (geburah.blackhelicopters.org [209.69.178.18]) by hub.freebsd.org (Postfix) with ESMTP id 5629537B71C for ; Fri, 30 Mar 2001 05:21:16 -0800 (PST) (envelope-from mwlucas@blackhelicopters.org) Received: (from mwlucas@localhost) by blackhelicopters.org (8.9.3/8.9.3) id IAA75102; Fri, 30 Mar 2001 08:21:14 -0500 (EST) (envelope-from mwlucas) Date: Fri, 30 Mar 2001 08:21:14 -0500 From: Michael Lucas To: "litos2001@libero.it" Cc: arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD Message-ID: <20010330082114.A75059@blackhelicopters.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from litos2001@libero.it on Fri, Mar 30, 2001 at 03:00:51PM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It's been discussed before. Every time it's been discussed, someone volunteers to do the work. The volunteers vanish into the aether, and are not heard from again. I can only assume that they're all secretly coordinating their work to ensure that it is bug-free and commit-ready, so that when they reappear we are all immediately overwhelmed by their technical excellence. ==ml On Fri, Mar 30, 2001 at 03:00:51PM +0200, litos2001@libero.it wrote: > Hi all, > > I'd like to know if there are any plans to use the new startup system > used in NetBSD 1.5 (/etc/rc.d/*) ... I searched in the archives but I > didn't find anything, if you discussed before, sorry! :) > > Thanks! > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- Michael Lucas mwlucas@blackhelicopters.org http://www.blackhelicopters.org/~mwlucas/ Big Scary Daemons: http://www.oreillynet.com/pub/q/Big_Scary_Daemons To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 5:36: 3 2001 Delivered-To: freebsd-arch@freebsd.org Received: from rapier.smartspace.co.za (rapier.smartspace.co.za [66.8.25.34]) by hub.freebsd.org (Postfix) with SMTP id 0211C37B71B for ; Fri, 30 Mar 2001 05:35:59 -0800 (PST) (envelope-from nbm@rapier.smartspace.co.za) Received: (qmail 58996 invoked by uid 1001); 30 Mar 2001 13:35:01 -0000 Date: Fri, 30 Mar 2001 15:35:01 +0200 From: Neil Blakey-Milner To: Michael Lucas Cc: "litos2001@libero.it" , arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD Message-ID: <20010330153501.B57394@rapier.smartspace.co.za> References: <20010330082114.A75059@blackhelicopters.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010330082114.A75059@blackhelicopters.org>; from mwlucas@blackhelicopters.org on Fri, Mar 30, 2001 at 08:21:14AM -0500 Organization: Building Intelligence X-Operating-System: FreeBSD 4.2-RELEASE i386 X-URL: http://rucus.ru.ac.za/~nbm/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri 2001-03-30 (08:21), Michael Lucas wrote: > It's been discussed before. > > Every time it's been discussed, someone volunteers to do the work. > The volunteers vanish into the aether, and are not heard from again. > > I can only assume that they're all secretly coordinating their work to > ensure that it is bug-free and commit-ready, so that when they > reappear we are all immediately overwhelmed by their technical > excellence. I haven't ever volunteered specifically, but some people know that I've worked on it. Unfortunately it's tough going lately, since I no longer have a "development" machine (and seem to have issues getting liquidated companies to pay me *grin*). I've got it to work previously, it's pretty arbitrary to do. It shouldn't take more than a few hours to get the basics going. The specifics get a bit more difficult, and it's quite a bit of manual work to get it to actually boot somewhat cleanly. I'm starting a new contract in two weeks, where I may have the resources to work on this (after some other FreeBSD-specific things the company is interested in - Hi Arri, Barry, Leon, &c.). Neil -- Neil Blakey-Milner nbm@mithrandr.moria.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 6: 5:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 94C0237B71B for ; Fri, 30 Mar 2001 06:05:23 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f2UE5Ah40563; Fri, 30 Mar 2001 09:05:10 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 30 Mar 2001 09:05:10 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Kirk McKusick Cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: <200103300558.VAA09201@beastie.mckusick.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk, I haven't had a chance to look at the tunefs source lately, but quick question: does tunefs block the setting of the soft updates flag on a dirty file system? It seems to me that, if it doesn't, a possible nasty sequence of events is: system does unclean shutdown without soft updates, administrator (possibly not realizing this) boots to single-user mode, and sets soft updates, then attempts to enter multi-user mode, where fsck says "ah, soft updates, doesn't matter if it's unclean, let's background fsck". Shortly thereafter, an inconsistency is discovered and the system panics. As such, tunefs should only allow the setting of soft updates on a file system marked clean. It may already do this, but figured I should ask. Thanks! Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 6:58:29 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailout06.sul.t-online.com (mailout06.sul.t-online.com [194.25.134.19]) by hub.freebsd.org (Postfix) with ESMTP id 490C937B71E for ; Fri, 30 Mar 2001 06:58:26 -0800 (PST) (envelope-from alex@cichlids.cichlids.com) Received: from fwd05.sul.t-online.com by mailout06.sul.t-online.com with smtp id 14j0Ly-0002Vi-09; Fri, 30 Mar 2001 16:58:18 +0200 Received: from neutron.cichlids.com (520050424122-0001@[62.158.38.143]) by fmrl05.sul.t-online.com with esmtp id 14j0Li-1HV3fEC; Fri, 30 Mar 2001 16:58:02 +0200 Received: from cichlids.cichlids.com (cichlids.cichlids.com [192.168.0.10]) by neutron.cichlids.com (Postfix) with ESMTP id 008E7AB44; Fri, 30 Mar 2001 17:00:17 +0200 (CEST) Received: by cichlids.cichlids.com (Postfix, from userid 1001) id F157814BBE; Fri, 30 Mar 2001 15:58:08 +0200 (CEST) Date: Fri, 30 Mar 2001 15:58:08 +0200 From: Alexander Langer To: Jeroen Ruigrok/Asmodai Cc: Tan Wei Chong , arch@freebsd.org Subject: Re: kernel programming newbie Message-ID: <20010330155808.A8920@cichlids.cichlids.com> References: <20010326043600.30077.qmail@bsdmail.com> <20010329171552.J90260@daemon.ninth-circle.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010329171552.J90260@daemon.ninth-circle.org>; from asmodai@wxs.nl on Thu, Mar 29, 2001 at 05:15:52PM +0200 X-PGP-Fingerprint: 44 28 CA 4C 46 5B D3 A8 A8 E3 BA F3 4E 60 7D 7F X-PGP-at: finger alex@big.endian.de X-Verwirrung: Dieser Header dient der allgemeinen Verwirrung. X-Sender: 520050424122-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Thus spake Jeroen Ruigrok/Asmodai (asmodai@wxs.nl): > http://people.freebsd.org/~asmodai/newbus-draft.txt might help. Yes :-) Also, try http://www.freebsd.org/tutorials/ Alex -- cat: /home/alex/.sig: No such file or directory To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 8:41:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from lindy.rusher.com (adsl-64-164-192-197.dsl.snfc21.pacbell.net [64.164.192.197]) by hub.freebsd.org (Postfix) with ESMTP id 20F9337B71C for ; Fri, 30 Mar 2001 08:41:57 -0800 (PST) (envelope-from jar@integratus.com) Received: from integratus.com (localhost [127.0.0.1]) by lindy.rusher.com (8.11.3/8.11.1) with ESMTP id f2UGivV00484; Fri, 30 Mar 2001 08:44:57 -0800 (PST) (envelope-from jar@integratus.com) Message-ID: <3AC4B808.9EB5806B@integratus.com> Date: Fri, 30 Mar 2001 08:44:56 -0800 From: Jack Rusher Organization: http://www.integratus.com X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Neil Blakey-Milner Cc: Michael Lucas , "litos2001@libero.it" , arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD References: <20010330082114.A75059@blackhelicopters.org> <20010330153501.B57394@rapier.smartspace.co.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Neil Blakey-Milner wrote: > > pretty arbitrary to do. It shouldn't take more than a few hours to get > the basics going. The specifics get a bit more difficult, and it's > quite a bit of manual work to get it to actually boot somewhat cleanly. It seems like a nit picky job, but not an overly difficult one. Do we know that people want this? Someone suggested a more SYSV like startup system as a port, which sounds much harder to me. -- Jack Rusher, Senior Engineer | mailto:jar@integratus.com Integratus, Inc. | http://www.integratus.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 9:23:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id A245F37B719 for ; Fri, 30 Mar 2001 09:23:34 -0800 (PST) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H70XLGG1; Fri, 30 Mar 2001 12:23:33 -0500 Reply-To: Randell Jesup To: Kirk McKusick Cc: Bakul Shah , arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103300558.VAA09201@beastie.mckusick.com> From: Randell Jesup Date: 30 Mar 2001 12:25:50 -0500 In-Reply-To: Kirk McKusick's message of "Thu, 29 Mar 2001 21:58:11 -0800" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk McKusick writes: > On a somewhat related note, I have always wondered if the > current fsck algorithm can be significantly improved or if it > is about as efficient as it can be (barring any peephole code > improvements). > >Many improvements have been made to fsck over the years. Through >there are undoubtedly more that could be made, there are no big >easy improvements left. Last time I checked, fsck basically calls exit() on a number of types of errors (inode out of range was one of them I think - you had to manually cleari for each one and restart fsck). I munged fsck to try to recover a coworker's files when something went foobar in her 2.2.8 machine and wrote garbage across sectors all over her disk. Does this fix those sorts of "I can't deal with it and won't try no matter how hard you force me" cases? From the current source - I know that many of these are internal errors that shouldn't happen, and are correct to exit out on. However, some (like inoinfo(), ginode(), maybe getnextinode()) should NOT just cause a blanket exit. utilities.c: (This is the one that caused me the big trouble, I think) ------- /* * Look up state information for an inode. */ struct inostat * inoinfo(inum) ino_t inum; { static struct inostat unallocated = { USTATE, 0, 0 }; struct inostatlist *ilp; int iloff; if (inum > maxino) errx(EEXIT, "inoinfo: inumber %d out of range", inum); inode.c: (this may very well have been it too) ------- /* * General purpose interface for reading inodes. */ struct dinode * ginode(inumber) ino_t inumber; { ufs_daddr_t iblk; if (inumber < ROOTINO || inumber > maxino) errx(EEXIT, "bad inode number %d to ginode", inumber); ... struct dinode * getnextinode(inumber) ino_t inumber; { long size; ufs_daddr_t dblk; static struct dinode *dp; if (inumber != nextino++ || inumber > maxino) errx(EEXIT, "bad inode number %d to nextinode", inumber); ... void setinodebuf(inum) ino_t inum; { if (inum % sblock.fs_ipg != 0) errx(EEXIT, "bad inode number %d to setinodebuf", inum); ... getinoinfo(inumber): errx(EEXIT, "cannot find inode %d", inumber); ... default: errx(EEXIT, "BAD STATE %d TO BLKERR", inoinfo(ino)->ino_state); dir.c: ------- if (idesc->id_type != DATA) errx(EEXIT, "wrong type to dirscan %d", idesc->id_type); pass2.c: ------- default: errx(EEXIT, "BAD STATE %d FOR ROOT INODE", inoinfo(ROOTINO)->ino_state); ... default: errx(EEXIT, "BAD STATE %d FOR INODE I=%d", inoinfo(dirp->d_ino)->ino_state, dirp->d_ino); pass4.c: ------- default: errx(EEXIT, "BAD STATE %d FOR INODE I=%d", inoinfo(inumber)->ino_state, inumber); pass5.c: ------- default: inomapsize = blkmapsize = sumsize = 0; /* keep lint happy */ errx(EEXIT, "UNKNOWN ROTATIONAL TABLE FORMAT %d", fs->fs_postblformat); ... default: if (j < ROOTINO) break; errx(EEXIT, "BAD STATE %d FOR INODE I=%ld", inoinfo(j)->ino_state, j); -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 9:27:35 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id 579A437B719; Fri, 30 Mar 2001 09:27:33 -0800 (PST) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H70XLGHB; Fri, 30 Mar 2001 12:27:33 -0500 Reply-To: Randell Jesup To: Kirk McKusick Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103300602.WAA09248@beastie.mckusick.com> From: Randell Jesup Date: 30 Mar 2001 12:29:50 -0500 In-Reply-To: Kirk McKusick's message of "Thu, 29 Mar 2001 22:02:10 -0800" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk McKusick writes: > > The default will be automatic background fsck of filesystems > > running with soft updates. It will be possible to override this > > default by adding the letter `F' to the passno in /etc/fstab > > (e.g., saying `2F' means do it in the second pass before going > > multiuser). > > Uppercase or either case? > >My plan was to make it upper case only. Do you feel that it should >be one or the other or case insensitive? Unix does not generally do >case insensitive. Case insensitive, PLEASE. Nothing should be case sensitive unless there's a reason for making a distinction. It merely adds more ways people can shoot themselves in the foot and waste innumerable hours wondering why something isn't working. -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 9:53:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from valiant.cnchost.com (valiant.concentric.net [207.155.252.9]) by hub.freebsd.org (Postfix) with ESMTP id ED3E637B719 for ; Fri, 30 Mar 2001 09:53:09 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com ([64.1.6.93]) by valiant.cnchost.com id MAA03709; Fri, 30 Mar 2001 12:53:07 -0500 (EST) [ConcentricHost SMTP Relay 1.10] Message-ID: <200103301753.MAA03709@valiant.cnchost.com> To: Kirk McKusick Cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: Your message of "Thu, 29 Mar 2001 21:58:11 PST." <200103300558.VAA09201@beastie.mckusick.com> Date: Fri, 30 Mar 2001 09:53:06 -0800 From: Bakul Shah Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Dumb question time. Why would I want to run a background > fsck on an active filesystem? One wouldn't mount an unsafe > filesystem in the first place. > > Background fsck can only be run on filesystems that are using > soft updates. It is safe to mount a soft updates filesystem > after a crash because the only inconsistencies will be lost > block and inodes. So, in a sense you are mouting a dirty > filesystem, albeit one on which it is safe to operate. Okay, this makes sense (modulo the questions Terry Lambert has raised). See below. > Perhaps you are talking about background garbage collection > on an active fs -- blocks and inodes not reachable from > the root set of objects (root inode + freelist + superblock?) > recovered lazily. If this is really what you have, wouldn't > it make sense to call it something else (e.g. fsgc)? > > I agree that background fsck is doing garbage collection, but then > one can argue that traditional fsck is doing garbage collection > too. It might have been reasonable to call the original program > fsgc, but it was called fsck. The background fsck is doing exactly > the same set of fixes that the traditional fsck does. The only > difference is that it has to do system calls to update the bitmaps > rather than writing them directly. So, changing the name seems > unnecessarily confusing to me. I am used to distinguish between the `structure repair of the FS' task from any garbage collection task of the traditional fsck. In fact the structural consistency of reachable nodes, dirs + files must be a precondition before you can do GC. Seems to me, the two tasks can be separated in two programs. Then perhaps even on a non soft-update system you can run the repair part in fg and the garbage collection part in bg. Of course, a soft-update enabled fs won't need the repair part. Anyway that is what I was thinking when I suggested a name change but the name issue by itself is not important to me; sorry I brought it up. But like Terry I worry about disks write errors or blocks going bad. And not knowing how well the bg fsck (or for that matter soft-updates) copes with that gives me an uneasy feeling. Perhaps bg fsck should not do anything if it discovers structural inconsistency? I know I don't have to run it if I feel this way but the point is to understand the behavior. > On a somewhat related note, I have always wondered if the > current fsck algorithm can be significantly improved or if it > is about as efficient as it can be (barring any peephole code > improvements). > > Many improvements have been made to fsck over the years. Through > there are undoubtedly more that could be made, there are no big > easy improvements left. I was less than clear; what I was asking is if anyone has looked at whether the fsck *algorithm* is optimal. I wondered about its complexity -- is it O(n^2) or O(n log n) or something else and whether it is the best possible algorithm. Thanks for your response (and others, especially Jordan as snaphots was one critical piece I had completely missed)! -- bakul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 10:13: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id A07EC37B71A for ; Fri, 30 Mar 2001 10:13:03 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id KAA26473; Fri, 30 Mar 2001 10:10:50 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda26471; Fri Mar 30 10:10:48 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2UIAg107019; Fri, 30 Mar 2001 10:10:42 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdXE7017; Fri Mar 30 10:10:20 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2UIAGK06787; Fri, 30 Mar 2001 10:10:16 -0800 (PST) Message-Id: <200103301810.f2UIAGK06787@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdck6764; Fri Mar 30 10:09:39 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Jack Rusher Cc: Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" , arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD In-reply-to: Your message of "Fri, 30 Mar 2001 08:44:56 PST." <3AC4B808.9EB5806B@integratus.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 30 Mar 2001 10:09:39 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <3AC4B808.9EB5806B@integratus.com>, Jack Rusher writes: > Neil Blakey-Milner wrote: > > > > pretty arbitrary to do. It shouldn't take more than a few hours to get > > the basics going. The specifics get a bit more difficult, and it's > > quite a bit of manual work to get it to actually boot somewhat cleanly. > > It seems like a nit picky job, but not an overly difficult one. Do we > know that people want this? Someone suggested a more SYSV like startup > system as a port, which sounds much harder to me. I'll fess up. I suggested the SYSV-like startup. I think it would be a great bonus to be compatible with much of the rest of the UNIX world. I would think that we should eventually want to have a SYSV style init, and ultimately a kernel with SYSV-style signal handling. We should pick away at it until we reach the ultimate goal. Becoming more SYSV-like is not something most people in FreeBSD are enamoured with. Hence I don't think it would be possible to implement all the changes I would like to see. Regarding the SYSV style init. The port would copy our init into its work directory, patch it build it, rename init to init.FreeBSD, then install the SYSV init into its place. Eventually I would see the SYSV part of init become #ifdefs in the FreeBSD init or a flag to be passed to init at boot. Given that one could implement any style of init using a SYSV style of init, e.g. if you want to use rc.local or rc3.d, just code it in inittab. I think this is a more flexible and compatible (with everyone else) solution without really sacrificing any capabilities we have now. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 10:28:27 2001 Delivered-To: freebsd-arch@freebsd.org Received: from bsdconspiracy.net (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id 1630637B71A for ; Fri, 30 Mar 2001 10:28:25 -0800 (PST) (envelope-from wes@softweyr.com) Received: from zaphod.softweyr.com ([204.68.178.35] helo=softweyr.com ident=wes) by bsdconspiracy.net with esmtp (Exim 3.14 #1) id 14j2wN-00003J-00; Fri, 30 Mar 2001 10:44:03 -0700 Message-ID: <3AC4CFCA.5BEC078E@softweyr.com> Date: Fri, 30 Mar 2001 11:26:18 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Jack Rusher Cc: Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" , arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD References: <20010330082114.A75059@blackhelicopters.org> <20010330153501.B57394@rapier.smartspace.co.za> <3AC4B808.9EB5806B@integratus.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jack Rusher wrote: > > Neil Blakey-Milner wrote: > > > > pretty arbitrary to do. It shouldn't take more than a few hours to get > > the basics going. The specifics get a bit more difficult, and it's > > quite a bit of manual work to get it to actually boot somewhat cleanly. > > It seems like a nit picky job, but not an overly difficult one. Do we > know that people want this? Someone suggested a more SYSV like startup > system as a port, which sounds much harder to me. I've done a lot of work like this lately, the DoBox starts from an rc.d system. Ours is a bit complicated, since we handle dynamic network reconfiguration via the rc.d files as well. I'll willing to help where I can. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 10:32:44 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 1AE1937B71A; Fri, 30 Mar 2001 10:32:42 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id KAA10132; Fri, 30 Mar 2001 10:32:41 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103301832.KAA10132@beastie.mckusick.com> To: Robert Watson Subject: Re: Background Fsck Cc: arch@freebsd.org In-Reply-To: Your message of "Fri, 30 Mar 2001 09:05:10 EST." Date: Fri, 30 Mar 2001 10:32:41 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Date: Fri, 30 Mar 2001 09:05:10 -0500 (EST) From: Robert Watson To: Kirk McKusick cc: arch@freebsd.org Subject: Re: Background Fsck Kirk, I haven't had a chance to look at the tunefs source lately, but quick question: does tunefs block the setting of the soft updates flag on a dirty file system? It seems to me that, if it doesn't, a possible nasty sequence of events is: system does unclean shutdown without soft updates, administrator (possibly not realizing this) boots to single-user mode, and sets soft updates, then attempts to enter multi-user mode, where fsck says "ah, soft updates, doesn't matter if it's unclean, let's background fsck". Shortly thereafter, an inconsistency is discovered and the system panics. As such, tunefs should only allow the setting of soft updates on a file system marked clean. It may already do this, but figured I should ask. Thanks! Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services Your observation is absolutely correct. I have modified tunefs exactly as you suggest and will be checking in that change as part of my next set of updates which enable background fsck. I will also note in passing that this is yet another reason why the setting of soft updates needs to be done in newfs and/or tunefs and not as an option in /etc/fstab. Kirk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 10:48: 3 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id DDF5437B71A for ; Fri, 30 Mar 2001 10:47:55 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id KAA10189; Fri, 30 Mar 2001 10:47:46 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103301847.KAA10189@beastie.mckusick.com> To: Terry Lambert Subject: Re: Background Fsck Cc: arch@freebsd.org In-Reply-To: Your message of "Fri, 30 Mar 2001 12:52:29 GMT." <200103301252.FAA06540@usr05.primenet.com> Date: Fri, 30 Mar 2001 10:47:46 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG From: Terry Lambert Subject: Re: Background Fsck To: mckusick@mckusick.com (Kirk McKusick) Date: Fri, 30 Mar 2001 12:52:29 +0000 (GMT) Cc: arch@FreeBSD.ORG I have a question avout the safety of this approach: You don't seem to be able to distinguish between: 1) Hardware crash without data coruption - e.g. power failure 2) Hardware crash with data corruption - e.g. disk/controller/memory failure 3) Software crash without data corruption - e.q. resource availability failure, or panic as a result of coding error 4) Software crash with data corruption - e.g. a panic resulting from kernel data becoming corrupt, with an unknown interval preceeding the crash in which some of these structures might have had FS data in them, or a such crash in the FS code path itself, where the data corruption was a primary effect instead of a side effect It seems to me tha background checking is only safe in cases 1 and 3, and (the current California power grid reliability not withstanding), that these cases are not provably the statistically most common cases. The reason Whistle did not do this work earlier was that we were unable to address this concern adequately without non-volatile RAM to store the failure reason and the disk write cache status. Since panic reasons are mathematically indistinguishable in the limit, were were also unable to address differentiating 3 and 4, without placing the FS and I/O subsystem into a seperate protection domain. Even doing this, we would only gain some statistical protection against #4, which means the only value which we could add was to case #1, were we to invest in the additional hardware. In other words, it was not speed of fsck which drove Whistle to soft updates. My question is this: how were you able to address these issues in your implementation? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. In general, your observations are correct. In the current framework it is not possible to guarrantee that you can always sort out which of the four cases above you are in and to then take the correct action. Whistle needed to make those sorts of guarantees, and consequently could not fall back to something like background fsck. I do not purport to make this sort of guarantee. I say only that I will do the right thing in cases #1 and #3 and that I will do my best to detect that I am in cases #2 and #4 and exit gracefully after logging a message saying that an unexpected inconsistency has arisen and that manual intervention is needed. For systems where this is not good enough, the system administrator has the option of forcing foreground checks or not using soft updates at all. Kirk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 11:55:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id 39B2637B718 for ; Fri, 30 Mar 2001 11:55:40 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f2UJqwG97104; Fri, 30 Mar 2001 11:52:58 -0800 (PST) (envelope-from obrien) Date: Fri, 30 Mar 2001 11:52:52 -0800 From: "David O'Brien" To: Cy Schubert - ITSD Open Systems Group Cc: Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" , arch@FreeBSD.ORG Subject: Re: Startup scripts a la NetBSD Message-ID: <20010330115252.A93566@dragon.nuxi.com> References: <3AC4B808.9EB5806B@integratus.com> <200103301810.f2UIAGK06787@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103301810.f2UIAGK06787@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Fri, Mar 30, 2001 at 10:09:39AM -0800 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Mar 30, 2001 at 10:09:39AM -0800, Cy Schubert - ITSD Open Systems Group wrote: > Becoming more SYSV-like is not something most people in FreeBSD are > enamoured with. Hence I don't think it would be possible to implement > all the changes I would like to see. Correct. Make that HELL NO WE WON'T GO SVR4!!! The BSD init + NetBSD granular rc files is a nice compromise. And thus why I am pushing a move in that direction. -- -- David (obrien@FreeBSD.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 12:34:11 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id A814037B71A for ; Fri, 30 Mar 2001 12:34:08 -0800 (PST) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H70XLHVH; Fri, 30 Mar 2001 15:34:01 -0500 Reply-To: Randell Jesup To: Bakul Shah Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103301753.MAA03709@valiant.cnchost.com> From: Randell Jesup Date: 30 Mar 2001 15:36:18 -0500 In-Reply-To: Bakul Shah's message of "Fri, 30 Mar 2001 09:53:06 -0800" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Bakul Shah writes: >But like Terry I worry about disks write errors or blocks >going bad. And not knowing how well the bg fsck (or for that >matter soft-updates) copes with that gives me an uneasy >feeling. Perhaps bg fsck should not do anything if it >discovers structural inconsistency? This worries me too. However, "not doing anything" can be a problem if the system isn't being actively monitored. >I know I don't have to run it if I feel this way but the >point is to understand the behavior. Yes. >> Many improvements have been made to fsck over the years. Through >> there are undoubtedly more that could be made, there are no big >> easy improvements left. > >I was less than clear; what I was asking is if anyone has >looked at whether the fsck *algorithm* is optimal. I >wondered about its complexity -- is it O(n^2) or O(n log n) >or something else and whether it is the best possible >algorithm. Also think about the memory usage requirements. FS checkers on big drives can be memory hogs; I don't know about fsck. I'm guessing that it's O(n) so long as you don't run over memory requirements, but that could easily happen. -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 13:41:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from babel.spoiled.org (babel.spoiled.org [212.84.234.227]) by hub.freebsd.org (Postfix) with SMTP id 18C7337B71A for ; Fri, 30 Mar 2001 13:41:45 -0800 (PST) (envelope-from list-freebsd.arch@spoiled.org) Received: (qmail 9505 invoked by uid 8); 30 Mar 2001 21:41:37 -0000 From: thomas graichen Reply-To: thomas graichen X-Newsgroups: spoiled.freebsd.arch Subject: Re: Background Fsck Date: Fri, 30 Mar 2001 23:20:03 +0200 Organization: spoiled dot org Lines: 23 Distribution: local Message-ID: References: <200103290522.VAA06966@beastie.mckusick.com> <200103300053.TAA27553@thunderer.cnchost.com> <20010329220128.B21838@peorth.iteration.net> <20010329201156.P9431@fw.wintelcom.net> Reply-To: thomas graichen X-Complaints-To: newsmaster@spoiled.org User-Agent: tin/1.4.4-20000803 ("Vet for the Insane") (UNIX) (Linux/2.4.2-XFS (i586)) To: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > As far as speeding up fsck in general, I haven't heard anything, > suggestions are welcome. :) it might be worth to talk to ted tso about this - i vaguely remember him talking about some small optimisations and fixes he has found while writing the ext2 fsck ... i am not shure if all of those ideas made it back to the bsd fsck ... at least worth a check i think hope that helps t p.s.: ted tso is the e2fs utils maintainer for linux - should be easy to find it's email adress (i'm offline right now so i can't) - i think he is at valinux now ... -- thomas graichen ... perfection is reached, not when there is no longer anything to add, but when there is no longer anything to take away. --- antoine de saint-exupery To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 15: 2: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 7CE1037B75E for ; Fri, 30 Mar 2001 15:02:04 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.2/8.11.2) with ESMTP id f2UN1fG78484; Fri, 30 Mar 2001 15:01:41 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200103300602.WAA09248@beastie.mckusick.com> Date: Fri, 30 Mar 2001 15:01:17 -0800 (PST) From: John Baldwin To: Kirk McKusick Subject: Re: Background Fsck Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 30-Mar-01 Kirk McKusick wrote: > Date: Thu, 29 Mar 2001 21:48:28 -0800 (PST) > From: John Baldwin > To: Kirk McKusick > Subject: RE: Background Fsck > Cc: arch@FreeBSD.ORG > Sender: owner-freebsd-arch@FreeBSD.ORG > > On 29-Mar-01 Kirk McKusick wrote: > > I now have the capability in fsck_ffs to run a background fsck on an > > active filesystem. The question is how to deploy this capability in > > the system. I have discussed how best to deploy it with several folks > > and have put together the following proposal. I am now soliciting > > input from the larger `arch' audience. > > > > The default will be automatic background fsck of filesystems > > running with soft updates. It will be possible to override this > > default by adding the letter `F' to the passno in /etc/fstab > > (e.g., saying `2F' means do it in the second pass before going > > multiuser). > > Uppercase or either case? > > My plan was to make it upper case only. Do you feel that it should > be one or the other or case insensitive? Unix does not generally do > case insensitive. I prefer case insensitive as I don't think there is a good reason to differentiate. Personally, I would prefer lower-case (I guess I'm used to assembly labels :)), but it's not a big deal. Just wanted to clarify mostly. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 15:38:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 0503A37B718; Fri, 30 Mar 2001 15:38:47 -0800 (PST) (envelope-from Cy.Schubert@uumail.gov.bc.ca) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id PAA27200; Fri, 30 Mar 2001 15:38:40 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda27194; Fri Mar 30 15:38:38 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f2UNcXN08372; Fri, 30 Mar 2001 15:38:33 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdtf8370; Fri Mar 30 15:38:30 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.3/8.9.1) id f2UNcTN08783; Fri, 30 Mar 2001 15:38:29 -0800 (PST) Message-Id: <200103302338.f2UNcTN08783@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdYK8779; Fri Mar 30 15:38:10 2001 X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Kirk McKusick Cc: Robert Watson , arch@FreeBSD.ORG Subject: Re: Background Fsck In-reply-to: Your message of "Fri, 30 Mar 2001 10:32:41 PST." <200103301832.KAA10132@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 30 Mar 2001 15:38:10 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200103301832.KAA10132@beastie.mckusick.com>, Kirk McKusick writes: > Date: Fri, 30 Mar 2001 09:05:10 -0500 (EST) > From: Robert Watson > To: Kirk McKusick > cc: arch@freebsd.org > Subject: Re: Background Fsck > > Kirk, > > I haven't had a chance to look at the tunefs source lately, > but quick question: does tunefs block the setting of the > soft updates flag on a dirty file system? It seems to me > that, if it doesn't, a possible nasty sequence of events > is: system does unclean shutdown without soft updates, > administrator (possibly not realizing this) boots to > single-user mode, and sets soft updates, then attempts to > enter multi-user mode, where fsck says "ah, soft updates, > doesn't matter if it's unclean, let's background fsck". > Shortly thereafter, an inconsistency is discovered and the > system panics. As such, tunefs should only allow the > setting of soft updates on a file system marked clean. It > may already do this, but figured I should ask. > > Thanks! > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > robert@fledge.watson.org NAI Labs, Safeport Network Services > > Your observation is absolutely correct. I have modified tunefs > exactly as you suggest and will be checking in that change as > part of my next set of updates which enable background fsck. I > will also note in passing that this is yet another reason why > the setting of soft updates needs to be done in newfs and/or > tunefs and not as an option in /etc/fstab. I would think the integrity checking patch to tunefs(8) should be in -stable too. Could this little change be MFC'd into -stable sometime soon? Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 30 15:38:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 781DC37B71A for ; Fri, 30 Mar 2001 15:38:49 -0800 (PST) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id PAA11228; Fri, 30 Mar 2001 15:38:42 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200103302338.PAA11228@beastie.mckusick.com> To: Randell Jesup Subject: Re: Background Fsck Cc: Bakul Shah , arch@FreeBSD.ORG In-Reply-To: Your message of "30 Mar 2001 12:25:50 EST." Date: Fri, 30 Mar 2001 15:38:42 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG As far as I can tell, you have gone through and listed every error exit in fsck... Most of these can only arise if the logic is broken in some fundamental way. For example: errx(EEXIT, "BAD STATE %d FOR INODE I=%d", can only occur if one of the states defined for an inode by fsck is found. The state is derived from a classification made in pass 1. If the inode has had random stuff put into it, then it will be classified as USTATE. Similarly, inode numbers that are out of bounds should have been discovered at a higher level and dealt with (e.g., an out of bounds directory entry should have been found and zapped). I do not disagree that there may be some possibilities that slip through, but going through and wholesale getting rid of error exits is not the right approach. In general `fsck -p' will not fix everything, but `fsck -y' should always succeed (though success may be an empty filesystem). Kirk =-=-=-=-=-=-=-= To: Kirk McKusick Cc: Bakul Shah , arch@FreeBSD.ORG Subject: Re: Background Fsck From: Randell Jesup Date: 30 Mar 2001 12:25:50 -0500 Kirk McKusick writes: > On a somewhat related note, I have always wondered if the > current fsck algorithm can be significantly improved or if it > is about as efficient as it can be (barring any peephole code > improvements). > >Many improvements have been made to fsck over the years. Through >there are undoubtedly more that could be made, there are no big >easy improvements left. Last time I checked, fsck basically calls exit() on a number of types of errors (inode out of range was one of them I think - you had to manually cleari for each one and restart fsck). I munged fsck to try to recover a coworker's files when something went foobar in her 2.2.8 machine and wrote garbage across sectors all over her disk. Does this fix those sorts of "I can't deal with it and won't try no matter how hard you force me" cases? >From the current source - I know that many of these are internal errors that shouldn't happen, and are correct to exit out on. However, some (like inoinfo(), ginode(), maybe getnextinode()) should NOT just cause a blanket exit. utilities.c: (This is the one that caused me the big trouble, I think) ------- /* * Look up state information for an inode. */ struct inostat * inoinfo(inum) ino_t inum; { static struct inostat unallocated = { USTATE, 0, 0 }; struct inostatlist *ilp; int iloff; if (inum > maxino) errx(EEXIT, "inoinfo: inumber %d out of range", inum); inode.c: (this may very well have been it too) ------- /* * General purpose interface for reading inodes. */ struct dinode * ginode(inumber) ino_t inumber; { ufs_daddr_t iblk; if (inumber < ROOTINO || inumber > maxino) errx(EEXIT, "bad inode number %d to ginode", inumber); ... struct dinode * getnextinode(inumber) ino_t inumber; { long size; ufs_daddr_t dblk; static struct dinode *dp; if (inumber != nextino++ || inumber > maxino) errx(EEXIT, "bad inode number %d to nextinode", inumber); ... void setinodebuf(inum) ino_t inum; { if (inum % sblock.fs_ipg != 0) errx(EEXIT, "bad inode number %d to setinodebuf", inum); ... getinoinfo(inumber): errx(EEXIT, "cannot find inode %d", inumber); ... default: errx(EEXIT, "BAD STATE %d TO BLKERR", inoinfo(ino)->ino_state); dir.c: ------- if (idesc->id_type != DATA) errx(EEXIT, "wrong type to dirscan %d", idesc->id_type); pass2.c: ------- default: errx(EEXIT, "BAD STATE %d FOR ROOT INODE", inoinfo(ROOTINO)->ino_state); ... default: errx(EEXIT, "BAD STATE %d FOR INODE I=%d", inoinfo(dirp->d_ino)->ino_state, dirp->d_ino); pass4.c: ------- default: errx(EEXIT, "BAD STATE %d FOR INODE I=%d", inoinfo(inumber)->ino_state, inumber); pass5.c: ------- default: inomapsize = blkmapsize = sumsize = 0; /* keep lint happy */ errx(EEXIT, "UNKNOWN ROTATIONAL TABLE FORMAT %d", fs->fs_postblformat); ... default: if (j < ROOTINO) break; errx(EEXIT, "BAD STATE %d FOR INODE I=%ld", inoinfo(j)->ino_state, j); -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 31 4:12:43 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freesbee.wheel.dk (freesbee.wheel.dk [193.162.159.97]) by hub.freebsd.org (Postfix) with ESMTP id 3485E37B747 for ; Sat, 31 Mar 2001 04:12:38 -0800 (PST) (envelope-from ncbp@bank-pedersen.dk) Received: by freesbee.wheel.dk (Postfix, from userid 1002) id AE1DC5D6C; Sat, 31 Mar 2001 14:12:36 +0200 (CEST) Date: Sat, 31 Mar 2001 14:12:36 +0200 From: "Niels Chr. Bank-Pedersen" To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010331141236.B98023@bank-pedersen.dk> Mail-Followup-To: "Niels Chr. Bank-Pedersen" , Kirk McKusick , arch@FreeBSD.ORG References: <200103302338.PAA11228@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103302338.PAA11228@beastie.mckusick.com>; from mckusick@mckusick.com on Fri, Mar 30, 2001 at 03:38:42PM -0800 X-PGP-Fingerprint: 18D0 73F3 767F 3A40 CEBA C595 4783 D7F5 5DD1 FB8C X-PGP-Public-Key: http://freesbee.wheel.dk/~ncbp/gpgkey.pub Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Mar 30, 2001 at 03:38:42PM -0800, Kirk McKusick wrote: [...] > In general `fsck -p' will not fix > everything, but `fsck -y' should always succeed (though > success may be an empty filesystem). This made me remember a problem I see on a -current machine (rebuild yesterday): THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: ufs: /dev/da0s1f (/var) Automatic file system check failed . . . help! Enter full pathname of shell or RETURN for /bin/sh: # fsck -y /var ** /dev/da0s1f ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes PARTIALLY TRUNCATED INODE I=31753 UNEXPECTED SOFT UPDATE INCONSISTENCY INCORRECT BLOCK COUNT I=39698 (1904 should be 1808) CORRECT? yes PARTIALLY TRUNCATED INODE I=87341 UNEXPECTED SOFT UPDATE INCONSISTENCY ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 391 files, 128648 used, 407159 free (351 frags, 50851 blocks, 0.1% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** # fsck -y /var ** /dev/da0s1f ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes PARTIALLY TRUNCATED INODE I=31753 UNEXPECTED SOFT UPDATE INCONSISTENCY PARTIALLY TRUNCATED INODE I=87341 UNEXPECTED SOFT UPDATE INCONSISTENCY ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 391 files, 128648 used, 407159 free (351 frags, 50851 blocks, 0.1% fragmentation) # tunefs -p /var tunefs: soft updates: (-n) enabled tunefs: maximum contiguous block count: (-a) 15 tunefs: rotational delay between contiguous blocks: (-d) 0 ms tunefs: maximum blocks per file in a cylinder group: (-e) 2048 tunefs: minimum percentage of free space: (-m) 8% tunefs: optimization preference: (-o) time As I read your mail, this should not be possible, or? > Kirk TIA, Niels Chr. -- Niels Christian Bank-Pedersen, NCB1-RIPE. Network Manager, Tele Danmark NET, IP-section. "Hey, are any of you guys out there actually *using* RFC 2549?" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message