From owner-freebsd-fs Sun Mar 11 21:22:43 2001 Delivered-To: freebsd-fs@freebsd.org Received: from klapaucius.zer0.org (klapaucius.zer0.org [204.152.186.45]) by hub.freebsd.org (Postfix) with ESMTP id B7EE337B718; Sun, 11 Mar 2001 21:22:40 -0800 (PST) (envelope-from gsutter@zer0.org) Received: by klapaucius.zer0.org (Postfix, from userid 1001) id 779A3239A54; Sun, 11 Mar 2001 21:22:40 -0800 (PST) Date: Sun, 11 Mar 2001 21:22:40 -0800 From: Gregory Sutter To: Robert Watson Cc: Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org Subject: Re: httpfs Message-ID: <20010311212240.D9369@klapaucius.zer0.org> References: <20010310031515.A8998@mollari.cthul.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from rwatson@FreeBSD.org on Sat, Mar 10, 2001 at 01:36:30PM -0500 Organization: Zer0 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 2001-03-10 13:36 -0500, Robert Watson wrote: > On Sat, 10 Mar 2001, Kris Kennaway wrote: > > > A few of us were talking on IRC tonight about how cool it would be to > > have an httpfs filesystem -- then it occurred to me we almost have > > this already, in the form of the (under-utilised) portalfs. Portalfs > > works by handing off everything to a userland daemon which handles the > > actual transaction request, so you could easily imagine extending it > > to provide an http method similar to the tcp method it currently has > > for initiating tcp connections. > > I need not remind you that file systems front-ending onto random > protocols are a bad idea for a huge number of reasons :-). Could you give me the three biggest reasons, IYO? I don't seem to know any of them. Thanks! Greg -- Gregory S. Sutter Five million battered women in mailto:gsutter@zer0.org this country, and I've always http://www.zer0.org/~gsutter/ eaten mine plain... hkp://wwwkeys.pgp.net/0x845DFEDD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 2:21:26 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mail.gmx.net (pop.gmx.net [194.221.183.20]) by hub.freebsd.org (Postfix) with SMTP id DA11137B718 for ; Mon, 12 Mar 2001 02:21:23 -0800 (PST) (envelope-from Eckhard.Kantz@gmx.de) Received: (qmail 25157 invoked by uid 0); 12 Mar 2001 10:21:22 -0000 Received: from unknown (HELO Maja) (212.12.57.178) by mail.gmx.net (mp006-rz3) with SMTP; 12 Mar 2001 10:21:22 -0000 Message-ID: <00f201c0aade$30b1e380$0100a8c0@wegalink.net> From: "Eckhard Kantz" To: References: Subject: Re: httpfs Date: Mon, 12 Mar 2001 11:21:22 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, 10 Mar 2001, Kris Kennaway wrote: > A fully navigable httpfs (e.g. one you can ls and cd around in) is > more work and probably requires a full stacking layer, but this would > still be pretty cool. > > Is anyone feeling inspired to implement this? :-) Starting most likely end of this month I am going to do a project where a httpfs modul will be implemented as a virtual filesystem. This module should allow to navigate through all files located at a web site as well as to modify, create and delete them. The httpfs module will behave accordingly to the WebDAV specification (see http://www.ics.uci.edu/~ejw/authoring/) and as such it should collaborate with all web servers supporting that specification. In opposite to other approaches no additional CGI is needed to access files, database entries, etc. at the web server since this functionality is provided by WebDAV. Until now I don't have much experience with implementing a new virtual filesystem in FreeBSD, so all advice and suggestions where to find more related information are really appreciated. I had allready a short look into the portalfs sources and it seems that this would be a good starting point? An open issue is to find the right strategy for authentication of a local user to the remote web site. Any ideas are welcome. Regards, Eckhard To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 15:24:48 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 84D2D37B71A; Mon, 12 Mar 2001 15:24:43 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2CNNOS25501; Mon, 12 Mar 2001 15:23:24 -0800 (PST) Date: Mon, 12 Mar 2001 15:23:24 -0800 From: Alfred Perlstein To: "Justin T. Gibbs" Cc: Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA Message-ID: <20010312152324.H18351@fw.wintelcom.net> References: <20010312151024.E18351@fw.wintelcom.net> <200103122319.f2CNJ8s42439@aslan.scsiguy.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200103122319.f2CNJ8s42439@aslan.scsiguy.com>; from gibbs@scsiguy.com on Mon, Mar 12, 2001 at 04:19:08PM -0700 X-all-your-base: are belong to us. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org cc'd -fs and Matt. :) * Justin T. Gibbs [010312 15:19] wrote: > >All i need to know is if there's a way to look at a particular > >struct bio/buf and determine if there's a need to write the > >data sync. Soren claims that B_ORDERED is not the magic bit that's > >needed, is there one? Or are we SOL on the kernel communicating the > >need for a block to be written "right now" and not complete until > >sync'd to the backing storage's non-volatile media? > > Most, if not all, of the kernel assumes that the data is committed to > non-volatile storage when the write is notified as complete. FFS > and softupdates could survive if they marked meta data or the first > buffer across a dependency domain respectively as B_ORDERED assuming > the device commits to media in the expected order, but softupdates > doesn't do this, and FFS probably doesn't do this correctly. Hmm, it wouldn't be that hard to have the bwrite() functions something with the buf that says, I'm waiting for this to complete and it damn better be complete when i get it back. (*) The ones that wait for completetion rather than bawrite() which just asks for an async write. The only problem with this is that sometimes bwrite is used when there's a shortage and it'd be nice to be able to take advantage of write caching in those instances. Hopefully one of us will look into it one of these days. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 15:34: 8 2001 Delivered-To: freebsd-fs@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id E29FF37B71B; Mon, 12 Mar 2001 15:33:59 -0800 (PST) (envelope-from mjacob@feral.com) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id PAA04673; Mon, 12 Mar 2001 15:33:40 -0800 Date: Mon, 12 Mar 2001 15:33:37 -0800 (PST) From: Matthew Jacob Reply-To: mjacob@feral.com To: Alfred Perlstein Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <20010312152324.H18351@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > cc'd -fs and Matt. :) > > * Justin T. Gibbs [010312 15:19] wrote: > > >All i need to know is if there's a way to look at a particular > > >struct bio/buf and determine if there's a need to write the > > >data sync. Soren claims that B_ORDERED is not the magic bit that's > > >needed, is there one? Or are we SOL on the kernel communicating the > > >need for a block to be written "right now" and not complete until > > >sync'd to the backing storage's non-volatile media? > > > > Most, if not all, of the kernel assumes that the data is committed to > > non-volatile storage when the write is notified as complete. FFS > > and softupdates could survive if they marked meta data or the first > > buffer across a dependency domain respectively as B_ORDERED assuming > > the device commits to media in the expected order, but softupdates > > doesn't do this, and FFS probably doesn't do this correctly. > > Hmm, it wouldn't be that hard to have the bwrite() functions > something with the buf that says, I'm waiting for this to complete > and it damn better be complete when i get it back. Ordered tags, I believe, just guarantee the order in which things are done. If you have caching enable, then you'd better have something else which does send a Synchronize Cache. Oh- I take it back- you don't need a separate command. You can actually set a bit in the WRITE command that forces things out to media. So, what you really need to do is set both B_ORDERED and a new B_FLUSH (say) if you want to not only guarantee a barrier between this op and ones that follow, but also force this op to go to disk. You could also do a Head of Queue tag if you want to guarantee that this op goes to disk write away (ahead of everyone else). If you need everything sync'd, you'll have to explicitly have the da driver send the SYNCHRONIZE CACHE command. > (*) > The ones that wait for completetion rather than bawrite() which > just asks for an async write. > > The only problem with this is that sometimes bwrite is used when > there's a shortage and it'd be nice to be able to take advantage > of write caching in those instances. > > Hopefully one of us will look into it one of these days. > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 15:46:21 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id F0CD937B718; Mon, 12 Mar 2001 15:46:11 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2CNiq126258; Mon, 12 Mar 2001 15:44:52 -0800 (PST) Date: Mon, 12 Mar 2001 15:44:52 -0800 From: Alfred Perlstein To: Matthew Jacob Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA Message-ID: <20010312154452.K18351@fw.wintelcom.net> References: <20010312152324.H18351@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from mjacob@feral.com on Mon, Mar 12, 2001 at 03:33:37PM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Matthew Jacob [010312 15:33] wrote: > > > > cc'd -fs and Matt. :) > > > > * Justin T. Gibbs [010312 15:19] wrote: > > > >All i need to know is if there's a way to look at a particular > > > >struct bio/buf and determine if there's a need to write the > > > >data sync. Soren claims that B_ORDERED is not the magic bit that's > > > >needed, is there one? Or are we SOL on the kernel communicating the > > > >need for a block to be written "right now" and not complete until > > > >sync'd to the backing storage's non-volatile media? > > > > > > Most, if not all, of the kernel assumes that the data is committed to > > > non-volatile storage when the write is notified as complete. FFS > > > and softupdates could survive if they marked meta data or the first > > > buffer across a dependency domain respectively as B_ORDERED assuming > > > the device commits to media in the expected order, but softupdates > > > doesn't do this, and FFS probably doesn't do this correctly. > > > > Hmm, it wouldn't be that hard to have the bwrite() functions > > something with the buf that says, I'm waiting for this to complete > > and it damn better be complete when i get it back. > > Ordered tags, I believe, just guarantee the order in which things are done. If > you have caching enable, then you'd better have something else which does send > a Synchronize Cache. > > Oh- I take it back- you don't need a separate command. You can actually set a > bit in the WRITE command that forces things out to media. > > So, what you really need to do is set both B_ORDERED and a new B_FLUSH > (say) if you want to not only guarantee a barrier between this op and ones > that follow, but also force this op to go to disk. > > You could also do a Head of Queue tag if you want to guarantee that this op > goes to disk write away (ahead of everyone else). > > If you need everything sync'd, you'll have to explicitly have the da driver > send the SYNCHRONIZE CACHE command. Afaik we never depend on actually B_ORDERED'd data, at least not for filesystem consistancy (as afaik it would violate USL patents). We really don't need the SYNCHRONIZE CACHE either, basically a B_FLUSH would be nice, the problem I see is making sure it doesn't get eaten by the millions :) of OR's and AND's on the b_flags fields. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 15:53:30 2001 Delivered-To: freebsd-fs@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id CDE8F37B71A; Mon, 12 Mar 2001 15:53:24 -0800 (PST) (envelope-from mjacob@feral.com) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id PAA04726; Mon, 12 Mar 2001 15:53:13 -0800 Date: Mon, 12 Mar 2001 15:53:10 -0800 (PST) From: Matthew Jacob Reply-To: mjacob@feral.com To: Alfred Perlstein Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <20010312154452.K18351@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > If you need everything sync'd, you'll have to explicitly have the da driver > > send the SYNCHRONIZE CACHE command. > > Afaik we never depend on actually B_ORDERED'd data, at least not > for filesystem consistancy (as afaik it would violate USL patents). Hmm? Wasn't there an assertion about this from Terry IIRC? Huh.. as far as I remember Steve Kleiman was talking about this at Sun in 1990- dunno if this has any pertinence to the patent or usage. > We really don't need the SYNCHRONIZE CACHE either, basically a B_FLUSH > would be nice, the problem I see is making sure it doesn't get eaten > by the millions :) of OR's and AND's on the b_flags fields. Well, if I understood your mail, you were trying to find out a way to live safely with not only ordering (which is done in s/w with softupdates for FreeBSD, right?- last I heard from Steve Tweedie was that he was considering using tags for ordering in ext3) but also to try and live safely with data that is cached on the drive electronics but not committed to media yet. Did I misunderstand your questions to Justin? It's quite possible as I've been switching between 5 different systems I'm working on today and I'm sure that things didn't committed to &my& media before things were wiped as switched from one KDE desktop to another..... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 16: 9:28 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id BC67337B71A; Mon, 12 Mar 2001 16:09:20 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2D07he27097; Mon, 12 Mar 2001 16:07:43 -0800 (PST) Date: Mon, 12 Mar 2001 16:07:43 -0800 From: Alfred Perlstein To: Matthew Jacob Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA Message-ID: <20010312160742.L18351@fw.wintelcom.net> References: <20010312154452.K18351@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from mjacob@feral.com on Mon, Mar 12, 2001 at 03:53:10PM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Matthew Jacob [010312 15:53] wrote: > > > If you need everything sync'd, you'll have to explicitly have the da driver > > > send the SYNCHRONIZE CACHE command. > > > > Afaik we never depend on actually B_ORDERED'd data, at least not > > for filesystem consistancy (as afaik it would violate USL patents). > > Hmm? Wasn't there an assertion about this from Terry IIRC? Huh.. as far as I > remember Steve Kleiman was talking about this at Sun in 1990- dunno if this > has any pertinence to the patent or usage. > > > We really don't need the SYNCHRONIZE CACHE either, basically a B_FLUSH > > would be nice, the problem I see is making sure it doesn't get eaten > > by the millions :) of OR's and AND's on the b_flags fields. > > Well, if I understood your mail, you were trying to find out a way to live > safely with not only ordering (which is done in s/w with softupdates for > FreeBSD, right?- last I heard from Steve Tweedie was that he was considering > using tags for ordering in ext3) but also to try and live safely with data > that is cached on the drive electronics but not committed to media yet. > > Did I misunderstand your questions to Justin? It's quite possible as I've been > switching between 5 different systems I'm working on today and I'm sure that > things didn't committed to &my& media before things were wiped as switched > from one KDE desktop to another..... Your previous paragraph "Well, if I understood your mail" is what I was looking for. Basically I was inquiring if the hardware could/would support a request to perform a write-through or not lie about the write completing for certain tagged writes. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 16:19:52 2001 Delivered-To: freebsd-fs@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id A38E937B718; Mon, 12 Mar 2001 16:19:47 -0800 (PST) (envelope-from mjacob@feral.com) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id QAA04885; Mon, 12 Mar 2001 16:18:44 -0800 Date: Mon, 12 Mar 2001 16:18:41 -0800 (PST) From: Matthew Jacob Reply-To: mjacob@feral.com To: Alfred Perlstein Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <20010312160742.L18351@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > We really don't need the SYNCHRONIZE CACHE either, basically a B_FLUSH > > > would be nice, the problem I see is making sure it doesn't get eaten > > > by the millions :) of OR's and AND's on the b_flags fields. > > > > Well, if I understood your mail, you were trying to find out a way to live > > safely with not only ordering (which is done in s/w with softupdates for > > FreeBSD, right?- last I heard from Steve Tweedie was that he was considering > > using tags for ordering in ext3) but also to try and live safely with data > > that is cached on the drive electronics but not committed to media yet. > > > > Did I misunderstand your questions to Justin? It's quite possible as I've been > > switching between 5 different systems I'm working on today and I'm sure that > > things didn't committed to &my& media before things were wiped as switched > > from one KDE desktop to another..... > > Your previous paragraph "Well, if I understood your mail" is what I was > looking for. Basically I was inquiring if the hardware could/would > support a request to perform a write-through or not lie about the > write completing for certain tagged writes. There's no 'lying' here- the writes are completing. Those are "done" operations. It just depends on the mode you've set the disk (or tape, for that matter, which is usual) whether or not done means "on media" or "in drive cache". These operations are so done, in fact, that if there is an error later in putting them to media itself, there'll be all kinds of interesting toe stubbing going on. What I referred to in my previous mail is setting the FUA bit in the write. In the SCSI-2 spec, section 9.1.6, one paragraph states, in part: ...For a write operation, setting FUA to one causes the direct-access device to complete the data write to the physical medium before completing the command.... Does think this sounds like what you wan? You can also force data to not be in the drive cache as well (even when committed to media), so you can do things like not blow your drive cache on data you don't expect to need to re-read for a while- but this is cleverer than what is needed. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 17:10:12 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id BF51A37B719; Mon, 12 Mar 2001 17:09:56 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2D196Q29022; Mon, 12 Mar 2001 17:09:06 -0800 (PST) Date: Mon, 12 Mar 2001 17:09:05 -0800 From: Alfred Perlstein To: Matthew Jacob Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA Message-ID: <20010312170905.O18351@fw.wintelcom.net> References: <20010312160742.L18351@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from mjacob@feral.com on Mon, Mar 12, 2001 at 04:18:41PM -0800 X-all-your-base: are belong to us. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Matthew Jacob [010312 16:18] wrote: > > > > > We really don't need the SYNCHRONIZE CACHE either, basically a B_FLUSH > > > > would be nice, the problem I see is making sure it doesn't get eaten > > > > by the millions :) of OR's and AND's on the b_flags fields. > > > > > > Well, if I understood your mail, you were trying to find out a way to live > > > safely with not only ordering (which is done in s/w with softupdates for > > > FreeBSD, right?- last I heard from Steve Tweedie was that he was considering > > > using tags for ordering in ext3) but also to try and live safely with data > > > that is cached on the drive electronics but not committed to media yet. > > > > > > Did I misunderstand your questions to Justin? It's quite possible as I've been > > > switching between 5 different systems I'm working on today and I'm sure that > > > things didn't committed to &my& media before things were wiped as switched > > > from one KDE desktop to another..... > > > > Your previous paragraph "Well, if I understood your mail" is what I was > > looking for. Basically I was inquiring if the hardware could/would > > support a request to perform a write-through or not lie about the > > write completing for certain tagged writes. > > There's no 'lying' here- the writes are completing. Those are "done" > operations. It just depends on the mode you've set the disk (or tape, for that > matter, which is usual) whether or not done means "on media" or "in drive > cache". These operations are so done, in fact, that if there is an error later > in putting them to media itself, there'll be all kinds of interesting toe > stubbing going on. > > What I referred to in my previous mail is setting the FUA bit in the write. > > In the SCSI-2 spec, section 9.1.6, one paragraph states, in part: > > ...For a write operation, setting FUA to one causes the direct-access > device to complete the data write to the physical medium before > completing the command.... Being able to tell the driver to do this from software would be the best. This would allow us to use writecaching for data, but force stable storage for meta-data. I think we'd also want to use this for forced data sync (fsync(2) and files opened with O_SYNC). > Does think this sounds like what you wan? Yes. > You can also force data to not be in > the drive cache as well (even when committed to media), so you can do things > like not blow your drive cache on data you don't expect to need to re-read for > a while- but this is cleverer than what is needed. This looks like an optimization that could be used for things like physio and probably any case mentioned above (B_PUSH). -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 12 20: 5:14 2001 Delivered-To: freebsd-fs@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 56E9737B719; Mon, 12 Mar 2001 20:05:09 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id UAA12199; Mon, 12 Mar 2001 20:59:11 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp04.primenet.com, id smtpdAAAG5aiOx; Mon Mar 12 20:59:01 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id VAA03536; Mon, 12 Mar 2001 21:04:44 -0700 (MST) From: Terry Lambert Message-Id: <200103130404.VAA03536@usr05.primenet.com> Subject: Re: Disk I/O problem in 4.3-BETA To: mjacob@feral.com Date: Tue, 13 Mar 2001 04:04:38 +0000 (GMT) Cc: bright@wintelcom.net (Alfred Perlstein), gibbs@scsiguy.com (Justin T. Gibbs), sos@freebsd.dk (Soren Schmidt), oberman@es.net (Kevin Oberman), scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG In-Reply-To: from "Matthew Jacob" at Mar 12, 2001 03:53:10 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Afaik we never depend on actually B_ORDERED'd data, at least not > > for filesystem consistancy (as afaik it would violate USL patents). > > Hmm? Wasn't there an assertion about this from Terry IIRC? Huh.. as far as I > remember Steve Kleiman was talking about this at Sun in 1990- dunno if this > has any pertinence to the patent or usage. Depends on how you use them; the actual assignee was Novell; I got to see the filing materials back in 1993, when I was a Novell/USG (former USL, from the Novell side) attributed FS guru; see: US5642501:Computer method and apparatus for asynchronous ordered operations http://www.delphion.com/details?pn=US05642501__ US5666532:Computer method and apparatus for asynchronous ordered operations http://www.delphion.com/details?pn=US05666532__ Be warned; they are 175 and 34 pages, respectively. > Did I misunderstand your questions to Justin? It's quite possible as I've been > switching between 5 different systems I'm working on today and I'm sure that > things didn't committed to &my& media before things were wiped as switched > from one KDE desktop to another..... I think Alfred just wants them to be forced to stable storage before a dependency is considered statisfied; if so, there's no patent issue with D.O.W. (Delayed Ordered Writes). There might be an issue with one of the claims in the first patent, if you were to do write gathering, like Alfred had suggested. Doing that would be a mistake anyway: you might as well mount the thing async if you are going to write cache dependencies such that they aren't committed in the correct order, and a partial commit would then be possible. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 8:28:36 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mx10.quantum.com (mx10.quantum.com [204.212.103.176]) by hub.freebsd.org (Postfix) with ESMTP id 7CFE837B721; Tue, 13 Mar 2001 08:28:25 -0800 (PST) (envelope-from Stephen.Byan@quantum.com) Received: from milcmima.qntm.com (milcmima.qntm.com [146.174.18.61]) by mx10.quantum.com (8.9.3+Sun/8.9.3) with ESMTP id IAA22570; Tue, 13 Mar 2001 08:24:53 -0800 (PST) Received: by milcmima.qntm.com with Internet Mail Service (5.5.2650.21) id ; Tue, 13 Mar 2001 08:26:17 -0800 Message-ID: <38E52A0B1357D411A42400805FA79384019631CE@shrcmsgb.tdh.qntm.com> From: Stephen Byan To: "'Alfred Perlstein'" , Matthew Jacob Cc: "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: RE: Disk I/O problem in 4.3-BETA Date: Tue, 13 Mar 2001 08:26:10 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred Perlstein [mailto:bright@wintelcom.net] wrote: > This would allow us to use writecaching for data, but force > stable storage for meta-data. I think we'd also want to use > this for forced data sync (fsync(2) and files opened with O_SYNC). I do hope this gets implemented. I did something similar at Hitachi in our OSF/1 port to the S/390 back around 1992. The disk controllers had a substantial amount of volatile RAM cache, and a lesser amount of NV-RAM cache. We directed the metadata writes to NV-RAM, and the data to volatile cache (with a flush at partition close, of course). Since the hit rate on metadata writes in UFS is very high, even with a small NV cache, we were able to get substantial speedups in metadata intensive operations such as a recursive directory copy. We also implemented sequential I/O hinting, detected by the read-ahead mechanism in the file system. Passing this hint down allowed the controller to do a better job of cache management: sequential I/O recycled the buffers after they had been read or written, rather than aging them through the LRU list, so sequential reads and writes didn't trash the cache. For NFS v2, it's also helpful to be able to mark the write I/O's as non-cachable, thus hinting them toward NV-RAM. In Windows NT, NTFS uses the SCSI Force Unit Access (FUA) bit on it's metadata writes (log writes for sure; it should also use FUA for lazy writes of metadata, else there is a race condition for recycling the log entry, no?, but I don't know whether it actually uses FUA for lazy writes of metadata). It doesn't hint the SCSI CDBs with sequential access information, but such info is available in the IRP presented to the SCSI class driver, and a filter driver could do some magic... If NT and FreeBSD both support hinting of metadata writes, it's only a matter of time before the hardware support appears. Regards, -Steve Steve Byan Design Engineer Quantum Corporation MS 1-3/E23 333 South Street Shrewsbury, MA 01545 (508)770-3414 fax: (508)770-2604 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 8:37:58 2001 Delivered-To: freebsd-fs@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.136]) by hub.freebsd.org (Postfix) with ESMTP id 2ADAC37B718; Tue, 13 Mar 2001 08:37:54 -0800 (PST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.9.3/8.9.3) with ESMTP id RAA45397; Tue, 13 Mar 2001 17:37:52 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f2DGcAp08823; Tue, 13 Mar 2001 17:38:10 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Stephen Byan Cc: "'Alfred Perlstein'" , Matthew Jacob , "Justin T. Gibbs" , Soren Schmidt , Kevin Oberman , scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: Your message of "Tue, 13 Mar 2001 08:26:10 PST." <38E52A0B1357D411A42400805FA79384019631CE@shrcmsgb.tdh.qntm.com> Date: Tue, 13 Mar 2001 17:38:10 +0100 Message-ID: <8821.984501490@critter> From: Poul-Henning Kamp Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >If NT and FreeBSD both support hinting of metadata writes, it's only a >matter of time before the hardware support appears. I have been looking for a long time for a PCI NVRAM card at a reasonable cost, anyone know of any with a reasonable price ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 12:28:11 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id B688937B719; Tue, 13 Mar 2001 12:28:06 -0800 (PST) (envelope-from cdillon@wolves.k12.mo.us) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id OAA54718; Tue, 13 Mar 2001 14:28:03 -0600 (CST) (envelope-from cdillon@wolves.k12.mo.us) Date: Tue, 13 Mar 2001 14:28:03 -0600 (CST) From: Chris Dillon To: Poul-Henning Kamp Cc: , Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <8821.984501490@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org CCs trimmed... On Tue, 13 Mar 2001, Poul-Henning Kamp wrote: > > >If NT and FreeBSD both support hinting of metadata writes, it's only a > >matter of time before the hardware support appears. > > I have been looking for a long time for a PCI NVRAM card at a > reasonable cost, anyone know of any with a reasonable price ? How "non-volatile" do you want it to be? I realize truly non-volatile means "forever" (i.e. Flash EEPROM, MRAM, or similar). However, a few minutes/hours (battery-backed volatile RAM) might be enough? The relatively cheap Mylex AcceleRAID 170 we just recently purchased has a "super-cap" (a very high-capacity capacitor, but not exactly a battery) on it, which could theoretically supply power to the on-board cache for at least a few minutes. The documentation and spec sheets for the board mention absolutely nothing about it, but its there. The low-profile version of the board doesn't appear to have the "super-cap", though. I've also got an AMI MegaRAID Enterprise 1200 at home with the small Ni-Cad battery backup module, and it should be able to keep the memory alive for at least a few hours. I think all of AMI's Enterprise level controllers offer that option. -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 12:44:54 2001 Delivered-To: freebsd-fs@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.136]) by hub.freebsd.org (Postfix) with ESMTP id 09A5937B71D; Tue, 13 Mar 2001 12:44:50 -0800 (PST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA48930; Tue, 13 Mar 2001 21:44:48 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f2DKj5p10188; Tue, 13 Mar 2001 21:45:05 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Chris Dillon Cc: scsi@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: Your message of "Tue, 13 Mar 2001 14:28:03 CST." Date: Tue, 13 Mar 2001 21:45:05 +0100 Message-ID: <10186.984516305@critter> From: Poul-Henning Kamp Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message , Ch ris Dillon writes: >> I have been looking for a long time for a PCI NVRAM card at a >> reasonable cost, anyone know of any with a reasonable price ? > >How "non-volatile" do you want it to be? I would want something on the order of an day or so, to cover catastrophic failures. My ideal product would be a PCI card with some SRAM, and a holder for a 9v DURACELL. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 12:52: 7 2001 Delivered-To: freebsd-fs@freebsd.org Received: from post.mail.nl.demon.net (post-11.mail.nl.demon.net [194.159.73.21]) by hub.freebsd.org (Postfix) with ESMTP id 7F5C337B71D; Tue, 13 Mar 2001 12:51:59 -0800 (PST) (envelope-from wkb@freebie.demon.nl) Received: from [212.238.54.101] (helo=freebie.demon.nl) by post.mail.nl.demon.net with smtp (Exim 3.14 #4) id 14cvlu-000Nc7-00; Tue, 13 Mar 2001 20:51:58 +0000 Received: (from wkb@localhost) by freebie.demon.nl (8.11.2/8.11.2) id f2DKs5N04591; Tue, 13 Mar 2001 21:54:05 +0100 (CET) (envelope-from wkb) Date: Tue, 13 Mar 2001 21:54:05 +0100 From: Wilko Bulte To: Poul-Henning Kamp Cc: Chris Dillon , scsi@freebsd.org, fs@freebsd.org Subject: Re: Disk I/O problem in 4.3-BETA Message-ID: <20010313215405.A4567@freebie.demon.nl> References: <10186.984516305@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <10186.984516305@critter>; from phk@critter.freebsd.dk on Tue, Mar 13, 2001 at 09:45:05PM +0100 X-OS: FreeBSD 4.2-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Mar 13, 2001 at 09:45:05PM +0100, Poul-Henning Kamp wrote: > In message , Ch > ris Dillon writes: > > >> I have been looking for a long time for a PCI NVRAM card at a > >> reasonable cost, anyone know of any with a reasonable price ? > > > >How "non-volatile" do you want it to be? > > I would want something on the order of an day or so, to cover > catastrophic failures. > > My ideal product would be a PCI card with some SRAM, and a holder > for a 9v DURACELL. Sounds like a PCI Prestoserve card to me. DEC used to produce these. Price? Dunno, probably steep. But Prestoserve cards might find themselves now on the scrap heaps. W/ -- | / o / / _ Arnhem, The Netherlands email: wilko@freebsd.org |/|/ / / /( (_) Bulte http://www.freebsd.org http://www.nlfug.nl To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 13:49: 2 2001 Delivered-To: freebsd-fs@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.136]) by hub.freebsd.org (Postfix) with ESMTP id 44E3D37B71B; Tue, 13 Mar 2001 13:48:57 -0800 (PST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.9.3/8.9.3) with ESMTP id WAA49915; Tue, 13 Mar 2001 22:48:55 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f2DLnDp10846; Tue, 13 Mar 2001 22:49:13 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Wilko Bulte Cc: Chris Dillon , scsi@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: Your message of "Tue, 13 Mar 2001 21:54:05 +0100." <20010313215405.A4567@freebie.demon.nl> Date: Tue, 13 Mar 2001 22:49:13 +0100 Message-ID: <10844.984520153@critter> From: Poul-Henning Kamp Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <20010313215405.A4567@freebie.demon.nl>, Wilko Bulte writes: >> My ideal product would be a PCI card with some SRAM, and a holder >> for a 9v DURACELL. > >Sounds like a PCI Prestoserve card to me. DEC used to produce these. >Price? Dunno, probably steep. But Prestoserve cards might find themselves >now on the scrap heaps. Yeah, well, I can't rely on them showing up on eBay if I want to deploy them, can I ? :-( Anyone know a sympathetic hardware vendor who might be co-opted into producing a product if we promise to make FreeBSD users want one ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 15:34:25 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 3D00437B718; Tue, 13 Mar 2001 15:34:19 -0800 (PST) (envelope-from cdillon@wolves.k12.mo.us) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id RAA58080; Tue, 13 Mar 2001 17:34:14 -0600 (CST) (envelope-from cdillon@wolves.k12.mo.us) Date: Tue, 13 Mar 2001 17:34:13 -0600 (CST) From: Chris Dillon To: Poul-Henning Kamp Cc: Wilko Bulte , , Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <10844.984520153@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 13 Mar 2001, Poul-Henning Kamp wrote: > In message <20010313215405.A4567@freebie.demon.nl>, Wilko Bulte writes: > > >> My ideal product would be a PCI card with some SRAM, and a holder > >> for a 9v DURACELL. > > > >Sounds like a PCI Prestoserve card to me. DEC used to produce these. > >Price? Dunno, probably steep. But Prestoserve cards might find themselves > >now on the scrap heaps. > > Yeah, well, I can't rely on them showing up on eBay if I want to > deploy them, can I ? :-( > > Anyone know a sympathetic hardware vendor who might be co-opted into > producing a product if we promise to make FreeBSD users want one ? It should be fairly trivial to increase the battery capacity to cover your needs on the existing boards that already offer the ability, even by yourself, if not by the manufacturer itself. I've seen external RAID controllers that used 6V lead-acid batteries for the memory backup... at that point, it would be very easy to stick a larger battery (or batteries) in place to meet your needs. I'm not sure who makes PCI boards which use external batteries, though, nor ones that use low-power SRAM instead of DRAM (which would actually be a good application for the super-caps, since they could power the SRAM much longer than DRAM). -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 15:46: 4 2001 Delivered-To: freebsd-fs@freebsd.org Received: from eozoon.coleman.org (adsl-209-233-238-136.dsl.snfc21.pacbell.net [209.233.238.136]) by hub.freebsd.org (Postfix) with ESMTP id A545637B741; Tue, 13 Mar 2001 15:45:49 -0800 (PST) (envelope-from don@eozoon.coleman.org) Received: from eozoon.coleman.org (eozoon.coleman.org [127.0.0.1] (may be forged)) by eozoon.coleman.org (8.9.3/8.9.3) with ESMTP id PAA28094; Tue, 13 Mar 2001 15:45:37 -0800 (PST) Message-Id: <200103132345.PAA28094@eozoon.coleman.org> X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Poul-Henning Kamp Reply-To: don@coleman.org Cc: Wilko Bulte , Chris Dillon , scsi@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-reply-to: Your message of "Tue, 13 Mar 2001 22:49:13 +0100." <10844.984520153@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Mar 2001 15:45:37 -0800 From: Don Coleman Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The original legato prestoserve board (VME bus) was manufactured by Micro Memory Inc, 9540 Vassar Ave Chatsworth, CA 91311 818-998-0070 Our contact was Mose' Jadon. The address and phone # are circa 1989. Amazingly enough, a web search turns up http://www.micromemory.com, and the old phone # is still valid. The original board was called the MM6704c by Micro Memory. A design firm we didn't pick wanted a bit under $100,000 for a custom engineering design, plus about $1000 per board. A PCI board is *much* smaller then a 9U VME board, and I'd expect the boards to be a lot cheaper. I think you'd want at least a couple weeks of backup, since if the machine crashes due to a hardware failure, it may not come back up soon, and the NVRAM is logically part of the disks... The original Preserve had 3 NiCd batteries to backup its low power static CMOS memory, good for about 6 months with no power (it also had a built in charger). don To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 18:27:34 2001 Delivered-To: freebsd-fs@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id B61BE37B718; Tue, 13 Mar 2001 18:27:28 -0800 (PST) (envelope-from mjacob@feral.com) Received: from beppo (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id SAA10845; Tue, 13 Mar 2001 18:25:58 -0800 Date: Tue, 13 Mar 2001 18:25:55 -0800 (PST) From: Matthew Jacob Reply-To: mjacob@feral.com To: Don Coleman Cc: Poul-Henning Kamp , Wilko Bulte , Chris Dillon , scsi@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: <200103132345.PAA28094@eozoon.coleman.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Yeah, he's still around. On Tue, 13 Mar 2001, Don Coleman wrote: > > The original legato prestoserve board (VME bus) was manufactured by > > Micro Memory Inc, > 9540 Vassar Ave > Chatsworth, CA 91311 > 818-998-0070 > > Our contact was Mose' Jadon. The address and phone # are circa 1989. > > Amazingly enough, a web search turns up http://www.micromemory.com, > and the old phone # is still valid. > > The original board was called the MM6704c by Micro Memory. > > A design firm we didn't pick wanted a bit under $100,000 for > a custom engineering design, plus about $1000 per board. > > A PCI board is *much* smaller then a 9U VME board, and I'd > expect the boards to be a lot cheaper. > > I think you'd want at least a couple weeks of backup, since if the > machine crashes due to a hardware failure, it may not come back > up soon, and the NVRAM is logically part of the disks... > > The original Preserve had 3 NiCd batteries to backup its low power > static CMOS memory, good for about 6 months with no power (it also > had a built in charger). > > don > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-scsi" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 21:55:41 2001 Delivered-To: freebsd-fs@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id F129837B71A; Tue, 13 Mar 2001 21:55:35 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id WAA81114; Tue, 13 Mar 2001 22:55:19 -0700 Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp10.phx.gblx.net, id smtpd344mya; Tue Mar 13 22:55:13 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id WAA04683; Tue, 13 Mar 2001 22:55:21 -0700 (MST) From: Terry Lambert Message-Id: <200103140555.WAA04683@usr05.primenet.com> Subject: Re: Disk I/O problem in 4.3-BETA To: phk@critter.freebsd.dk (Poul-Henning Kamp) Date: Wed, 14 Mar 2001 05:55:21 +0000 (GMT) Cc: Stephen.Byan@quantum.com (Stephen Byan), bright@wintelcom.net ('Alfred Perlstein'), mjacob@feral.com (Matthew Jacob), gibbs@scsiguy.com (Justin T. Gibbs), sos@freebsd.dk (Soren Schmidt), oberman@es.net (Kevin Oberman), scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG In-Reply-To: <8821.984501490@critter> from "Poul-Henning Kamp" at Mar 13, 2001 05:38:10 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >If NT and FreeBSD both support hinting of metadata writes, it's only a > >matter of time before the hardware support appears. > > I have been looking for a long time for a PCI NVRAM card at a reasonable > cost, anyone know of any with a reasonable price ? How much NVRAM do you need? It seems to me that you can get a PCI card to plug a PC-CARD into the back of a regular PC at Fry's for pretty cheap, and that there is no lack of memory cards in the PC-CARD/PCMCIA form factor. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 13 21:57: 0 2001 Delivered-To: freebsd-fs@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.136]) by hub.freebsd.org (Postfix) with ESMTP id 86EF937B719; Tue, 13 Mar 2001 21:56:55 -0800 (PST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.9.3/8.9.3) with ESMTP id GAA57433; Wed, 14 Mar 2001 06:56:53 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f2E5vAp16089; Wed, 14 Mar 2001 06:57:10 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Terry Lambert Cc: Stephen.Byan@quantum.com (Stephen Byan), bright@wintelcom.net ('Alfred Perlstein'), mjacob@feral.com (Matthew Jacob), gibbs@scsiguy.com (Justin T. Gibbs), sos@freebsd.dk (Soren Schmidt), oberman@es.net (Kevin Oberman), scsi@FreeBSD.ORG, fs@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA In-Reply-To: Your message of "Wed, 14 Mar 2001 05:55:21 GMT." <200103140555.WAA04683@usr05.primenet.com> Date: Wed, 14 Mar 2001 06:57:10 +0100 Message-ID: <16087.984549430@critter> From: Poul-Henning Kamp Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200103140555.WAA04683@usr05.primenet.com>, Terry Lambert writes: >> >If NT and FreeBSD both support hinting of metadata writes, it's only a >> >matter of time before the hardware support appears. >> >> I have been looking for a long time for a PCI NVRAM card at a reasonable >> cost, anyone know of any with a reasonable price ? > >How much NVRAM do you need? > >It seems to me that you can get a PCI card to plug a PC-CARD >into the back of a regular PC at Fry's for pretty cheap, and >that there is no lack of memory cards in the PC-CARD/PCMCIA >form factor. There you run into a speed problem. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Mar 14 15:43:11 2001 Delivered-To: freebsd-fs@freebsd.org Received: from gate.lustig.com (lustig.ne.mediaone.net [24.91.125.166]) by hub.freebsd.org (Postfix) with SMTP id E95DF37B719 for ; Wed, 14 Mar 2001 15:43:07 -0800 (PST) (envelope-from barry@lustig.com) Received: (qmail 82388 invoked from network); 14 Mar 2001 23:43:06 -0000 Received: from gate.lustig.com (HELO lustig.com) (barry@205.246.2.242) by gate.lustig.com with SMTP; 14 Mar 2001 23:43:06 -0000 Message-ID: <3AB00209.DEE27796@lustig.com> Date: Wed, 14 Mar 2001 18:43:05 -0500 From: Barry Lustig Organization: Barry Lustig & Associates, Inc. X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 Cc: scsi@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Disk I/O problem in 4.3-BETA References: <10186.984516305@critter> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Poul-Henning Kamp wrote: > > My ideal product would be a PCI card with some SRAM, and a holder > for a 9v DURACELL. > Does anyone know what NetApp uses? Their NVRAM board is PCI. barry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Mar 14 23:56:28 2001 Delivered-To: freebsd-fs@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 8E3B137B71A for ; Wed, 14 Mar 2001 23:56:18 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 13120 invoked by uid 1000); 15 Mar 2001 07:55:33 -0000 Date: Thu, 15 Mar 2001 09:55:33 +0200 From: Peter Pentchev To: Kris Kennaway Cc: hackers@FreeBSD.org, fs@FreeBSD.org Subject: Re: httpfs Message-ID: <20010315095533.C12432@ringworld.oblivion.bg> Mail-Followup-To: Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org References: <20010310031515.A8998@mollari.cthul.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010310031515.A8998@mollari.cthul.hu>; from kris@obsecurity.org on Sat, Mar 10, 2001 at 03:15:15AM -0800 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, Mar 10, 2001 at 03:15:15AM -0800, Kris Kennaway wrote: > A few of us were talking on IRC tonight about how cool it would be to > have an httpfs filesystem -- then it occurred to me we almost have > this already, in the form of the (under-utilised) portalfs. Portalfs > works by handing off everything to a userland daemon which handles the > actual transaction request, so you could easily imagine extending it > to provide an http method similar to the tcp method it currently has > for initiating tcp connections. > > One could probably make this more generic and finish implementing the > undocumented 'exec' method (which currently exists as a stub): this > would run an administrator-specified command (i.e. fixed in > /etc/portal.conf) and pipe the output back to the user. > > A fully navigable httpfs (e.g. one you can ls and cd around in) is > more work and probably requires a full stacking layer, but this would > still be pretty cool. > > Is anyone feeling inspired to implement this? :-) OK, as I've mentioned in a private mail to Kris several hours after he sent out this message, I've done some initial hacking on mount_portal which lets me put: http/ exec http/ /usr/bin/fetch fetch -q -o - http://$1- into /etc/portal.conf, and then do cat /p/http/www.FreeBSD.org/handbook/ (the $1- part refers to path components below /p/http/; $1- means 'path components from 1 to last, separated by /') The code still needs a lot of cleanup before I would dare submit it for review and comments; my question is, should I bother^W^W^Wdoes this look like a reasonable extension to mount_portal, or are there other ways/places that such functionality should be implemented, if at all? G'luck, Peter -- If the meanings of 'true' and 'false' were switched, then this sentence wouldn't be false. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 0:45:48 2001 Delivered-To: freebsd-fs@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 813C437B719; Thu, 15 Mar 2001 00:45:45 -0800 (PST) (envelope-from jkh@osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.2/8.11.1) with ESMTP id f2F8iQH09975; Thu, 15 Mar 2001 00:44:26 -0800 (PST) (envelope-from jkh@osd.bsdi.com) To: roam@orbitel.bg Cc: kris@obsecurity.org, hackers@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: httpfs In-Reply-To: <20010315095533.C12432@ringworld.oblivion.bg> References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010315004426A.jkh@osd.bsdi.com> Date: Thu, 15 Mar 2001 00:44:26 -0800 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 4 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'd say it would be an excellent example of how to use portals if nothing else, given that almost nobody understands them. :) - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 2:39:26 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mailgate.Cadence.COM (mailgate.Cadence.COM [158.140.2.1]) by hub.freebsd.org (Postfix) with ESMTP id C6A3137B719; Thu, 15 Mar 2001 02:39:17 -0800 (PST) (envelope-from dmlb@dmlb.org) Received: from symnt3.Cadence.COM (symnt3.Cadence.COM [194.32.101.100]) by mailgate.Cadence.COM (8.9.3/8.9.3) with ESMTP id CAA25326; Thu, 15 Mar 2001 02:38:34 -0800 (PST) Received: from pc610cam (pc610-cam.cadence.com [194.32.96.210]) by symnt3.Cadence.COM with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id GM9QP9LQ; Thu, 15 Mar 2001 10:37:38 -0000 Message-ID: <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> From: "Duncan Barclay" To: "Peter Pentchev" , "Kris Kennaway" Cc: , References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> Subject: Re: httpfs Date: Thu, 15 Mar 2001 10:38:21 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 X-Received: By mailgate.Cadence.COM as CAA25326 at Thu Mar 15 02:38:34 2001 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi A thing to keep in mind about the portal file system is that it designed to provide a means of getting a file handle to an object that could be obtained by a call to open(2). It does not then provide a means of reading/writing etc. to that object. If you take a look at the example portal.conf then you can see how this can be used to open a socket via a pathname. Operations on the socket are then make using write(2) etc. I don't really think that portalfs is the right thing to use to build an httpfs with, but I would like to see how you managed to get your example to work. Are you using stdout to create an anonymous file handle? What happens if two processes concurrently read from /p/http/*? Duncan -- _____________________________________________________________ Duncan Barclay | God smiles upon the little children, dmlb@dmlb.org | the alcoholics, and the permanently stoned. dmlb@freebsd.org| Steven King > On Sat, Mar 10, 2001 at 03:15:15AM -0800, Kris Kennaway wrote: > > A few of us were talking on IRC tonight about how cool it would be to > > have an httpfs filesystem -- then it occurred to me we almost have > > this already, in the form of the (under-utilised) portalfs. Portalfs > > works by handing off everything to a userland daemon which handles the > > actual transaction request, so you could easily imagine extending it > > to provide an http method similar to the tcp method it currently has > > for initiating tcp connections. > > > > One could probably make this more generic and finish implementing the > > undocumented 'exec' method (which currently exists as a stub): this > > would run an administrator-specified command (i.e. fixed in > > /etc/portal.conf) and pipe the output back to the user. > > > > A fully navigable httpfs (e.g. one you can ls and cd around in) is > > more work and probably requires a full stacking layer, but this would > > still be pretty cool. > > > > Is anyone feeling inspired to implement this? :-) > > OK, as I've mentioned in a private mail to Kris several hours after he > sent out this message, I've done some initial hacking on mount_portal > which lets me put: > > http/ exec http/ /usr/bin/fetch fetch -q -o - http://$1- > > into /etc/portal.conf, and then do cat /p/http/www.FreeBSD.org/handbook/ > (the $1- part refers to path components below /p/http/; $1- means > 'path components from 1 to last, separated by /') > > The code still needs a lot of cleanup before I would dare submit it for > review and comments; my question is, should I bother^W^W^Wdoes this look > like a reasonable extension to mount_portal, or are there other ways/places > that such functionality should be implemented, if at all? > > G'luck, > Peter > > -- > If the meanings of 'true' and 'false' were switched, then this sentence wouldn't be false. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 2:43:30 2001 Delivered-To: freebsd-fs@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 606C437B718 for ; Thu, 15 Mar 2001 02:43:25 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 1019 invoked by uid 1000); 15 Mar 2001 10:42:44 -0000 Date: Thu, 15 Mar 2001 12:42:44 +0200 From: Peter Pentchev To: Duncan Barclay Cc: Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org Subject: Re: httpfs Message-ID: <20010315124244.A442@ringworld.oblivion.bg> Mail-Followup-To: Duncan Barclay , Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM>; from dmlb@dmlb.org on Thu, Mar 15, 2001 at 10:38:21AM -0000 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, Mar 15, 2001 at 10:38:21AM -0000, Duncan Barclay wrote: > Hi > > A thing to keep in mind about the portal file system is that it > designed to provide a means of getting a file handle to an object that > could be obtained by a call to open(2). It does not then provide > a means of reading/writing etc. to that object. > > If you take a look at the example portal.conf then you can see how > this can be used to open a socket via a pathname. Operations on the > socket are then make using write(2) etc. > > I don't really think that portalfs is the right thing to use to build > an httpfs with, but I would like to see how you managed to get your example > to work. Are you using stdout to create an anonymous file handle? What happens > if two processes concurrently read from /p/http/*? What I did was implement an 'exec' portal method, which executes a program with given arguments, obtained from the path components and portal.conf rules, and returns a - basically read-only - descriptor connected to its stdout and stderr. Kind of simple, pipe(), fork(), dup2(), exec().. the main trouble was with parsing the argument rules :) I'll clean it up in a few hours, and post it somewhere.. G'luck, Peter -- If there were no counterfactuals, this sentence would not have been paradoxical. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 18:11:11 2001 Delivered-To: freebsd-fs@freebsd.org Received: from mail2.netvision.com.br (nv37.netvision.com.br [200.215.94.37]) by hub.freebsd.org (Postfix) with ESMTP id B342137B719 for ; Thu, 15 Mar 2001 18:11:08 -0800 (PST) (envelope-from andre@netvision.com.br) Received: from nv12 (nv12.netvision.com.br [200.247.217.134]) by mail2.netvision.com.br (Postfix) with SMTP id 96FC11869 for ; Fri, 16 Mar 2001 00:10:32 -0300 (BET) Content-Type: text/plain; charset="iso-8859-1" From: =?iso-8859-1?q?Andr=E9=20Luiz=20dos=20Santos?= Reply-To: andre@netvision.com.br To: fs@freebsd.org Subject: Truncating a file. Date: Thu, 15 Mar 2001 02:20:14 -0500 X-Mailer: KMail [version 1.2] MIME-Version: 1.0 Message-Id: <01031502201400.28129@nv12> Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org With ftruncate, you can remove part of the end of a file. Is there a way to remove part of the beginning of a file? I'm developing a SOCKS5 server that stores the data received from the first connection to a local file, and when the second connection is writable, read that file and write the data to this second connection. As data is read from the local file, its beginning becomes useless, so I'd like to truncate it out. Is it possible? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 18:16:16 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 2686937B71A for ; Thu, 15 Mar 2001 18:16:14 -0800 (PST) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f2G2G1u26317; Thu, 15 Mar 2001 18:16:01 -0800 (PST) Date: Thu, 15 Mar 2001 18:16:01 -0800 From: Alfred Perlstein To: =?iso-8859-1?Q?Andr=E9_Luiz_dos_Santos?= Cc: fs@FreeBSD.ORG Subject: Re: Truncating a file. Message-ID: <20010315181601.O29888@fw.wintelcom.net> References: <01031502201400.28129@nv12> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5i In-Reply-To: <01031502201400.28129@nv12>; from andre@netvision.com.br on Thu, Mar 15, 2001 at 02:20:14AM -0500 X-all-your-base: are belong to us. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * André Luiz dos Santos [010315 18:11] wrote: > > With ftruncate, you can remove part of the end of a file. Is there a way to > remove part of the beginning of a file? > I'm developing a SOCKS5 server that stores the data received from the first > connection to a local file, and when the second connection is writable, read > that file and write the data to this second connection. As data is read from > the local file, its beginning becomes useless, so I'd like to truncate it > out. Is it possible? No. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 15 22: 0:21 2001 Delivered-To: freebsd-fs@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 97E9C37B718 for ; Thu, 15 Mar 2001 22:00:15 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id WAA27841; Thu, 15 Mar 2001 22:54:42 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp05.primenet.com, id smtpdAAAkYaGv2; Thu Mar 15 22:54:38 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id XAA04319; Thu, 15 Mar 2001 23:00:07 -0700 (MST) From: Terry Lambert Message-Id: <200103160600.XAA04319@usr05.primenet.com> Subject: Re: Truncating a file. To: andre@netvision.com.br Date: Fri, 16 Mar 2001 06:00:02 +0000 (GMT) Cc: fs@FreeBSD.ORG In-Reply-To: <01031502201400.28129@nv12> from "=?iso-8859-1?q?Andr=E9=20Luiz=20dos=20Santos?=" at Mar 15, 2001 02:20:14 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > With ftruncate, you can remove part of the end of a file. Is there a way to > remove part of the beginning of a file? > I'm developing a SOCKS5 server that stores the data received from the first > connection to a local file, and when the second connection is writable, read > that file and write the data to this second connection. As data is read from > the local file, its beginning becomes useless, so I'd like to truncate it > out. Is it possible? FreeBSD doesn't support the defacto industry standard F_FREESP fcntl(2) command argument (which would free the area referred to by the contents of a flock structure). It would be trivial to implement support for it, if you wanted to do so, as long as you realize that not all FS types actually support sparse files (which is what you are actually asking to be able to create from a preexisting non-spparse file). -- I'd actually recommend that you divide the file up into an index block region and a data block region. Use the index blocks at the front of the file to find data blocks at the end. This will let you move the data blocks around to get rid of space in the middle when a data block becomes empty, and then truncate the end of the file. Wasted space in an index block is not really a problem, since it can be small (I suggest an off_t for the data block for the index start, plus a bitmap of data blocks allocated contiguously for the number of bits in the bitmap). Don't use [f]truncate(2) if you are using mmap(2), unless you first unmap the region and remap it (incremental munmap(2) is undefined, or was, the last time I looked). You will only be able to truncate on FS block (or fragment) size boundaries, in any case, regardless of what the file size says, since those allocations will be there, whether they are reflected or not... so you ought to use statfs(2) to the the f_bsize parameter, and allocate data blocks in those increments and on that boundary (e.g. start_log % f_bsize == 0). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Mar 16 7:41:19 2001 Delivered-To: freebsd-fs@freebsd.org Received: from hand.dotat.at (inch.demon.co.uk [194.222.223.128]) by hub.freebsd.org (Postfix) with ESMTP id 7294B37B718; Fri, 16 Mar 2001 07:41:15 -0800 (PST) (envelope-from fanf@dotat.at) Received: from fanf by hand.dotat.at with local (Exim 3.20 #3) id 14dn4b-0000AF-00; Fri, 16 Mar 2001 05:46:49 +0000 Date: Fri, 16 Mar 2001 05:46:49 +0000 From: Tony Finch To: Peter Pentchev Cc: Duncan Barclay , Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org Subject: Re: httpfs Message-ID: <20010316054649.F385@hand.dotat.at> References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM>; <20010315124244.A442@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010315124244.A442@ringworld.oblivion.bg> Organization: Covalent Technologies, Inc Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Peter Pentchev wrote: > >What I did was implement an 'exec' portal method, which executes a program >with given arguments, obtained from the path components and portal.conf >rules, and returns a - basically read-only - descriptor connected to its >stdout and stderr. Kind of simple, pipe(), fork(), dup2(), exec().. Nice. Is there any reason not to add some bidirectional support by connecting the descriptor to stdin as well? Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Mar 16 7:45:11 2001 Delivered-To: freebsd-fs@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 477B037B719 for ; Fri, 16 Mar 2001 07:45:07 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 3455 invoked by uid 1000); 16 Mar 2001 15:44:24 -0000 Date: Fri, 16 Mar 2001 17:44:24 +0200 From: Peter Pentchev To: Tony Finch Cc: Duncan Barclay , Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org Subject: Re: httpfs Message-ID: <20010316174424.A428@ringworld.oblivion.bg> Mail-Followup-To: Tony Finch , Duncan Barclay , Kris Kennaway , hackers@FreeBSD.org, fs@FreeBSD.org References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM>; <20010315124244.A442@ringworld.oblivion.bg> <20010316054649.F385@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010316054649.F385@hand.dotat.at>; from dot@dotat.at on Fri, Mar 16, 2001 at 05:46:49AM +0000 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, Mar 16, 2001 at 05:46:49AM +0000, Tony Finch wrote: > Peter Pentchev wrote: > > > >What I did was implement an 'exec' portal method, which executes a program > >with given arguments, obtained from the path components and portal.conf > >rules, and returns a - basically read-only - descriptor connected to its > >stdout and stderr. Kind of simple, pipe(), fork(), dup2(), exec().. > > Nice. Is there any reason not to add some bidirectional support by > connecting the descriptor to stdin as well? There was at the time - socketpair(2) had totally slipped my mind ;) G'luck, Peter -- This sentence contradicts itself - or rather - well, no, actually it doesn't! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 7:53:45 2001 Delivered-To: freebsd-fs@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 8110837B71A; Sat, 17 Mar 2001 07:53:41 -0800 (PST) (envelope-from des@ofug.org) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id QAA73535; Sat, 17 Mar 2001 16:53:35 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Peter Pentchev Cc: Tony Finch , Duncan Barclay , Kris Kennaway , hackers@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: httpfs References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <20010315124244.A442@ringworld.oblivion.bg> <20010316054649.F385@hand.dotat.at> <20010316174424.A428@ringworld.oblivion.bg> From: Dag-Erling Smorgrav Date: 17 Mar 2001 16:53:34 +0100 In-Reply-To: Peter Pentchev's message of "Fri, 16 Mar 2001 17:44:24 +0200" Message-ID: Lines: 8 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Peter Pentchev writes: > There was at the time - socketpair(2) had totally slipped my mind ;) Umm, you want pipe(2), not socketpair(2). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 8: 1:47 2001 Delivered-To: freebsd-fs@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id A857D37B718 for ; Sat, 17 Mar 2001 08:01:42 -0800 (PST) (envelope-from roam@orbitel.bg) Received: (qmail 911 invoked by uid 1000); 17 Mar 2001 16:00:55 -0000 Date: Sat, 17 Mar 2001 18:00:55 +0200 From: Peter Pentchev To: Dag-Erling Smorgrav Cc: Tony Finch , Duncan Barclay , Kris Kennaway , hackers@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: httpfs Message-ID: <20010317180055.A486@ringworld.oblivion.bg> Mail-Followup-To: Dag-Erling Smorgrav , Tony Finch , Duncan Barclay , Kris Kennaway , hackers@FreeBSD.ORG, fs@FreeBSD.ORG References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <20010315124244.A442@ringworld.oblivion.bg> <20010316054649.F385@hand.dotat.at> <20010316174424.A428@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Sat, Mar 17, 2001 at 04:53:34PM +0100 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, Mar 17, 2001 at 04:53:34PM +0100, Dag-Erling Smorgrav wrote: > Peter Pentchev writes: > > There was at the time - socketpair(2) had totally slipped my mind ;) > > Umm, you want pipe(2), not socketpair(2). Actually, I want socketpair(2). pipe(2) was what I used before, and that's the reason I had a read-only file descriptor - the portalfs architecture allows for only one fd to be returned, and pipe(2) provides a one-way pipe. I dup2'd stdout and stderr of the executed program to the child fd, and the parent could read its output, yet not write to its stdin. With socketpair(2), I can dup stdin, too, and have mount_portal return a two-way pipe/fd/socket to whoever requested it. At least, that's the common/standard/easiest way to create a two-way pipe on the same fd, described in APUE :) G'luck, Peter -- This would easier understand fewer had omitted. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 8: 3:52 2001 Delivered-To: freebsd-fs@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 7E84737B719; Sat, 17 Mar 2001 08:03:46 -0800 (PST) (envelope-from des@ofug.org) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA73593; Sat, 17 Mar 2001 17:03:43 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Peter Pentchev Cc: Tony Finch , Duncan Barclay , Kris Kennaway , hackers@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: httpfs References: <20010310031515.A8998@mollari.cthul.hu> <20010315095533.C12432@ringworld.oblivion.bg> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <000d01c0ad3c$0ed83fb0$d26020c2@Cadence.COM> <20010315124244.A442@ringworld.oblivion.bg> <20010316054649.F385@hand.dotat.at> <20010316174424.A428@ringworld.oblivion.bg> <20010317180055.A486@ringworld.oblivion.bg> From: Dag-Erling Smorgrav Date: 17 Mar 2001 17:03:42 +0100 In-Reply-To: Peter Pentchev's message of "Sat, 17 Mar 2001 18:00:55 +0200" Message-ID: Lines: 15 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Peter Pentchev writes: > On Sat, Mar 17, 2001 at 04:53:34PM +0100, Dag-Erling Smorgrav wrote: > > Peter Pentchev writes: > > > There was at the time - socketpair(2) had totally slipped my mind ;) > > Umm, you want pipe(2), not socketpair(2). > Actually, I want socketpair(2). pipe(2) was what I used before, > and that's the reason I had a read-only file descriptor - the portalfs > architecture allows for only one fd to be returned, and pipe(2) > provides a one-way pipe. Not in FreeBSD. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 8:15:25 2001 Delivered-To: freebsd-fs@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id 0C3F837B719; Sat, 17 Mar 2001 08:15:17 -0800 (PST) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.11.3/8.9.3) with ESMTP id f2HGF6i12988; Sun, 18 Mar 2001 02:45:06 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010317180055.A486@ringworld.oblivion.bg> Date: Sun, 18 Mar 2001 02:44:00 +1030 (CST) From: "Daniel O'Connor" To: Peter Pentchev Subject: Re: httpfs Cc: fs@FreeBSD.ORG, hackers@FreeBSD.ORG, Kris Kennaway , Duncan Barclay , Tony Finch , Dag-Erling Smorgrav Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 17-Mar-01 Peter Pentchev wrote: > Actually, I want socketpair(2). pipe(2) was what I used before, > and that's the reason I had a read-only file descriptor - the portalfs > architecture allows for only one fd to be returned, and pipe(2) > provides a one-way pipe. I dup2'd stdout and stderr of the executed > program to the child fd, and the parent could read its output, yet > not write to its stdin. pipe's are bidirectional in FreeBSD.. --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 16: 7:34 2001 Delivered-To: freebsd-fs@freebsd.org Received: from smtp02.teb1.iconnet.net (smtp02.teb1.iconnet.net [209.3.218.43]) by hub.freebsd.org (Postfix) with ESMTP id 8CFED37B718; Sat, 17 Mar 2001 16:07:27 -0800 (PST) (envelope-from babkin@bellatlantic.net) Received: from bellatlantic.net (client-151-198-135-1.nnj.dialup.bellatlantic.net [151.198.135.1]) by smtp02.teb1.iconnet.net (8.9.1/8.9.1) with ESMTP id TAA20966; Sat, 17 Mar 2001 19:07:24 -0500 (EST) Message-ID: <3AB3FC38.94711FFF@bellatlantic.net> Date: Sat, 17 Mar 2001 19:07:20 -0500 From: Sergey Babkin X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 4.0-19990626-CURRENT i386) X-Accept-Language: en, ru MIME-Version: 1.0 To: security@freebsd.org, Wes Peters , Robert Watson , fs@freebsd.org Subject: about common group & user ID space (PR kern/14584) Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org All, I want to commit PR kern/14584. I've been told that it's good to discuss it in -arch, -security and -fs. (It has been sort of discussed on -hackers already, there were not much replies). So I've posted a message on -arch, and now on -security and -fs. I've also discussed this idea shortly with Kirk McKusick at Usenix-2000 at the BSD BOF and he generally liked it and suggested to review further. There is a rather long description in the PR. In short, the idea is that all the IDs above some value get shared by both users and groups. That is, not only two users can't have the same IDs (unless they are just aliases like root and toor) and two groups can't have the same ID, but an user and a group can't have the same ID as well. This allows to use the UID field in the inodes to give permissions in the unified UID&GID space, and thus give two groups (say, "writers" and "readers") different permissions to the file wtihout resorting to trickery with subdirectories. The ID space below this some value is kept separate for UIDs and GIDs for compatibility with the historic IDs. In the patch this feature is enabled by a kernel compilation option, plus even with this option compiled a sysctl has to be set. So it would not affect the unsuspecting users. Why not leave it to the real ACLs ? The problem I see with ACLs is that they break all the standard Unix commands dealing with displaying or storing the permissions, such as ls, tar, cpio and others of this sort. Probably the ACLs are _the_ way to go for the high-security environments. But from my personal experience with systems administration of HP-UX and NetWare in not-so-high-security environments, the careless application of ACLs tends to cause quite a systems administration nighmare. So I personally would avoid them for as long as possible and use only when really neccessary. And that seems to be not only my experience. For example, in UnixWare the ACLs were implemented and then essentially scrapped (never ported to VxFS and left working only as remnants in SFS, a version of FFS with ACLs which does not seem to be used by anyone any more and which may not be used as a root filesystem any more). This is the reason why I think that the classic Unix permissions still have a long live ahead, so some backwards-compatible extensions to them might be quite usable. Any comments are welcome. -SB To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 17 23:38:58 2001 Delivered-To: freebsd-fs@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id BE3BB37B725; Sat, 17 Mar 2001 23:38:51 -0800 (PST) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id AAA96598; Sun, 18 Mar 2001 00:38:34 -0700 Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp10.phx.gblx.net, id smtpdiFiFMa; Sun Mar 18 00:38:26 2001 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id AAA03250; Sun, 18 Mar 2001 00:38:33 -0700 (MST) From: Terry Lambert Message-Id: <200103180738.AAA03250@usr05.primenet.com> Subject: Re: about common group & user ID space (PR kern/14584) To: babkin@bellatlantic.net (Sergey Babkin) Date: Sun, 18 Mar 2001 07:38:31 +0000 (GMT) Cc: security@FreeBSD.ORG, wes@softweyr.com (Wes Peters), rwatson@FreeBSD.ORG (Robert Watson), fs@FreeBSD.ORG In-Reply-To: <3AB3FC38.94711FFF@bellatlantic.net> from "Sergey Babkin" at Mar 17, 2001 07:07:20 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I want to commit PR kern/14584. I've been told that it's good > to discuss it in -arch, -security and -fs. (It has been sort of > discussed on -hackers already, there were not much replies). > So I've posted a message on -arch, and now on -security and -fs. > I've also discussed this idea shortly with Kirk McKusick at > Usenix-2000 at the BSD BOF and he generally liked it and suggested > to review further. You could do this a bit more cleanly by just stealing the sign bit, and setting if the uid field contained a group ID. There would be no conversion problem for an existing system. The sign bit would not be "stolen", unless the sysctl was in the "active" state. This changes the check to a one line change, conditional on the high bit being set. In trade, the "set group owner" code gets a bit more complicated, but that's in the user space "chown" code, where you have to tell it to set a group, explicitly (so that it will look up the group, not the user, for a non-numeric ID, and set the high bit when stuffing it in the chown id field). Note that this change is really necessary in the user space code anyway: even if you make the UID and GID numeric values not intersect, there is still the possibility of a group and user having the same name, so a set-by-name needs a seperate flag (thing "chown bin.bin foo", for example). The benefits in not having the grovel through the FS contents, or do a more complex ID space transformations, and the moving of the majority of changes to user space, combined with the fact that if you turn it off, the ownership doesn't need to be reverted, are all plusses. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message