From owner-freebsd-fs Sun May 5 1:48:24 2002 Delivered-To: freebsd-fs@freebsd.org Received: from quic.net (romulus.quic.net [216.23.27.8]) by hub.freebsd.org (Postfix) with SMTP id 48B5737B404 for ; Sun, 5 May 2002 01:48:21 -0700 (PDT) Received: (qmail 19835 invoked by uid 1032); 5 May 2002 08:48:27 -0000 From: utsl@quic.net Date: Sun, 5 May 2002 04:48:27 -0400 To: Terry Lambert Cc: Bakul Shah , Scott Hess , "Vladimir B. Grebenschikov" , fs@FreeBSD.ORG Subject: Re: Filesystem Message-ID: <20020505084827.GA3688@quic.net> References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3CD3FB02.3EC1DA29@mindspring.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Sat, May 04, 2002 at 08:15:14AM -0700, Terry Lambert wrote: > utsl@quic.net wrote: > > [ ... linear directory search times on the majority of systems ... ] > I wasn't really trying to exhasutively list all the reasons that > it was bad to put a bunch of files in a large directory. There > are an incredibly large number of reasons for it to be bad, and > I have better things to do than spending the rest of time pointing > out impedence mismatches in algorithms. 8-). Yup. I could think of quite a few more, myself. For most people, one or two should be sufficient. > My take on an application that doesn't scale is that "fixing" the > application by changing the behaviour of the underlying system is > just propping up bad code. Bad code deserves to lose. So if > someone wrote an application like that, it's just as well that the > programmer who failed to consider scaling issues lose out to the > programmer who considered them. After all, it's very likely that > the failure to consider scaling issues is more of an "all or nothing" > thing, and that the failure to consider one means that solving it in > the OS will just expose the next one. There's really no way you > can make the OS behave perfectly for all applications. At some > point, applications programmers will have to learn how to program, > or all bets are off. Yes. Most people that supported the application I described would have liked to catch the application programmers in a dark alley. People who put 100,000 files in a single directory deserve what happens to them, IMHO. However, it is nice to have the tools to do things right. Given how common this particulary problem seems to be, I think a library might be a good idea. I may write one, I'm working on an application now that needs to be able to scale to at least 1M files. :( ---Nathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 6:24:13 2002 Delivered-To: freebsd-fs@freebsd.org Received: from bilver.wjv.com (spdsl-033.wanlogistics.net [63.209.115.33]) by hub.freebsd.org (Postfix) with ESMTP id 9055837B404 for ; Sun, 5 May 2002 06:24:09 -0700 (PDT) Received: (from bv@localhost) by bilver.wjv.com (8.11.6/8.11.6) id g45DO8B34352 for freebsd-fs@freebsd.org; Sun, 5 May 2002 09:24:08 -0400 (EDT) (envelope-from bv) Date: Sun, 5 May 2002 09:24:08 -0400 From: Bill Vermillion To: freebsd-fs@freebsd.org Subject: Re: Filesystem Message-ID: <20020505132408.GA34282@wjv.com> Reply-To: bv@wjv.com References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020505084827.GA3688@quic.net> User-Agent: Mutt/1.3.25i Organization: W.J.Vermillion / Orlando - Winter Park Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Putting quill to paper and scribbling furiously on Sun, May 05, 2002 at 04:48 , utsl@quic.net missed achieving immortality when he said: > On Sat, May 04, 2002 at 08:15:14AM -0700, Terry Lambert wrote: > > utsl@quic.net wrote: > > My take on an application that doesn't scale is that "fixing" > > the application by changing the behaviour of the underlying > > system is just propping up bad code. Bad code deserves to > > lose. So if someone wrote an application like that, it's just > > as well that the programmer who failed to consider scaling > > issues lose out to the programmer who considered them. After > > all, it's very likely that the failure to consider scaling > > issues is more of an "all or nothing" thing, and that the > > failure to consider one means that solving it in the OS will > > just expose the next one. There's really no way you can make > > the OS behave perfectly for all applications. At some point, > > applications programmers will have to learn how to program, > > or all bets are off. > Yes. Most people that supported the application I described would have > liked to catch the application programmers in a dark alley. People who > put 100,000 files in a single directory deserve what happens to them, > IMHO. Like the old dumb blonde joke about the secretary filing all the letters under L. Bill -- Bill Vermillion - bv @ wjv . com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 7:40:11 2002 Delivered-To: freebsd-fs@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id D850A37B404 for ; Sun, 5 May 2002 07:40:08 -0700 (PDT) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020505144008.XPNH5896.rwcrmhc53.attbi.com@InterJet.elischer.org>; Sun, 5 May 2002 14:40:08 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id HAA92539; Sun, 5 May 2002 07:32:54 -0700 (PDT) Date: Sun, 5 May 2002 07:32:53 -0700 (PDT) From: Julian Elischer To: utsl@quic.net Cc: Terry Lambert , Bakul Shah , Scott Hess , "Vladimir B. Grebenschikov" , fs@FreeBSD.ORG Subject: Re: Filesystem In-Reply-To: <20020505084827.GA3688@quic.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Sun, 5 May 2002 utsl@quic.net wrote: > On Sat, May 04, 2002 at 08:15:14AM -0700, Terry Lambert wrote: > > However, it is nice to have the tools to do things right. Given how > common this particulary problem seems to be, I think a library might be > a good idea. I may write one, I'm working on an application now that > needs to be able to scale to at least 1M files. :( > consider using a database for indexing and using ifs to store the data That is what it is for.. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 8:38:20 2002 Delivered-To: freebsd-fs@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 973F937B405 for ; Sun, 5 May 2002 08:38:18 -0700 (PDT) Received: from pool0053.cvx21-bradley.dialup.earthlink.net ([209.179.192.53] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 174O5R-0004yk-00; Sun, 05 May 2002 08:38:09 -0700 Message-ID: <3CD551C3.BE1BD549@mindspring.com> Date: Sun, 05 May 2002 08:37:39 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: utsl@quic.net, Bakul Shah , Scott Hess , "Vladimir B. Grebenschikov" , fs@FreeBSD.ORG Subject: Re: Filesystem References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Julian Elischer wrote: > On Sun, 5 May 2002 utsl@quic.net wrote: > > However, it is nice to have the tools to do things right. Given how > > common this particulary problem seems to be, I think a library might be > > a good idea. I may write one, I'm working on an application now that > > needs to be able to scale to at least 1M files. :( > > > > consider using a database for indexing and using ifs to store the data > That is what it is for.. Careful with attribution. You left in one too many "xxx wrote:"'s. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 10:40: 6 2002 Delivered-To: freebsd-fs@freebsd.org Received: from nitrogen.wanadoo.fr (ca-ol-sqy-21-146.abo.wanadoo.fr [80.8.58.146]) by hub.freebsd.org (Postfix) with ESMTP id F34F537B404 for ; Sun, 5 May 2002 10:39:57 -0700 (PDT) Received: from nitrogen.wanadoo.fr (nitrogen [127.0.0.1]) by nitrogen.wanadoo.fr (8.12.3/8.12.3) with ESMTP id g45HctfV000545 for ; Sun, 5 May 2002 19:38:55 +0200 (CEST) (envelope-from dak@nitrogen.wanadoo.fr) Received: (from dak@localhost) by nitrogen.wanadoo.fr (8.12.3/8.12.3/Submit) id g45HctLu000544 for freebsd-fs@freebsd.org; Sun, 5 May 2002 19:38:55 +0200 (CEST) Date: Sun, 5 May 2002 19:38:55 +0200 From: dak To: freebsd-fs@freebsd.org Subject: Implementing a new FS with loadable modules Message-ID: <20020505173855.GA528@nitrogen.WorkGroup> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.99i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, I'm making my own file system (just for fun and to understand how it works) and I want to implement it in a module. Can anybody points me to a skeleton code which regroups code to mount and everything the system needs to 'understand' my module ? I've already looked at 'ntfs' module and others but I don't realy understand the way to follow to make a module that works :/ Thanks in advance. -- dak To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 11:25:24 2002 Delivered-To: freebsd-fs@freebsd.org Received: from juno.com (ntserver.sosuo.cz [193.179.195.131]) by hub.freebsd.org (Postfix) with SMTP id 156A837B407; Sun, 5 May 2002 11:24:15 -0700 (PDT) Received: from unknown (HELO rly-yk04.mx.aol.com) (74.9.38.143) by mx.rootsystems.net with QMQP; 03 Jan 2000 12:11:40 -0500 Reply-To: Message-ID: <023b46e82d4c$2821b8b1$1ba31be2@wdtulj> From: To: Cc: , , Subject: Would You Like To Look & Feel 10-20 Years Younger?? 3237l4 Date: Mon, 03 Jan 2000 11:02:32 -0400 MiME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_00B7_16C07C6E.A3114E54" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2615.200 Importance: Normal Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ------=_NextPart_000_00B7_16C07C6E.A3114E54 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: base64 IFdvdWxkIFlvdSBMaWtlIFRvIExvb2sgJiBGZWVsIDEwLTIwIFllYXJzIFlv dW5nZXI/Pw0KDQpXb3VsZCB5b3UgbGlrZSB0byBpbmNyZWFzZSBNdXNjbGUg U3RyZW5ndGggYnkgODglICYNCnJlZHVjZSBCb2R5IEZhdCBieSA3MiUNCi0g LS1XSVRIT1VUIEVYRVJDSVNFIT8hDQoNCkhvdyBhYm91dCBpbmNyZWFzaW5n IGVuZXJneSBsZXZlbHMgYnkgODQlID8NCk9yIEluY3JlYXNpbmcgU2V4dWFs IFBvdGVuY3kgJiBGcmVxdWVuY3kgYnkgNzUlID8NCg0KQUxMIE9GIFRISVMg SVMgTk9XIFBPU1NJQkxFOiBXZSBvZmZlciB0aGUgTW9zdCBQb3RlbnQNCk9y YWwgR0ggRm9ybXVsYSBhdmFpbGFibGUtLWJhY2tlZCB1cCBieSA3IHllYXJz IG9mIHJlc2VhcmNoDQogLS10byBoZWxwIHlvdSBhY2hpZXZlIGFsbCB0aGlz ICYgbW9yZSENCiANClNUQVJUIFJFVkVSU0lORyBUSEUgQUdJTkcgUFJPQ0VT UyBUT0RBWSENCg0KSW4gdGhvdXNhbmRzIG9mIGNsaW5pY2FsIHN0dWRpZXMg KHdpdGggbm8gc2lkZQ0KZWZmZWN0cyksIEdIIGhhcyBiZWVuIHNob3duDQog dG8gYWNjb21wbGlzaCB0aGUgZm9sbG93aW5nOg0KDQogKiBSZWR1Y2UgYm9k eSBmYXQgJiBidWlsZCBsZWFuIG11c2NsZSB3aXRob3V0IGV4ZXJjaXNlIQ0K ICogRW5oYW5jZSBzZXh1YWwgcGVyZm9ybWFuY2UNCiAqIFJlbW92ZSB3cmlu a2xlcyBhbmQgY2VsbHVsaXRlDQogKiBMb3dlciBibG9vZCBwcmVzc3VyZSBh bmQgaW1wcm92ZSBjaG9sZXN0ZXJvbCBwcm9maWxlDQogKiBJbXByb3ZlIHNs ZWVwLCB2aXNpb24gYW5kIG1lbW9yeQ0KICogUmVzdG9yZSBoYWlyIGNvbG9y IGFuZCBncm93dGgNCiAqIFN0cmVuZ3RoZW4gdGhlIGltbXVuZSBzeXN0ZW0N CiAqIEluY3JlYXNlIGVuZXJneSBhbmQgY2FyZGlhYyBvdXRwdXQNCiAqIFR1 cm4gYmFjayB5b3VyIGJvZHkncyBiaW9sb2dpY2FsIHRpbWUgY2xvY2sgMTAt MjANCiAgIHllYXJzIGluIDYgbW9udGhzIHVzZSAhIQ0KIA0KIEZvciBtb3Jl IEZSRUUgSU5GT1JNQVRJT04gb3IgdG8gT1JERVIgUFJPRFVDVCwgcGxlYXNl DQogdmlzaXQgb3VyIHdlYiBzaXRlIGJ5IGNsaWNraW5nIG9uIHRoaXMgbnVt YmVyZWQgbGluazoNCiBodHRwOi8vNjYuMTA3LjEwNy42DQogDQogb3IgQ0FM TCBvdXIgMjQgSFIgVm9pY2VtYWlsIHdpdGggeW91ciBuYW1lLCBudW1iZXIg Jg0KIHRoZSBiZXN0IHRpbWVzIGZvciB1cyB0byBjYWxsIHlvdTogKDg4OCkg NjI0LTk4NTINCiBUaGFuayB5b3UhDQogDQogV2hvbGVzYWxlIElucXVpcmll cyBhcmUgYWxzbyBXRUxDT01FIChXZSBhcmUgbm90IE1MTSkuDQogDQoNCnRv IGJlIHJlbW92ZWQgZnJvbSBvdXIgc3Vic2NyaWJlciBsaXN0IG1haWx0bzpy ZW1tZTYyOThAeWFob28uY29tP3N1YmplY3Q9cmVtb3ZlIA0KIHRoYW5rIHlv dQ0KDQoNCjY1MzRhcVp2MS0xMDF4SURsMDEzOVJvalcwLTYwMWp6VlU1Mjc1 V01WajQtNjg4T0xsNDYNCg== To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 12: 6:10 2002 Delivered-To: freebsd-fs@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 0C64037B401 for ; Sun, 5 May 2002 12:06:01 -0700 (PDT) Received: from dialup-209.245.138.39.dial1.sanjose1.level3.net ([209.245.138.39] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 174RKP-0002fP-00; Sun, 05 May 2002 12:05:49 -0700 Message-ID: <3CD58270.DF9B8F06@mindspring.com> Date: Sun, 05 May 2002 12:05:20 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: dak Cc: freebsd-fs@freebsd.org Subject: Re: Implementing a new FS with loadable modules References: <20020505173855.GA528@nitrogen.WorkGroup> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org dak wrote: > I'm making my own file system (just for fun and to understand how it works) > and I want to implement it in a module. > > Can anybody points me to a skeleton code which regroups code to mount and > everything the system needs to 'understand' my module ? > > I've already looked at 'ntfs' module and others but I don't realy understand > the way to follow to make a module that works :/ There is no such thing as "skeleton code" for a working FS. The FreeBSD VFS layer doesn't really support the concept of skeletonizing FS code very well, and has actually evloved further from that goal than it was originally, imeediately after 4.4.-Lite and 4.4.-Lite2. In simple terms, there are three types of VFS implementations: 1) Local media FS Examples: FFS, NTFS, EXT2FS This implementation writes data to local storage, and retrieves it from local storage. In most cases, this local storage is a disk drive. To write this type of FS, you will need a good understanding of the VM system. The simplest implementation is probably the MS-DOS FS. This is because it does not address the NFS export, the root mount, or other important issues, and it does not address a lot of the UNIX permissions and other access semantics, including the idea of file ownership, etc. (all of which are not supported by the on disk structure of this FS). 2) Stacking VFS layer Examples: nullfs; there are no other working examples. This implementation calls down to the underlying layer. A stacking VFS layer has the same interface on the bottom that it has on the top. Rather than being a consumer of the VM services, it's a consumer of the VFS interface. Because there are cache coherency issues when you attempt to do anything with the data, since each vnode has a pointer to a vm_object_t, if you do anything more complicated than a pass through layer (e.g. "nullfs"), then you will need a good understanding of the VM system. The simplest possible fully functional VFS stacking layer implementation is nullfs; anythiing more complicates could require some changes the the FreeBSD VFS interface itself, to actually address, rahter than work around, the coherency issues. In particular, this would be required of any VFS that wanted to manipulate data, rather than passing it through without any modification. This effectively limits stacking layers to semantic enforcement. 3) Pseudo VFS layer Examples: procfs, devfs, specfs This implementation does not consume the lower layer, and it is not (normally) a consumer of the VM system. Operations on it are handled within the VFS itself. Because it does not have dependencies on more complex interfaces (unless the implementation demands it), this is the simplest VFS layer. #3 is easiest to write, but least useful. A minimally full functional FS is a complicated thing to write. It implements a "struct vfsops" , which contains pointers to the functions that operate on file systems. It exports it both as a module interface list, and as a linker set entry, so that it can be automatically aggregated into the list of system file system types when statically linked into the kernel. These are generally called "VFSOPs". It also implements a "struct vnodeopv_desc", which contains a pointer to the descriptor list "struct vnodeopv_entry_desc", which contains pointers to the functions that operate on objects within a single instance of a file system (files and directories). If the FS supports FIFOs and special device nodes as file system objects, then it will also implement a "struct vnodeopv_desc" for each of these. In effect, the specfs and fifofs are implicitly stacked already, rather than being explicitly stacked (see /sys/ufs/ffs/ffs_vnops.c). This works because the stacking layers *replace*, rather than *augment and pass through* in this case. In other words, they are a tiny subset of what you are supposed to be able to do with stacking layers. The operations that are defined here are generally called "VOPs" (or "VNOPs", if you have an SVR4 background). The nullfs is more than "the minimum possible FS"; it implements all of the necessary VFSOPS and VOPS to be able to pass through all requests to an underlying layer. A "minimum possible FS" would implement only the VFSOPs necessary to support mount and unmount (not FS export, rout mount, file handle conversion, or extended attribute definition, etc. ...the VFSOPs have become overly complicated recently, actually). It would also implement only VOPs necessary for minimum mount related directory operations... thus allowing you to mount and unmount the FS, cd into the root (since the same operations are required to support access to the root vnode of the FS for the mount), but nothing else. A "somewhat more than minimum possible FS" would implement directory elelment lookup and iteration operations. You would be able to "cd .." to get out of the mounted FS, without having to cd "/", since the ".." relative path would then exist (and reference the parent directory). You would also be able to "ls -la" (to see the "." and ".." entries), and you would be able to stat the "." inode to see its size, etc.. But you would not be able to "cat .", as would would on another FS, since the read/write/open/close on files themselves would not exist. Really, FSs are complicated things. They touch on many of the services exported by the operating system. It's important to understand these services before you go off and try to write an FS. You will be lucky to get off with implementing 30 functions, plus the supporting data structures to make these functions visible to the OS. Also, last time I looked, FFS consumes 148 kernel interfaces -- different functions -- to get its work done (you can "look", too: compile a kernel, then go into the kernel build directory, and link all the ufs_*.o and ffs_*.o files together, and count the number of unique unresolved externals). I *would* suggest that you look at FreeBSD's DDI/DKI (device driver interface, device/kernel interface) documentation, but the DDI and DKI change quickly enough that they are very poorly documented. It would be incredibly beneficial if they were designed based on some foresight into future use, *then left the heck alone*, so they could be adequately documented. Basically, this means that you need to be pretty good at reading source code and chasing a moving target, if you want to be able to write an FS and have it work when you are done. Most people start with an existing FS similar to their target FS, and then modify the code. Not a good example of the use of "first principles", I know. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 20: 6:38 2002 Delivered-To: freebsd-fs@freebsd.org Received: from smtp2.mivlmd.cablespeed.com (smtp2.mivlmd.cablespeed.com [216.45.64.147]) by hub.freebsd.org (Postfix) with SMTP id 6B84037B40A for ; Sun, 5 May 2002 20:06:35 -0700 (PDT) Received: (qmail 18092 invoked by uid 0); 6 May 2002 03:06:06 -0000 Received: from unknown (HELO cablespeed.com) (216.45.72.227) by 0 with SMTP; 6 May 2002 03:06:06 -0000 Message-ID: <3CD5F31D.F30573F1@cablespeed.com> Date: Sun, 05 May 2002 23:06:05 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: dak Cc: freebsd-fs@freebsd.org Subject: Re: Implementing a new FS with loadable modules References: <20020505173855.GA528@nitrogen.WorkGroup> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org dak wrote: > > Hi, > > I'm making my own file system (just for fun and to understand how it works) and I want to implement it > in a module. > > Can anybody points me to a skeleton code which regroups code to mount and everything the system needs > to 'understand' my module ? > I've already looked at 'ntfs' module and others but I don't realy understand the way to follow to make > a module that works :/ > > Thanks in advance. > > -- dak > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message I don't remember there being anything specific about making a module vs. statically linking to the kernel. I have the following in my ODS2 file system implementation: VFS_SET(ods2_vfsops, ods2, VFCF_READONLY); And the Makefile: KMOD= ods2 .include And I get an ods2.ko which can be kldload'ed either manually from the command line or from a "mount_ods2" mount utility. If you like, I can send you the source for ODS-2. I haven't tried it recently, but it did work under 3.x, 4.x upto 4.5. I would agree with what Terry Lambert said about no "skeleton" file system - any file system you would do involves more of the on-disk structure management than anything else - there's very little to skeleton-ize. Chuck McCrobie -- -- ­ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 20:13:44 2002 Delivered-To: freebsd-fs@freebsd.org Received: from smtp2.mivlmd.cablespeed.com (smtp2.mivlmd.cablespeed.com [216.45.64.147]) by hub.freebsd.org (Postfix) with SMTP id 3A43837B404 for ; Sun, 5 May 2002 20:13:41 -0700 (PDT) Received: (qmail 18666 invoked by uid 0); 6 May 2002 03:13:18 -0000 Received: from unknown (HELO cablespeed.com) (216.45.72.227) by 0 with SMTP; 6 May 2002 03:13:18 -0000 Message-ID: <3CD5F4CD.2F394EFE@cablespeed.com> Date: Sun, 05 May 2002 23:13:17 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: New File system to commit to FreeBSD Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I have had an OpenVMS ODS-2 read-only file system on my disk for about three years now. I've finally put it into style(9) (ugh!), and would like to get it into the FreeBSD source tree. 1. How does one commit the code? 2. What version of FreeBSD should a new file system support (it currently runs under 3.x and 4.x). 3. OpenVMS and ODs-2 keep record attributes and embedded meta-data in the regular file data. I've been writing some utility programs that understand the meta-data, like "ods2_cat" and "ods2_cp". Also, some work has progressed on an RMS library. Should and how are such things committed? Thank you, Chuck McCrobie mccrobie@cablespeed.com -- ­ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun May 5 21: 4:41 2002 Delivered-To: freebsd-fs@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id B51FF37B400 for ; Sun, 5 May 2002 21:04:38 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id 90D9BAE1C6; Sun, 5 May 2002 21:04:38 -0700 (PDT) Date: Sun, 5 May 2002 21:04:38 -0700 From: Alfred Perlstein To: Chuck McCrobie Cc: freebsd-fs@freebsd.org Subject: Re: New File system to commit to FreeBSD Message-ID: <20020506040438.GZ36741@elvis.mu.org> References: <3CD5F4CD.2F394EFE@cablespeed.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3CD5F4CD.2F394EFE@cablespeed.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Chuck McCrobie [020505 20:13] wrote: > I have had an OpenVMS ODS-2 read-only file system on my disk for about > three years now. I've finally put it into style(9) (ugh!), and would > like to get it into the FreeBSD source tree. > > 1. How does one commit the code? Submit a PR. Toss a message about it here or to -hackers with a URL. > > 2. What version of FreeBSD should a new file system support (it > currently runs under 3.x and 4.x). 5.x. > > 3. OpenVMS and ODs-2 keep record attributes and embedded meta-data in > the regular file data. I've been writing some utility programs that > understand the meta-data, like "ods2_cat" and "ods2_cp". Also, some > work has progressed on an RMS library. Should and how are such things > committed? This sounds like quite a suite of utilities, my _guess_ is that the FS code could/would be committed to FreeBSD, but the utilities to manipulate the FS (ods2_cat, ods2_cp) would go into a port. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 1:18:44 2002 Delivered-To: freebsd-fs@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 1D27C37B406 for ; Mon, 6 May 2002 01:18:41 -0700 (PDT) Received: from pool0126.cvx22-bradley.dialup.earthlink.net ([209.179.198.126] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #2) id 174dhf-00048S-00; Mon, 06 May 2002 01:18:39 -0700 Message-ID: <3CD63C3F.2F4E8F02@mindspring.com> Date: Mon, 06 May 2002 01:18:07 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Alfred Perlstein Cc: Chuck McCrobie , freebsd-fs@freebsd.org Subject: Re: New File system to commit to FreeBSD References: <3CD5F4CD.2F394EFE@cablespeed.com> <20020506040438.GZ36741@elvis.mu.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alfred Perlstein wrote: > * Chuck McCrobie [020505 20:13] wrote: > > 3. OpenVMS and ODs-2 keep record attributes and embedded meta-data in > > the regular file data. I've been writing some utility programs that > > understand the meta-data, like "ods2_cat" and "ods2_cp". Also, some > > work has progressed on an RMS library. Should and how are such things > > committed? > > This sounds like quite a suite of utilities, my _guess_ is that > the FS code could/would be committed to FreeBSD, but the utilities > to manipulate the FS (ods2_cat, ods2_cp) would go into a port. You could either hack into the namespace, or you could provide an fcntl() that gets passed down to the FS for an open descriptor, to turn these utilities tiny, and incorporate most of the code into the FS itself (you are much better off doing this, if you can, since it means that the facilities won't be "lost" into a port for which FTP services will have to be maintained in perpetuity). Right now, since FreeBSD doesn't propagate namespace selection information from the POSIX namespsace escape down properly, you would have to hack it to operate on a per component basis (not as bad as you might imagine, but still an annoying thing to have to do), so for my money, an fcntl() would be the best approach. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 1:21:31 2002 Delivered-To: freebsd-fs@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 6738B37B406 for ; Mon, 6 May 2002 01:21:25 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id 3C19BAE1D4; Mon, 6 May 2002 01:21:25 -0700 (PDT) Date: Mon, 6 May 2002 01:21:25 -0700 From: Alfred Perlstein To: Terry Lambert Cc: Chuck McCrobie , freebsd-fs@freebsd.org Subject: Re: New File system to commit to FreeBSD Message-ID: <20020506082125.GC36741@elvis.mu.org> References: <3CD5F4CD.2F394EFE@cablespeed.com> <20020506040438.GZ36741@elvis.mu.org> <3CD63C3F.2F4E8F02@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3CD63C3F.2F4E8F02@mindspring.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Terry Lambert [020506 01:18] wrote: > Alfred Perlstein wrote: > > * Chuck McCrobie [020505 20:13] wrote: > > > 3. OpenVMS and ODs-2 keep record attributes and embedded meta-data in > > > the regular file data. I've been writing some utility programs that > > > understand the meta-data, like "ods2_cat" and "ods2_cp". Also, some > > > work has progressed on an RMS library. Should and how are such things > > > committed? > > > > This sounds like quite a suite of utilities, my _guess_ is that > > the FS code could/would be committed to FreeBSD, but the utilities > > to manipulate the FS (ods2_cat, ods2_cp) would go into a port. > > You could either hack into the namespace, or you could provide > an fcntl() that gets passed down to the FS for an open descriptor, > to turn these utilities tiny, and incorporate most of the code into > the FS itself (you are much better off doing this, if you can, since > it means that the facilities won't be "lost" into a port for which > FTP services will have to be maintained in perpetuity). > > Right now, since FreeBSD doesn't propagate namespace selection > information from the POSIX namespsace escape down properly, you > would have to hack it to operate on a per component basis (not > as bad as you might imagine, but still an annoying thing to have > to do), so for my money, an fcntl() would be the best approach. Yeah, I find it really odd that one would expose the internals of the actual meta-data to applications. ...wandering off topic... Is that how VMS does it (in the kernel)? Or is there some userland library that you _must use_ to access files that does this sort of thing for you? Or does each userspace application have to do magic to parse meta-data? -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 8:40:30 2002 Delivered-To: freebsd-fs@freebsd.org Received: from sluis.cs.vu.nl (sluis.cs.vu.nl [130.37.192.174]) by hub.freebsd.org (Postfix) with ESMTP id 5D93337B408 for ; Mon, 6 May 2002 08:40:16 -0700 (PDT) Received: from centaur.cs.vu.nl (centaur.cs.vu.nl [192.35.191.4]) by sluis.cs.vu.nl with esmtp (Smail #74) id m174kb0-000OesC; Mon, 6 May 2002 17:40 +0200 Received: from cs.vu.nl (localhost [127.0.0.1]) by centaur.cs.vu.nl with esmtp (Smail #74) id m174kb0-0014NkC; Mon, 6 May 2002 17:40 +0200 Message-Id: To: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> In-reply-to: Your message of "Sun, 5 May 2002 04:48:27 -0400 ." <20020505084827.GA3688@quic.net> Date: Mon, 06 May 2002 17:40:14 +0200 From: Philip Homburg Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In your letter dated Sun, 5 May 2002 04:48:27 -0400 utsl@quic.net wrote: >On Sat, May 04, 2002 at 08:15:14AM -0700, Terry Lambert wrote: >> My take on an application that doesn't scale is that "fixing" the >> application by changing the behaviour of the underlying system is >> just propping up bad code. Bad code deserves to lose. So if >> someone wrote an application like that, it's just as well that the >> programmer who failed to consider scaling issues lose out to the >> programmer who considered them. After all, it's very likely that >> the failure to consider scaling issues is more of an "all or nothing" >> thing, and that the failure to consider one means that solving it in >> the OS will just expose the next one. There's really no way you >> can make the OS behave perfectly for all applications. At some >> point, applications programmers will have to learn how to program, >> or all bets are off. > >Yes. Most people that supported the application I described would have >liked to catch the application programmers in a dark alley. People who >put 100,000 files in a single directory deserve what happens to them, >IMHO. > >However, it is nice to have the tools to do things right. Given how >common this particulary problem seems to be, I think a library might be >a good idea. I may write one, I'm working on an application now that >needs to be able to scale to at least 1M files. :( From an O.S. point of view, what's the problem with providing large directories? Some tools, such as ls and find, may need some attention due to scaling issues. Backup software is needed to handle the new filesystem layout. If somebody would like to create 1M files, and you know that, when properly implemented, directories accesses cost O(log(n)), then what's the problem? I know from experience that implementing a directory as a btree is trivial if lower layers of the filesystem have (rudimentary) transaction support. I don't want to think about crash recovery of btrees in a normal FFS filesystem. Philip Homburg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 10: 3:20 2002 Delivered-To: freebsd-fs@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id 0411B37B406 for ; Mon, 6 May 2002 10:03:15 -0700 (PDT) Received: from pool0013.cvx22-bradley.dialup.earthlink.net ([209.179.198.13] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 174lsv-00059v-00; Mon, 06 May 2002 10:02:50 -0700 Message-ID: <3CD6B71C.C0A502A1@mindspring.com> Date: Mon, 06 May 2002 10:02:20 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Philip Homburg Cc: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Philip Homburg wrote: > >From an O.S. point of view, what's the problem with providing large > directories? Some tools, such as ls and find, may need some attention due to > scaling issues. Backup software is needed to handle the new filesystem > layout. #1 Binary backwards compatability with millions of installed systems > If somebody would like to create 1M files, and you know that, when properly > implemented, directories accesses cost O(log(n)), then what's the problem? #1 No one has written the code to optionally store directories this way (and made it publically available) #2 (minor nit) A btree is O(log2(N)), which is not as happy a number as your O(log(N)) > I know from experience that implementing a directory as a btree is trivial > if lower layers of the filesystem have (rudimentary) transaction support. #1 Bell the cat > I don't want to think about crash recovery of btrees in a normal FFS > filesystem. No one does, except the people asking us to switch to their pet FS by default, but not releasing it under a usable license, and expecting that we would just "eat" the backward compatability issue. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 14:46:16 2002 Delivered-To: freebsd-fs@freebsd.org Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by hub.freebsd.org (Postfix) with ESMTP id 12C2937B401 for ; Mon, 6 May 2002 14:46:02 -0700 (PDT) Received: from fwd00.sul.t-online.de by mailout09.sul.t-online.com with smtp id 174qIw-0000cs-01; Mon, 06 May 2002 23:45:58 +0200 Received: from pc5.abc (520067998749-0001@[217.233.125.12]) by fmrl00.sul.t-online.com with esmtp id 174qIj-25lQvoC; Mon, 6 May 2002 23:45:45 +0200 Received: (from nicolas@localhost) by pc5.abc (8.11.6/8.11.6) id g46Ljiv03152 for fs@FreeBSD.ORG; Mon, 6 May 2002 23:45:44 +0200 (CEST) (envelope-from list@rachinsky.de) Date: Mon, 6 May 2002 23:45:44 +0200 From: Nicolas Rachinsky To: fs@FreeBSD.ORG Subject: Re: Filesystem Message-ID: <20020506214544.GD2891@pc5.abc> Mail-Followup-To: fs@FreeBSD.ORG References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> <3CD6B71C.C0A502A1@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3CD6B71C.C0A502A1@mindspring.com> User-Agent: Mutt/1.3.28i X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: C11ABC0E X-PGP-Fingerprint: 19DB 8392 8FE0 814A 7362 EEBD A53B 526A C11A BC0E X-PGP-Key: http://www.rachinsky.de/nicolas/nicolas_rachinsky.asc X-Sender: 520067998749-0001@t-dialin.net Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Terry Lambert [2002-05-06 10:02:20 -0700]: > #2 (minor nit) A btree is O(log2(N)), which is not as happy a > number as your O(log(N)) log2(N)=log(N)/log(2) 1/log(2) is just an constant coefficient, which can be ignored. Nicolas To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 15:18:20 2002 Delivered-To: freebsd-fs@freebsd.org Received: from sluis.cs.vu.nl (sluis.cs.vu.nl [130.37.192.174]) by hub.freebsd.org (Postfix) with ESMTP id C42A637B400 for ; Mon, 6 May 2002 15:18:15 -0700 (PDT) Received: from cs.vu.nl (localhost [127.0.0.1]) by sluis.cs.vu.nl with esmtp (Smail #74) id m174qoA-000OesC; Tue, 7 May 2002 00:18 +0200 Message-Id: To: Terry Lambert Cc: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> <3CD6B71C.C0A502A1@mindspring.com> In-reply-to: Your message of "Mon, 06 May 2002 10:02:20 -0700 ." <3CD6B71C.C0A502A1@mindspring.com> Date: Tue, 07 May 2002 00:18:14 +0200 From: Philip Homburg Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Terry Lambert wrote: >Philip Homburg wrote: >> >From an O.S. point of view, what's the problem with providing large >> directories? Some tools, such as ls and find, may need some attention due to >> scaling issues. Backup software is needed to handle the new filesystem >> layout. > >#1 Binary backwards compatability with millions of installed > systems What kind backwards compatability? The usual Unix system calls are uneffected (creat, open, link, unlink, getdents, etc.). Even then, it is user's choice to create large directories, the system just supports it. >> If somebody would like to create 1M files, and you know that, when properly >> implemented, directories accesses cost O(log(n)), then what's the problem? > >#1 No one has written the code to optionally store directories > this way (and made it publically available) A small practical problem, that can be dealt with easily. (Unless you are asking for production quality code :-) >#2 (minor nit) A btree is O(log2(N)), which is not as happy a > number as your O(log(N)) O(log2(N)) = O(log(N)/log(2)) = O(log(N)) I don't see where you get the log2. The fan-out is determined by the avarage size of a directory. >> I don't want to think about crash recovery of btrees in a normal FFS >> filesystem. > >No one does, except the people asking us to switch to their pet >FS by default, but not releasing it under a usable license, and >expecting that we would just "eat" the backward compatability >issue. But that is just an implementation issue. The binary backwards compatability issues that you hinted at are more fundamental. Philip Homburg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 17:15:24 2002 Delivered-To: freebsd-fs@freebsd.org Received: from wellington.cnchost.com (wellington.concentric.net [207.155.252.14]) by hub.freebsd.org (Postfix) with ESMTP id 0E89637B403 for ; Mon, 6 May 2002 17:15:14 -0700 (PDT) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by wellington.cnchost.com id UAA15261; Mon, 6 May 2002 20:15:11 -0400 (EDT) [ConcentricHost SMTP Relay 1.14] Message-ID: <200205070015.UAA15261@wellington.cnchost.com> To: Eric Jacobs Cc: fs@FreeBSD.ORG Subject: Re: Filesystem In-reply-to: Your message of "Fri, 03 May 2002 22:41:26 EDT." Date: Mon, 06 May 2002 17:15:10 -0700 From: Bakul Shah Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Eric Jacobs writes: > Bakul Shah wrote: > > Plan9 does ".." right. The same can be done in Unix by > > storing the rooted path in the kernel for a process'es > > current working dir. and by following some path rewrite > > rules: > > > > //.. == > > //../ == / > > /../ == / > > Those rules aren't valid on the account of syntax alone. You would > have to know which components are symbolic links. And once you take > into account symbolic links, you have essentially what namei does > anyway. Actually the main reason for wanting these rules is to remove the surprise associated with symbolic links. Of course, by now people have accepted that bar/foo/.. may land you not in bar/ but somewhere totally different if foo is a symlink and some have perhaps even built programs that rely on that fact.... Think about it this way: With symlinks and mounts you can have more than one __path__ to a file or directory. ".." merely get you back one component on that particular path. This works without a surprise with mounts but not symlinks. For example, # mount server:/usr/src /usr/src # cd /usr/src # cd .. # /bin/pwd /usr # umount /usr/src # rmdir src # ln -s /host/server/usr/src /usr/src # cd /usr/src # cd .. # /bin/pwd /.amd_mnt/server/host/usr In effect with symlinks you can't retrace the path you took; you have to go the long way back. Read Rob Pike's paper "On getting dot-dot right" for a better explanation. > I think what Terry Lambert was saying was that since hard-linking > directories isn't allowed anyway, there's no need to refcount them, > except for the subdirectory counting tricks. Yes. > If you can handle access considerations yourself, one creative solution > might be to use getfh(2) and fhopen(2) and store the file handles however > way you want. This bypasses the kernel lookup entirely. Yes, you can always work around kernel limitations. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 19:17:19 2002 Delivered-To: freebsd-fs@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 29FCE37B405 for ; Mon, 6 May 2002 19:17:14 -0700 (PDT) Received: from pool0679.cvx21-bradley.dialup.earthlink.net ([209.179.194.169] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #2) id 174uXN-0005u1-00; Mon, 06 May 2002 19:17:09 -0700 Message-ID: <3CD73907.7FEEA69E@mindspring.com> Date: Mon, 06 May 2002 19:16:39 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Philip Homburg Cc: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> <3CD6B71C.C0A502A1@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Philip Homburg wrote: > >#1 Binary backwards compatability with millions of installed > > systems > > What kind backwards compatability? The usual Unix system calls are uneffected > (creat, open, link, unlink, getdents, etc.). > Even then, it is user's choice to create large directories, the system > just supports it. The ability to continue using my old disks with their old data on them, without having to buy enough backup media that I can back them up enough times I feel safe running the command "newfs -new_directory_format" on them, and restoring from tape (assuming I even trust this new format). > >> If somebody would like to create 1M files, and you know that, when properly > >> implemented, directories accesses cost O(log(n)), then what's the problem? > > > >#1 No one has written the code to optionally store directories > > this way (and made it publically available) > > A small practical problem, that can be dealt with easily. (Unless you > are asking for production quality code :-) Just code the same quality as the directory handling code already in UFS... > >#2 (minor nit) A btree is O(log2(N)), which is not as happy a > > number as your O(log(N)) > > O(log2(N)) = O(log(N)/log(2)) = O(log(N)) > > I don't see where you get the log2. The fan-out is determined by the > avarage size of a directory. Balanced btree. The number of compares necessary for the lookup for an arbitrary individual file is based on the depth of the btree (actually, thinking about this, a Fibonnaci or Particia tree would be a better structure, but we were talking btree directories). > >> I don't want to think about crash recovery of btrees in a normal FFS > >> filesystem. > > > >No one does, except the people asking us to switch to their pet > >FS by default, but not releasing it under a usable license, and > >expecting that we would just "eat" the backward compatability > >issue. > > But that is just an implementation issue. The binary backwards compatability > issues that you hinted at are more fundamental. Yes. Which is why when someone who sees only the implementation isses comes at the problem, they end up frustrated. 8-) 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 19:44:58 2002 Delivered-To: freebsd-fs@freebsd.org Received: from smtp2.mivlmd.cablespeed.com (smtp2.mivlmd.cablespeed.com [216.45.64.147]) by hub.freebsd.org (Postfix) with SMTP id AB93037B401 for ; Mon, 6 May 2002 19:44:50 -0700 (PDT) Received: (qmail 29900 invoked by uid 0); 7 May 2002 02:43:49 -0000 Received: from unknown (HELO cablespeed.com) (216.45.72.227) by 0 with SMTP; 7 May 2002 02:43:49 -0000 Message-ID: <3CD73F65.1D9241A2@cablespeed.com> Date: Mon, 06 May 2002 22:43:49 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 Cc: freebsd-fs@freebsd.org Subject: Re: New File system to commit to FreeBSD References: <3CD5F4CD.2F394EFE@cablespeed.com> <20020506040438.GZ36741@elvis.mu.org> <3CD63C3F.2F4E8F02@mindspring.com> <20020506082125.GC36741@elvis.mu.org> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alfred Perlstein wrote: > > * Terry Lambert [020506 01:18] wrote: > > Alfred Perlstein wrote: > > > * Chuck McCrobie [020505 20:13] wrote: > > > > 3. OpenVMS and ODs-2 keep record attributes and embedded meta-data in > > > > the regular file data. I've been writing some utility programs that > > > > understand the meta-data, like "ods2_cat" and "ods2_cp". Also, some > > > > work has progressed on an RMS library. Should and how are such things > > > > committed? > > > > > > This sounds like quite a suite of utilities, my _guess_ is that > > > the FS code could/would be committed to FreeBSD, but the utilities > > > to manipulate the FS (ods2_cat, ods2_cp) would go into a port. > > > > You could either hack into the namespace, or you could provide > > an fcntl() that gets passed down to the FS for an open descriptor, > > to turn these utilities tiny, and incorporate most of the code into > > the FS itself (you are much better off doing this, if you can, since > > it means that the facilities won't be "lost" into a port for which > > FTP services will have to be maintained in perpetuity). > > > > Right now, since FreeBSD doesn't propagate namespace selection > > information from the POSIX namespsace escape down properly, you > > would have to hack it to operate on a per component basis (not > > as bad as you might imagine, but still an annoying thing to have > > to do), so for my money, an fcntl() would be the best approach. > > Yeah, I find it really odd that one would expose the internals > of the actual meta-data to applications. > > ...wandering off topic... > Is that how VMS does it (in the kernel)? Or is there some userland > library that you _must use_ to access files that does this sort of > thing for you? Or does each userspace application have to do magic > to parse meta-data? > > -- > -Alfred Perlstein [alfred@freebsd.org] > 'Instead of asking why a piece of software is using "1970s technology," > start asking why software is ignoring 30 years of accumulated wisdom.' > Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ This fcntl would place the open descriptor in "record mode", during which the application would have to specify a buffer large enough to get the entire record. Most files in OpenVMS are record oriented, not byte-stream oriented as in Unix. Do you mean by "namespace selection" something like "/openvms/dir1/file1:record_access" as in the concept of "streams"? --- off-topic on OpenVMS --- OpenVMS has a "semi-user-land" layer called Record Management Services (RMS). You get RMS by default with the "RMS Services". One can completely by-pass this layer and do the meta-data handling in the application, but why bother. One can also create files without any embedded meta-data. My idea is to duplicate this RMS layer which sits between the file system and the user applications. Since most application in OpenVMS deal with Records, applications to read data from OpenVMS systems would also need to deal with records - there are exceptions, of course, like .jpg, .gif, etc. type files. Think UDP socket reading - if you don't specify a buffer big enough for the record, only part is read - how does one get the rest of the record? The entire intent of this project is to provide OpenVMS users with the ability to access their data on ODS-2 optical media. My initial goal was to make understanding and using the file system and utilities similar as on OpenVMS... -- ­ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon May 6 23:32:28 2002 Delivered-To: freebsd-fs@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id 251B337B404 for ; Mon, 6 May 2002 23:32:21 -0700 (PDT) Received: from pool0102.cvx21-bradley.dialup.earthlink.net ([209.179.192.102] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 174yWD-0001UD-00; Mon, 06 May 2002 23:32:14 -0700 Message-ID: <3CD774CF.3EC90036@mindspring.com> Date: Mon, 06 May 2002 23:31:43 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Chuck McCrobie Cc: freebsd-fs@freebsd.org Subject: Re: New File system to commit to FreeBSD References: <3CD5F4CD.2F394EFE@cablespeed.com> <20020506040438.GZ36741@elvis.mu.org> <3CD63C3F.2F4E8F02@mindspring.com> <20020506082125.GC36741@elvis.mu.org> <3CD73F65.1D9241A2@cablespeed.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Chuck McCrobie wrote: > This fcntl would place the open descriptor in "record mode", during > which the application would have to specify a buffer large enough to get > the entire record. Most files in OpenVMS are record oriented, not > byte-stream oriented as in Unix. > > Do you mean by "namespace selection" something like > "/openvms/dir1/file1:record_access" as in the concept of "streams"? /// ...per POSIX, a doubled initial slash is to be treated as implementation defined. The collapse of multiple intermediate "/" to a single "/" is also documented, but it's really only applicable to the default namespace. Doing the POSIX namespace escape thing should cause all subsequent path components to be interpreted as being in the selected namespace (the namespace selection should be inherited). This is kind of hard with the FreeBSD vop_lookup interface, unforunately. The first hard part is that it doesn't use real recursion, but uses mutual recursion, instead. You could handle that part by preparsing the "//" off in an upper level routine, and then calling the current upper level lookup code from the higher level code (basically, insert a function call layer to parse out the namespace to hold it constant for all subsequent path components). This takes care of the complication introduced by necessity of handling symbolic link information by expansion into the same path name buffer that was copied into with the user argument to the pathname taking functions. The second hard part is that there are instances internally where lookups are called multiple times. Rename is pretty hard, as is obtaining an empty directory entry slot in which to place the name of a newly create file. THe problem is also complicated, if you attempt to handle Macintosh style semantics ("case sensitive on storage, case insensitive on lookup") or multiple namespaces (e.g. "the 8.3 vs. the Unicode namespace under VFAT32 or NTFS", particularly when it comes to renaming that results in naming moving from one name space to another, and coupling the renaming of both names). The third hard part is that VFS layers are permitted, by the definition of VOP_LOOKUP, to consume multiple intermediate path components, and return an answer that isn't only per component. For this to be completely handled, you'd probably have to modify the FreeBSD path management code to use a Windows-like path component list descriptor "parsed path" object. THat's possible, but it's a lot of work, and would be a hard sell to the BSD community, I think. > --- off-topic on OpenVMS --- > > OpenVMS has a "semi-user-land" layer called Record Management Services > (RMS). You get RMS by default with the "RMS Services". One can > completely by-pass this layer and do the meta-data handling in the > application, but why bother. One can also create files without any > embedded meta-data. > > My idea is to duplicate this RMS layer which sits between the file > system and the user applications. Since most application in OpenVMS > deal with Records, applications to read data from OpenVMS systems would > also need to deal with records - there are exceptions, of course, like > .jpg, .gif, etc. type files. Think UDP socket reading - if you don't > specify a buffer big enough for the record, only part is read - how does > one get the rest of the record? Seeks get pretty interesting, too, particularly for language files with "variable length records with implied carriage return carriage control" -- the default text file type created by editors, and used for almost all source files. You would have to be careful here. Your library for using the code on FreBSD would have to make a decision here (and several other places) between bug-for-bug compatability, and POLA from someone coming into it as their first experience. One of the problems here is that, at least until the mid 1990's, VMS had the nasty habit of advancing the pointer after the character following the implied carriage return carriage control was read, rather than merely after reading the implied terminator. It also put the seek onto a record boundary (seeking to a saved position is implementation defined, and VMS defined it to be at the record boundary). So for parsed languages, you got the wrong start of line for the loop control structures for the loop start in one token look-ahead parsers, unless before recording you did an "ungetc(getc());". 8-). There are a lot of advantages to record oriented I/O that UNIX could benefit from learning from Files-11. The major one is that single record insertion in a list is trivial, which makes the FS *much* better at supporting databases. It's all very entithetical to C, so a library would probably be the correct way of handling the translation from stream. But I don't think it will work for records, which are intrinsic to the FS structure. You might be able to provide a record interface on top of a stream interface, but providing record access to an underlying record structured FS is hard, through UNIX interfaces (as "Eunice", the UNIX interface emulation package for VMS, really demonstrated). > The entire intent of this project is to provide OpenVMS users with the > ability to access their data on ODS-2 optical media. My initial goal > was to make understanding and using the file system and utilities > similar as on OpenVMS... Wow. Are you going to implement the entire HSM, or are you planning on single volume access only? The former would be a heck of an involved project... though FreeBSD could really use an HSM project (IMO). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue May 7 2:11:43 2002 Delivered-To: freebsd-fs@freebsd.org Received: from thebsh.namesys.com (thebsh.namesys.com [212.16.7.65]) by hub.freebsd.org (Postfix) with SMTP id C578637B404 for ; Tue, 7 May 2002 02:11:37 -0700 (PDT) Received: (qmail 29092 invoked from network); 7 May 2002 09:11:36 -0000 Received: from laputa.namesys.com.7.16.212.in-addr.arpa (HELO laputa.namesys.com) (212.16.7.124) by thebsh.namesys.com with SMTP; 7 May 2002 09:11:36 -0000 Received: by laputa.namesys.com (Postfix on SuSE Linux 7.3 (i386), from userid 511) id 2D30FB1CE; Tue, 7 May 2002 13:11:35 +0400 (MSD) From: Nikita Danilov MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15575.39495.117903.517273@laputa.namesys.com> Date: Tue, 7 May 2002 13:11:35 +0400 X-PGP-Fingerprint: 43CE 9384 5A1D CD75 5087 A876 A1AA 84D0 CCAA AC92 X-PGP-Key-ID: CCAAAC92 X-PGP-Key-At: http://wwwkeys.pgp.net:11371/pks/lookup?op=get&search=0xCCAAAC92 To: Terry Lambert Cc: Philip Homburg , fs@FreeBSD.ORG Subject: Re: Filesystem In-Reply-To: <3CD73907.7FEEA69E@mindspring.com> References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> <3CD6B71C.C0A502A1@mindspring.com> <3CD73907.7FEEA69E@mindspring.com> X-Mailer: VM 7.03 under 21.4 (patch 3) "Academic Rigor" XEmacs Lucid X-Tom-Swifty: "Condensed chicken soup," was Tom's canned response. Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Terry Lambert writes: > Philip Homburg wrote: > > >#1 Binary backwards compatability with millions of installed > > > systems > > > > What kind backwards compatability? The usual Unix system calls are uneffected > > (creat, open, link, unlink, getdents, etc.). > > Even then, it is user's choice to create large directories, the system > > just supports it. > > The ability to continue using my old disks with their old data > on them, without having to buy enough backup media that I can > back them up enough times I feel safe running the command > "newfs -new_directory_format" on them, and restoring from tape > (assuming I even trust this new format). > > > > >> If somebody would like to create 1M files, and you know that, when properly > > >> implemented, directories accesses cost O(log(n)), then what's the problem? > > > > > >#1 No one has written the code to optionally store directories > > > this way (and made it publically available) > > > > A small practical problem, that can be dealt with easily. (Unless you > > are asking for production quality code :-) > > Just code the same quality as the directory handling code already > in UFS... > > > > >#2 (minor nit) A btree is O(log2(N)), which is not as happy a > > > number as your O(log(N)) > > > > O(log2(N)) = O(log(N)/log(2)) = O(log(N)) > > > > I don't see where you get the log2. The fan-out is determined by the > > avarage size of a directory. > > Balanced btree. The number of compares necessary for the lookup > for an arbitrary individual file is based on the depth of the > btree (actually, thinking about this, a Fibonnaci or Particia > tree would be a better structure, but we were talking btree > directories). Still, fanout of btree is determined by block, pointer, and key size. Binary tree (where one will get log2(N) comparisons) is horribly inefficient as external indexing method. > > > > >> I don't want to think about crash recovery of btrees in a normal FFS > > >> filesystem. > > > > > >No one does, except the people asking us to switch to their pet > > >FS by default, but not releasing it under a usable license, and > > >expecting that we would just "eat" the backward compatability > > >issue. > > > > But that is just an implementation issue. The binary backwards compatability > > issues that you hinted at are more fundamental. > > Yes. Which is why when someone who sees only the implementation > isses comes at the problem, they end up frustrated. 8-) 8-). > > -- Terry Nikita. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue May 7 7:39:17 2002 Delivered-To: freebsd-fs@freebsd.org Received: from sluis.cs.vu.nl (sluis.cs.vu.nl [130.37.192.174]) by hub.freebsd.org (Postfix) with ESMTP id C0FB037B403 for ; Tue, 7 May 2002 07:39:04 -0700 (PDT) Received: from centaur.cs.vu.nl (centaur.cs.vu.nl [192.35.191.4]) by sluis.cs.vu.nl with esmtp (Smail #74) id m17567J-000OesC; Tue, 7 May 2002 16:39 +0200 Received: from cs.vu.nl (localhost [127.0.0.1]) by centaur.cs.vu.nl with esmtp (Smail #74) id m17567J-0014NkC; Tue, 7 May 2002 16:39 +0200 Message-Id: To: Terry Lambert Cc: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> <3CD6B71C.C0A502A1@mindspring.com> <3CD73907.7FEEA69E@mindspring.com> In-reply-to: Your message of "Mon, 06 May 2002 19:16:39 -0700 ." <3CD73907.7FEEA69E@mindspring.com> Date: Tue, 07 May 2002 16:39:01 +0200 From: Philip Homburg Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In your letter dated Mon, 06 May 2002 19:16:39 -0700 you wrote: >Philip Homburg wrote: >> >#1 Binary backwards compatability with millions of installed >> > systems >> >> What kind backwards compatability? The usual Unix system calls are uneffecte >d >> (creat, open, link, unlink, getdents, etc.). >> Even then, it is user's choice to create large directories, the system >> just supports it. > >The ability to continue using my old disks with their old data >on them, without having to buy enough backup media that I can >back them up enough times I feel safe running the command >"newfs -new_directory_format" on them, and restoring from tape >(assuming I even trust this new format). Are you saying that, apart from licensing and implementation issues, the only problem is whether or not you can continue to use disks with old filesystems? Of course you can simply leave support for FFS and soft-updates in the kernel. You only convert filesystems if you expect significant benefits. Philip Homburg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue May 7 14:23: 4 2002 Delivered-To: freebsd-fs@freebsd.org Received: from web11102.mail.yahoo.com (web11102.mail.yahoo.com [216.136.131.149]) by hub.freebsd.org (Postfix) with SMTP id 5893B37B40A for ; Tue, 7 May 2002 14:22:38 -0700 (PDT) Message-ID: <20020507212238.45104.qmail@web11102.mail.yahoo.com> Received: from [200.225.195.237] by web11102.mail.yahoo.com via HTTP; Tue, 07 May 2002 18:22:37 ART Date: Tue, 7 May 2002 18:22:37 -0300 (ART) From: "=?iso-8859-1?q?Rodrigo=20F.=20Baroni?=" Reply-To: rodrigobaroni@yahoo.com.br Subject: FreeBSD development in university To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hello all ! I'm a computer science student from Sao Paulo, Brazil , I've studied OS ( so very ), and I would like to learn much more about FreeBSD - kernel, MM, FS, ... but seems that everything points to "FreeBSD Design and Implementation" book,.. Could please anyone tell me about another springs of FreeBSD development? Thanks so very Rodrigo F Baroni CS B's Degree Ribeirao Preto , Sao Paulo, Brazil _______________________________________________________________________ Yahoo! Encontros O lugar certo para você encontrar aquela pessoa que falta na sua vida. Cadastre-se hoje mesmo! http://br.encontros.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 10 11:19:47 2002 Delivered-To: freebsd-fs@freebsd.org Received: from mel-rto2.wanadoo.fr (smtp-out-2.wanadoo.fr [193.252.19.254]) by hub.freebsd.org (Postfix) with ESMTP id E26ED37B40A; Fri, 10 May 2002 11:19:39 -0700 (PDT) Received: from mel-rta9.wanadoo.fr (193.252.19.69) by mel-rto2.wanadoo.fr (6.5.007) id 3CDBAC16000325CA; Fri, 10 May 2002 20:11:16 +0200 Received: from mqueue1.sunpoint.net (193.252.56.169) by mel-rta9.wanadoo.fr (6.5.007) id 3CD663FF00237D23; Fri, 10 May 2002 20:11:16 +0200 Date: Fri, 10 May 2002 20:11:16 +0200 (added by postmaster@wanadoo.fr) Message-ID: <3CD663FF00237D23@> (added by postmaster@wanadoo.fr) From: "Kelle" To: "etkmwuepb@msn.com" Subject: Feel great,look super this summer Content-Type: text/plain; charset="us-ascii";format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org As seen on NBC, CBS, CNN, and even Oprah! The health discovery that actually reverses aging while burning fat, without dieting or exercise! This proven discovery has even been reported on by the New England Journal of Medicine. Forget aging and dieting forever! And it's Guaranteed! Click here: http://66.231.133.70/sj1/index.html Would you like to lose weight while you sleep! No dieting! No hunger pains! No Cravings! No strenuous exercise! Change your life forever! 100% GUARANTEED! 1.Body Fat Loss 82% improvement. 2.Wrinkle Reduction 61% improvement. 3.Energy Level 84% improvement. 4.Muscle Strength 88% improvement. 5.Sexual Potency 75% improvement. 6.Emotional Stability 67% improvement. 7.Memory 62% improvement. *********************************************************** You are receiving this email as a subscriber to the Opt-In America Mailing List. To unsubscribe from future offers, just click here: mailto:affiliateoptout@btamail.net.cn?Subject=off To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 10 13:13:22 2002 Delivered-To: freebsd-fs@freebsd.org Received: from viola.sinor.ru (viola.sinor.ru [217.70.106.9]) by hub.freebsd.org (Postfix) with ESMTP id 0E71637B405; Fri, 10 May 2002 13:13:09 -0700 (PDT) Received: from tlg5-ppp59.sibnet.ru (tlg5-ppp59.sibnet.ru [217.70.97.60]) by viola.sinor.ru (8.9.3/8.9.3) with ESMTP id DAA05084; Sat, 11 May 2002 03:13:01 +0700 Date: Sat, 11 May 2002 03:12:56 +0700 (NOVST) From: "Semen A. Ustimenko" X-X-Sender: semenu@def.the.net To: freebsd-fs@FreeBSD.org Cc: freebsd-hackers@FreeBSD.org, "Flood, Jim" Subject: NULLFS-related possible deadlock + fix proposal Message-ID: <20020511005932.S1705-100000@def.the.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi! Preface: Same directory is null-mounted to "/mnt" and "/mnt2". The directory contain "dir/foofile". Two processes concurently lookup "/mnt/dir/foofile" and "/mnt2/dir/foofile". Action: P1: in lookup(): in VOP_LOOKUP(dvp (== "/mnt/dir"), "foofile"): in null_lookup(): in null_node_create(): in malloc() | getnewvnode() | somewhere(): in tsleep() -> P1 is preempted by P2 (Note, that "foofile"'s lower vnode is locked by P1, "dir"'s lower vnode is unlocked, thus "/mnt2/dir" is also unlocked) P2: in lookup(): in VOP_LOOKUP(dvp (== "/mnt2/dir"), "foofile): in null_lookup(): in VOP_LOOKUP(lowerdvp, "foofile"): in tsleep(), waiting for "foofile"'s lower vnode, held by P1 (Note, the "/mnt2/dir"'s vnode and thus its lower vnode is still locked by P2, the "foofile"'s lower vnode is locked by P1) P1: in lookup(): in vrele(dvp (== "/mnt/dir")): in vn_lock(dvp): in tsleep(), waiting for "/mnt/dir"'s lower vnode, held by P2 DEADLOCK... Analysis: The lookup() routine can call vrele(), in its turn vrele() can vn_lock() parent directory, while holding lock on file from this directory. This isn't a problem for nonstacking FSes as vrele() will only vn_lock if it were the last reference. For NULLFS this is a problem because completely different vnodes can share the lock structure. Solution: Make vn_lock() in vrele() lock vnode only LK_THISLAYER. Obviously, the NULLFS and other stacking FSes will have to deal with this in their VOP_INACTIVE() handlers. This changes won't touch real FSes as they ignore the LK_THISLAYER, don't they? Bye! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message