From owner-freebsd-fs Tue Jan 18 18:53:28 2000 Delivered-To: freebsd-fs@freebsd.org Received: from altair.origenbio.com (altair.origenbio.com [216.30.62.130]) by hub.freebsd.org (Postfix) with ESMTP id 83CF31529B for ; Tue, 18 Jan 2000 18:53:20 -0800 (PST) (envelope-from dmartin@origen.com) Received: from origen.com (dubhe.origen [192.168.0.5]) by altair.origenbio.com (8.9.3/8.9.3) with ESMTP id UAA25353; Tue, 18 Jan 2000 20:53:16 -0600 (CST) (envelope-from dmartin@origen.com) Message-ID: <388526E1.E4B90225@origen.com> Date: Tue, 18 Jan 2000 20:52:17 -0600 From: Richard Martin X-Mailer: Mozilla 4.6 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: hou-freebsd@cityscope.net Cc: freebsd-fs@FreeBSD.ORG Subject: CD-R mount solved References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org After beating away on this with the -v switch, I discovered the way to mount CD-Rs burned with Adaptec (and probably other) software on a different system. The problem is that the mount_cd9660 command is looking for the start sector and making its best guess where it is. Possibly the burned CDs do not lay down a table track at the end of the write area. The CD can be mounted by specifying the start sector at track 0 using this: mount_cd9660 -s 0 /dev/cd0c /cdrom The mount command returns a note from the kernel that this is CD9660 with Joliet extensions. Hope this helps someone else -- Richard Martin dmartin@origen.com OriGen Biomedical Tel: +1 512 474 7278 2525 Hartford Rd. Fax: +1 512 708 8522 Austin, TX 78703 http://www.cardiacdocs.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Jan 20 4:24:59 2000 Delivered-To: freebsd-fs@freebsd.org Received: from zeus.openline.com.br (openline.com.br [200.241.199.2]) by hub.freebsd.org (Postfix) with ESMTP id 5BF2B153DA for ; Thu, 20 Jan 2000 04:24:52 -0800 (PST) (envelope-from mailbr@mailbr.com.br) Received: from oemcomputer (JPA-2-S35.openline.com.br [200.249.178.100]) by zeus.openline.com.br (8.8.5/8.8.7) with SMTP id KAA19399 for ; Thu, 20 Jan 2000 10:24:45 -0200 Message-ID: <004d01bf6342$6aa43140$64b2f9c8@oemcomputer> Reply-To: =?iso-8859-1?B?QWRtaW5pc3RyYefjbyBNYWlsQlI=?= From: =?iso-8859-1?B?QWRtaW5pc3RyYefjbyBNYWlsQlI=?= To: References: <20000119211627.47573152A9@hub.freebsd.org> Subject: NFS: Linux Client and BSD Server Date: Thu, 20 Jan 2000 10:32:26 -0200 Organization: =?iso-8859-1?B?TWFpbEJSIENvbXVuaWNh5+Nv?= MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi People, How can I increse the linux client performance running a NFS server in BSD? My FreeBSD is 3.4 and Linux 2.2.12-20 Thanks Pedro Anisio de Luna e Silva MailBR To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Jan 20 4:34:53 2000 Delivered-To: freebsd-fs@freebsd.org Received: from email.spcgroup.nl (email.spcgroup.nl [212.206.124.3]) by hub.freebsd.org (Postfix) with ESMTP id 5455C153A5 for ; Thu, 20 Jan 2000 04:34:46 -0800 (PST) (envelope-from e.mons@spcgroup.nl) Received: from spcgroup.nl (fw.spcgroup.nl [212.206.124.2]) by email.spcgroup.nl (8.9.3/8.9.3) with ESMTP id OAA27476; Thu, 20 Jan 2000 14:44:18 +0100 Message-ID: <388700C2.8820CB91@spcgroup.nl> Date: Thu, 20 Jan 2000 13:34:10 +0100 From: Edwin Mons X-Mailer: Mozilla 4.7 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: =?iso-8859-1?Q?Administra=E7=E3o?= MailBR , freebsd-fs@freebsd.org Subject: Re: NFS: Linux Client and BSD Server References: <20000119211627.47573152A9@hub.freebsd.org> <004d01bf6342$6aa43140$64b2f9c8@oemcomputer> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Administração MailBR wrote: > > Hi People, > > How can I increse the linux client performance running a NFS server in > BSD? My FreeBSD is 3.4 and Linux 2.2.12-20 > > Thanks > Pedro Anisio de Luna e Silva > MailBR > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message Well, you could set the option vfs.nfs.async to 1 on your BSD server with sysctl -w vfs.nfs.async=1 When I did this while using an NFS (BSD <-> BSD) the speed went up to 1 MB/sec (on 10baseT). Theoretical maximum for 10 Mbit ethernet :-) YMMV. Edwin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Jan 20 8: 3:15 2000 Delivered-To: freebsd-fs@freebsd.org Received: from queeg.ludd.luth.se (queeg.ludd.luth.se [130.240.16.109]) by hub.freebsd.org (Postfix) with ESMTP id 6267314EC9 for ; Thu, 20 Jan 2000 08:03:09 -0800 (PST) (envelope-from pantzer@ludd.luth.se) Received: from speedy.ludd.luth.se (pantzer@speedy.ludd.luth.se [130.240.16.164]) by queeg.ludd.luth.se (8.9.3/8.9.3) with ESMTP id RAA13433; Thu, 20 Jan 2000 17:02:25 +0100 (CET) Message-Id: <200001201602.RAA13433@queeg.ludd.luth.se> X-Mailer: exmh version 2.0.2 2/24/98 To: Edwin Mons Cc: =?iso-8859-1?Q?Administra=E7=E3o?= MailBR , freebsd-fs@FreeBSD.ORG, pantzer@ludd.luth.se Subject: Re: NFS: Linux Client and BSD Server In-Reply-To: Message from Edwin Mons of "Thu, 20 Jan 2000 13:34:10 +0100." <388700C2.8820CB91@spcgroup.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Thu, 20 Jan 2000 17:02:25 +0100 From: Mattias Pantzare Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Administra=E7=E3o MailBR wrote: > > = > > Hi People, > > = > > How can I increse the linux client performance running a NFS serv= er in > > BSD? My FreeBSD is 3.4 and Linux 2.2.12-20 > = > Well, you could set the option vfs.nfs.async to 1 on your BSD server > with > sysctl -w vfs.nfs.async=3D1 > = > When I did this while using an NFS (BSD <-> BSD) the speed went up to 1= > MB/sec (on 10baseT). Theoretical maximum for 10 Mbit ethernet :-) And break the NFS protocol. Pray that the NFS server won't crash if you h= ave = that on. On FreeBSD you shold chek that you have started some nfsiod on the client= s. I = don't know if something like that exists on Linux. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Jan 21 10:18: 8 2000 Delivered-To: freebsd-fs@freebsd.org Received: from mail4.aracnet.com (mail4.aracnet.com [216.99.193.36]) by hub.freebsd.org (Postfix) with ESMTP id BA511154DB for ; Fri, 21 Jan 2000 10:18:00 -0800 (PST) (envelope-from beattie@aracnet.com) Received: from shell1.aracnet.com (IDENT:root@shell1.aracnet.com [216.99.193.21]) by mail4.aracnet.com (8.9.3/8.9.3) with ESMTP id KAA03560 for ; Fri, 21 Jan 2000 10:18:05 -0800 Received: from localhost by shell1.aracnet.com (8.9.3) id KAA29529; Fri, 21 Jan 2000 10:19:36 -0800 X-Authentication-Warning: shell1.aracnet.com: beattie owned process doing -bs Date: Fri, 21 Jan 2000 10:19:36 -0800 (PST) From: Brian Beattie To: fs@freebsd.org Subject: UDF, userfs Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have made a couple of posts to hackers, that probably should have gone here to fs. I and thinking about implementing a UDF filesystem. The plan I am considering, is to implement a "userfs" to allow me to do most of the work in a user process. I have been thinking about the userfs implementation. I will need some way for the user process to talk the backend of the userfs kernel code. The two ways I have thought of are I/O,, probably ioctl's or a new system call. I assume that it is possible, using a module to add an entry to the syscall table, but I lean more towards a new pseudo device to hang the ioctl's off of. I would be ineterested in any comments. I would also like to hear from anybody who has thought about a userfs implementation for FreeBSD. Brian Beattie | The only problem with beattie@aracnet.com | winning the rat race ... www.aracnet.com/~beattie | in the end you're still a rat To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Jan 21 13:27:10 2000 Delivered-To: freebsd-fs@freebsd.org Received: from kronos.alcnet.com (kronos.alcnet.com [63.69.28.22]) by hub.freebsd.org (Postfix) with ESMTP id 839841571D for ; Fri, 21 Jan 2000 13:27:06 -0800 (PST) (envelope-from kbyanc@posi.net) X-Provider: ALC Communications, Inc. http://www.alcnet.com/ Received: from localhost (kbyanc@localhost) by kronos.alcnet.com (8.9.3/8.9.3/antispam) with ESMTP id QAA72534; Fri, 21 Jan 2000 16:26:59 -0500 (EST) Date: Fri, 21 Jan 2000 16:26:59 -0500 (EST) From: Kelly Yancey X-Sender: kbyanc@kronos.alcnet.com To: Brian Beattie Cc: fs@FreeBSD.ORG Subject: Re: UDF, userfs In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 21 Jan 2000, Brian Beattie wrote: > I have made a couple of posts to hackers, that probably should have gone > here to fs. I and thinking about implementing a UDF filesystem. The plan > I am considering, is to implement a "userfs" to allow me to do most of the > work in a user process. > > I have been thinking about the userfs implementation. I will need some > way for the user process to talk the backend of the userfs kernel code. > The two ways I have thought of are I/O,, probably ioctl's or a new system > call. > > I assume that it is possible, using a module to add an entry to the > syscall table, but I lean more towards a new pseudo device to hang the > ioctl's off of. > > I would be ineterested in any comments. I would also like to hear from > anybody who has thought about a userfs implementation for FreeBSD. > I would love this functionality (I've often considered implementing it myself before being occupied by other projects). A userfs implemention would open the doors to all kinds of filesystems and make development of new filesystems much simpler (as debugging could be done in user mode). Currently, filesystems have to pretend to be NFS (i.e. sharity) to bridge the userland/kernel filesystem boundary. However, it is probably important to note again that I have always gotten distracted before investing significant time into pursuing this (read: I'm no expert :) ). Perhaps you have put more thought into it, but the concern that immediately comes to my mind is how to get VOP function calls from kernel-space to user-space. Some kind of up-call mechanism seems in order. You had suggested a pseudo-device, presumably with the intent to have daemons which implement file systems to read from to catch VOPs and write to to return data. I am curious, however, as to how NFS handles the problem. I have never looked at the NFS implemention in FreeBSD (mostly due to lack of time, and partly out of fear :) ). Assuming that it is not too much of a kludge, it might be better not to reinvent that wheel (which may mean the userfs becomes a NFS wrapper much like sharity is). Anyway, those are my somewhat disjoint thoughts, offered at face value ($0.02). Kelly -- Kelly Yancey - kbyanc@posi.net - Richmond, VA Analyst / E-business Development, Bell Industries http://www.bellind.com/ Maintainer, BSD Driver Database http://www.posi.net/freebsd/drivers/ Coordinator, Team FreeBSD http://www.posi.net/freebsd/Team-FreeBSD/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Jan 21 14:20:23 2000 Delivered-To: freebsd-fs@freebsd.org Received: from mail4.aracnet.com (mail4.aracnet.com [216.99.193.36]) by hub.freebsd.org (Postfix) with ESMTP id F32481553B for ; Fri, 21 Jan 2000 14:20:20 -0800 (PST) (envelope-from beattie@aracnet.com) Received: from shell1.aracnet.com (IDENT:root@shell1.aracnet.com [216.99.193.21]) by mail4.aracnet.com (8.9.3/8.9.3) with ESMTP id OAA30191; Fri, 21 Jan 2000 14:20:26 -0800 Received: from localhost by shell1.aracnet.com (8.9.3) id OAA03529; Fri, 21 Jan 2000 14:21:58 -0800 X-Authentication-Warning: shell1.aracnet.com: beattie owned process doing -bs Date: Fri, 21 Jan 2000 14:21:58 -0800 (PST) From: Brian Beattie To: Kelly Yancey Cc: fs@FreeBSD.ORG Subject: Re: UDF, userfs In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 21 Jan 2000, Kelly Yancey wrote: > On Fri, 21 Jan 2000, Brian Beattie wrote: > > > I have been thinking about the userfs implementation. I will need some > > way for the user process to talk the backend of the userfs kernel code. > > The two ways I have thought of are I/O,, probably ioctl's or a new system > > call. > > ... > Perhaps you have put more thought into it, but > the concern that immediately comes to my mind is how to get VOP function > calls from kernel-space to user-space. Some kind of up-call mechanism > seems in order. You had suggested a pseudo-device, presumably with the > intent to have daemons which implement file systems to read from to > catch VOPs and write to to return data. Basicaly, the user process that implements the filesystem, would issue a calls (i assume an ioctl on a pseudo device, but I am open to suggestions). The userfs module would pass any VOP's back to the process, or return an empty result. One could also implement select support. (exact details still open) This is somewhat counter-intutive, in that one normally thinks of the fs code calling something, and in this case it is called, but the effect is ultimately the same, the operations get passed where they need to go. Brian Beattie | The only problem with beattie@aracnet.com | winning the rat race ... www.aracnet.com/~beattie | in the end you're still a rat To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Jan 21 15:54:14 2000 Delivered-To: freebsd-fs@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id B72AB1595D for ; Fri, 21 Jan 2000 15:54:09 -0800 (PST) (envelope-from robert@cyrus.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id SAA71740; Fri, 21 Jan 2000 18:54:15 -0500 (EST) (envelope-from robert@cyrus.watson.org) Date: Fri, 21 Jan 2000 18:54:15 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org Reply-To: Robert Watson To: Brian Beattie Cc: fs@freebsd.org Subject: Re: UDF, userfs In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Brian, Both the Arla and Coda file systems are distributed file systems managed from userland processes. They do this by providing loadable kernel modules (or static compiled in code) to allow userland processes to listen on a device file (/dev/xfsX for Arla, /dev/codaX for Coda) and receive "upcalls" from the kernel. Usually, the userland process is multi-threaded so that it can handle in parallel requests that might involve extensive waiting (for example, transfering a large file using a network file system). Before writing something from scratch, you may want to investigate these and see if they meet your needs. Coda is available in the base FreeBSD distribution, and Arla as a package. I know that the Arla project has put significant work into making their code useful beyond just an AFS client with the intent in mind that other developers could use the module for other file systems. I built a fair portion of a caching file system based on the Arla kernel module at one point. Hope this is helpful, Robert On Fri, 21 Jan 2000, Brian Beattie wrote: > I have made a couple of posts to hackers, that probably should have gone > here to fs. I and thinking about implementing a UDF filesystem. The plan > I am considering, is to implement a "userfs" to allow me to do most of the > work in a user process. > > I have been thinking about the userfs implementation. I will need some > way for the user process to talk the backend of the userfs kernel code. > The two ways I have thought of are I/O,, probably ioctl's or a new system > call. > > I assume that it is possible, using a module to add an entry to the > syscall table, but I lean more towards a new pseudo device to hang the > ioctl's off of. > > I would be ineterested in any comments. I would also like to hear from > anybody who has thought about a userfs implementation for FreeBSD. > > Brian Beattie | The only problem with > beattie@aracnet.com | winning the rat race ... > www.aracnet.com/~beattie | in the end you're still a rat > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message > Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Jan 21 17:57:25 2000 Delivered-To: freebsd-fs@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id D390D15096 for ; Fri, 21 Jan 2000 17:57:15 -0800 (PST) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id SAA00441; Fri, 21 Jan 2000 18:56:58 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp03.primenet.com, id smtpdAAAd6a41a; Fri Jan 21 18:56:54 2000 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id SAA25999; Fri, 21 Jan 2000 18:57:08 -0700 (MST) From: Terry Lambert Message-Id: <200001220157.SAA25999@usr09.primenet.com> Subject: Re: UDF, userfs To: beattie@aracnet.com (Brian Beattie) Date: Sat, 22 Jan 2000 01:57:07 +0000 (GMT) Cc: fs@FreeBSD.ORG In-Reply-To: from "Brian Beattie" at Jan 21, 2000 10:19:36 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I have made a couple of posts to hackers, that probably should have gone > here to fs. I and thinking about implementing a UDF filesystem. The plan > I am considering, is to implement a "userfs" to allow me to do most of the > work in a user process. > > I have been thinking about the userfs implementation. I will need some > way for the user process to talk the backend of the userfs kernel code. > The two ways I have thought of are I/O,, probably ioctl's or a new system > call. > > I assume that it is possible, using a module to add an entry to the > syscall table, but I lean more towards a new pseudo device to hang the > ioctl's off of. > > I would be ineterested in any comments. I would also like to hear from > anybody who has thought about a userfs implementation for FreeBSD. The Apple people have already implemented one of these. The main problem is that you have to access part of the disk in ISO9660 mode, and part of it in raw mode. This is a problem in the current FreeBSD because the Apple people used one mount on the character device and one on the block device to accomplish this magic. Effectively, you will probably need to define a new slicing mechanism, to sit at a level equal to the DOS partitioning, extended parititioning, and the BSD disklabel to get around the inability to do this any more. This would require non-userspace work, but it would work rather well. As far as proxying into user space for FS developement, there is still a coherency problem. You will need to use the VOP_READ and VOP_WRITE to implement VOP_GETPAGES and VOP_PUTPAGES in order to get around this. See the hacked up NULLFS that was posted about to the FreeBSD-FS list as a means of getting around the kernel VFS and vmobject_t brain damage. You could try to modify "amd" or a similar user space NFS server to push the data, but for DVD, I think the data rate you would see doing this would be terrible. As an alternative, I'd suggest maybe developing it in user space, but expecting to deploy it in kernel space. This is not really as hard as it sounds, if you buy into the slice handler for seperating the ISO9660 and non-ISO9660 data on the disk. As far as the proxy itself, I suggest a "tun" or "bpf" style pseudodevice, since it has worked well for me in the past. The trick is that the VOP's are all passed around as descriptors, and they are opaque to crossing user/kernel boundaries. You will need to be able to lock down the references, if you intend to access them via /dev/kmem. Instead, I suggest, you mmap() the device into your process address space, and access it that way, since it greatly simplifies the faulting. In terms of proxying VOPs across an opaque boundary, you should read John Heidemann's master's thesis from the FICUS project out of UCLA; it is available at ftp.cs.ucla.edu; if you need a better URL, I can find it for you. It was intended to support proxy of data over a network, but the principle extends to any opaque boundary crossing, where the address space is not the same on both sides of the boundary. This will also show you why the FreeBSD implemetnations of VOP_READ, VOP_WRITE, VOP_{GET|PUT}PAGES, and VOP_READDIR (cookies) are currently broken. You will have to hack the mount stuff to access through either a seperate psuedodevice, or some form of covert channel on the same pseudo device. The way I personally handle it is to collapse all the mount code, and seperate it into to VFS_ routines (VFS_MOUNT, VFS_SETINFO), and move all of the code that can differentiate between root and non-root mounts to higher level code (this lets me do things like boot with a UMSDOS layer stacked over an MSDOS layer as my root filesystem, for example). Then I start my user space server, and use two pseudo devices, /dev/vfs0 and /dev/vfs0vop. The VFS ops are proxied over the first, the VOPs over the second. Then the mount can proceed. If no one has the /dev/vfs0 open, the mount attempt fails. Hope this helps, and hope you can get your code for it back into the tree. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 11: 4:57 2000 Delivered-To: freebsd-fs@freebsd.org Received: from aplcenMP.apl.jhu.edu (apl.jhu.edu [128.220.101.100]) by hub.freebsd.org (Postfix) with ESMTP id D206A14EB5 for ; Sat, 22 Jan 2000 11:04:53 -0800 (PST) (envelope-from mccrobi@aplcenMP.apl.jhu.edu) Received: from apl.jhu.edu (kslip13.apl.jhu.edu [128.220.108.23]) by aplcenMP.apl.jhu.edu (8.9.3/8.9.1) with ESMTP id OAA14223 for ; Sat, 22 Jan 2000 14:05:55 -0500 (EST) Message-ID: <388A004B.12203610@apl.jhu.edu> Date: Sat, 22 Jan 2000 14:08:59 -0500 From: Chuck McCrobie X-Mailer: Mozilla 4.51 [en] (X11; I; FreeBSD 3.3-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: fs@FreeBSD.ORG Subject: Re: UDF, userfs References: <200001220157.SAA25999@usr09.primenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org There are some file systems in the commerical world that "uplink" to a user mode daemon that does the actual on-disk work. Without naming names, I can think of at least two that actually do this... They present a pseudo-device which the end-user mounts. This pseudo-device catches all the file system requests and "up calls" the user mode daemon. The user mode daemon than talks to the real device to perform the i/o. I may be able to provide details of the process from another Unix system, but not actual code. This approach seems to handle many o/s'es in that the daemon can be made somewhat portable. The pseudo-device, however, is of necessity o/s specific. One may send flames this way, but I'm also wondering about doing this for NT also. A pseudo-device to catch the file system IRP's, up call some "NT Service" which actually does the on-disk managment and I/O to the real device. Please note that I'm not interested in placing more than one slice on a disk. My background is optical where, typically, one file system slice is laid down on the entire surface. I'm wondering about the actual performance of such a file system. The user mode daemon would have to compete for disk/page with other user mode programs. Perhaps this is not too bad. Chuck McCrobie (** MAD VAX **) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 11:29:33 2000 Delivered-To: freebsd-fs@freebsd.org Received: from aplcenMP.apl.jhu.edu (apl.jhu.edu [128.220.101.100]) by hub.freebsd.org (Postfix) with ESMTP id 8E04214C2D for ; Sat, 22 Jan 2000 11:29:21 -0800 (PST) (envelope-from mccrobi@aplcenMP.apl.jhu.edu) Received: from apl.jhu.edu (kslip13.apl.jhu.edu [128.220.108.23]) by aplcenMP.apl.jhu.edu (8.9.3/8.9.1) with ESMTP id OAA14733 for ; Sat, 22 Jan 2000 14:29:54 -0500 (EST) Message-ID: <388A05EA.E8CE2EBA@apl.jhu.edu> Date: Sat, 22 Jan 2000 14:32:58 -0500 From: Chuck McCrobie X-Mailer: Mozilla 4.51 [en] (X11; I; FreeBSD 3.3-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: fs@FreeBSD.ORG Subject: Userfs Daemon Dies, What Happens? & Caching Questions Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I've been following the discussion about the userfs. I'm wondering what happens if, while the pseudo-device is mounted, the daemon dies. Doesn't that tend to "fudge" up the pseudo-device? I've only just started into file system implementations for FreeBSD, so sorry if the answer is obvious... It would seem that some type of recovery would be necessary for the new daemon when its started. Does the userfs handle caching of disk blocks? Does it handle caching of inodes? Does it handle caching of directory data? If so, the userfs would then not benefit from the caching algorithms in the kernel? Chuck McCrobie (** MAD VAX **) mccrobi@apl.jhu.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 22: 7:16 2000 Delivered-To: freebsd-fs@freebsd.org Received: from yana.lemis.com (yana.lemis.com [192.109.197.140]) by hub.freebsd.org (Postfix) with ESMTP id 5C55C150ED for ; Sat, 22 Jan 2000 22:06:58 -0800 (PST) (envelope-from grog@mojave.worldwide.lemis.com) Received: from mojave.worldwide.lemis.com (j39.klt31.jaring.my [161.142.169.233]) by yana.lemis.com (8.8.8/8.8.8) with ESMTP id QAA11616; Sun, 23 Jan 2000 16:35:55 +1030 (CST) (envelope-from grog@mojave.worldwide.lemis.com) Received: (from grog@localhost) by mojave.worldwide.lemis.com (8.9.3/8.9.3) id NAA00466; Sat, 22 Jan 2000 13:19:43 +0800 (SGT) (envelope-from grog) Date: Sat, 22 Jan 2000 13:19:42 +0800 From: Greg Lehey To: Robert Watson Cc: Brian Beattie , fs@FreeBSD.ORG Subject: Passing information from kernel to user (was: UDF, userfs) Message-ID: <20000122131942.D391@mojave.worldwide.lemis.com> Reply-To: Greg Lehey References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from robert@cyrus.watson.org on Fri, Jan 21, 2000 at 06:54:15PM -0500 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Friday, 21 January 2000 at 18:54:15 -0500, Robert Watson wrote: > Both the Arla and Coda file systems are distributed file systems > managed from userland processes. They do this by providing loadable > kernel modules (or static compiled in code) to allow userland > processes to listen on a device file (/dev/xfsX for Arla, /dev/codaX > for Coda) and receive "upcalls" from the kernel. This is an interesting concept. One of the ideas I have on a back burner (well, at least I haven't forgotten about it :-) is a Samba issue. The way the SMB protocol works, it's possible that a remote workstation will send a message to the owner of -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 22: 7:44 2000 Delivered-To: freebsd-fs@freebsd.org Received: from yana.lemis.com (yana.lemis.com [192.109.197.140]) by hub.freebsd.org (Postfix) with ESMTP id EF51715083 for ; Sat, 22 Jan 2000 22:07:31 -0800 (PST) (envelope-from grog@mojave.worldwide.lemis.com) Received: from mojave.worldwide.lemis.com (j39.klt31.jaring.my [161.142.169.233]) by yana.lemis.com (8.8.8/8.8.8) with ESMTP id QAA11619; Sun, 23 Jan 2000 16:36:42 +1030 (CST) (envelope-from grog@mojave.worldwide.lemis.com) Received: (from grog@localhost) by mojave.worldwide.lemis.com (8.9.3/8.9.3) id NAA00455; Sat, 22 Jan 2000 13:16:57 +0800 (SGT) (envelope-from grog) Date: Sat, 22 Jan 2000 13:16:57 +0800 From: Greg Lehey To: Robert Watson Cc: Brian Beattie , fs@FreeBSD.ORG Subject: Re: UDF, userfs Message-ID: <20000122131656.C391@mojave.worldwide.lemis.com> Reply-To: Greg Lehey References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from robert@cyrus.watson.org on Fri, Jan 21, 2000 at 06:54:15PM -0500 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Friday, 21 January 2000 at 18:54:15 -0500, Robert Watson wrote: > On Fri, 21 Jan 2000, Brian Beattie wrote: > >> I have made a couple of posts to hackers, that probably should have gone >> here to fs. I and thinking about implementing a UDF filesystem. The plan >> I am considering, is to implement a "userfs" to allow me to do most of the >> work in a user process. >> >> I have been thinking about the userfs implementation. I will need some >> way for the user process to talk the backend of the userfs kernel code. >> The two ways I have thought of are I/O,, probably ioctl's or a new system >> call. >> >> I assume that it is possible, using a module to add an entry to the >> syscall table, but I lean more towards a new pseudo device to hang the >> ioctl's off of. >> >> I would be ineterested in any comments. I would also like to hear from >> anybody who has thought about a userfs implementation for FreeBSD. > > Both the Arla and Coda file systems are distributed file systems > managed from userland processes. They do this by providing loadable > kernel modules (or static compiled in code) to allow userland > processes to listen on a device file (/dev/xfsX for Arla, /dev/codaX > for Coda) and receive "upcalls" from the kernel. Usually, the > userland process is multi-threaded so that it can handle in parallel > requests that might involve extensive waiting (for example, > transfering a large file using a network file system). Before > writing something from scratch, you may want to investigate these > and see if they meet your needs. Coda is available in the base > FreeBSD distribution, and Arla as a package. I know that the Arla > project has put significant work into making their code useful > beyond just an AFS client with the intent in mind that other > developers could use the module for other file systems. I built a > fair portion of a caching file system based on the Arla kernel > module at one point. Hmm. A kld runs in kernel context, not user context. Sure, it's easier to load than rebuilding a kernel, and I believe klds are the correct approach to added kernel functionality, but it doesn't offer one of the prime advantages of userland development: if your program crashes, your program crashes, not the system. If you're developing a kld, a bug can crash the system. Still, it might be an answer to my previous message about (now) missing block devices: if we want access to the block devices (which are still there, just no longer accessible from userland), maybe a kld is a good way of implementing them. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 22: 9:12 2000 Delivered-To: freebsd-fs@freebsd.org Received: from yana.lemis.com (yana.lemis.com [192.109.197.140]) by hub.freebsd.org (Postfix) with ESMTP id 5EFBA14EF9; Sat, 22 Jan 2000 22:08:46 -0800 (PST) (envelope-from grog@mojave.worldwide.lemis.com) Received: from mojave.worldwide.lemis.com (j39.klt31.jaring.my [161.142.169.233]) by yana.lemis.com (8.8.8/8.8.8) with ESMTP id QAA11622; Sun, 23 Jan 2000 16:37:28 +1030 (CST) (envelope-from grog@mojave.worldwide.lemis.com) Received: (from grog@localhost) by mojave.worldwide.lemis.com (8.9.3/8.9.3) id NAA00441; Sat, 22 Jan 2000 13:12:17 +0800 (SGT) (envelope-from grog) Date: Sat, 22 Jan 2000 13:12:17 +0800 From: Greg Lehey To: Brian Beattie Cc: Kelly Yancey , fs@FreeBSD.ORG, Poul-Henning Kamp Subject: Re: UDF, userfs Message-ID: <20000122131216.B391@mojave.worldwide.lemis.com> Reply-To: Greg Lehey References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from beattie@aracnet.com on Fri, Jan 21, 2000 at 02:21:58PM -0800 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Friday, 21 January 2000 at 14:21:58 -0800, Brian Beattie wrote: > On Fri, 21 Jan 2000, Kelly Yancey wrote: > >> On Fri, 21 Jan 2000, Brian Beattie wrote: >> >>> I have been thinking about the userfs implementation. I will need some >>> way for the user process to talk the backend of the userfs kernel code. >>> The two ways I have thought of are I/O,, probably ioctl's or a new system >>> call. >>> > ... >> Perhaps you have put more thought into it, but >> the concern that immediately comes to my mind is how to get VOP function >> calls from kernel-space to user-space. Some kind of up-call mechanism >> seems in order. You had suggested a pseudo-device, presumably with the >> intent to have daemons which implement file systems to read from to >> catch VOPs and write to to return data. > > Basicaly, the user process that implements the filesystem, would issue a > calls (i assume an ioctl on a pseudo device, but I am open to > suggestions). The userfs module would pass any VOP's back to the process, > or return an empty result. One could also implement select support. > (exact details still open) Well, in fact, you just need to interface directly to a block device. Oh. We don't have any block devices any more. phk, remember the discussion we had about this? I was saying "just because nobody can think of a good use for a block device right now doesn't mean that somebody won't come". Now Brian has come. Can we somehow resurrect a block device interface from userland? I don't think we should reintroduce block devices per se, but it would nice to have an ioctl call which says "use block device semantics for this file handle". What do you think? Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 23:53:19 2000 Delivered-To: freebsd-fs@freebsd.org Received: from assaris.sics.se (assaris.sics.se [193.10.66.234]) by hub.freebsd.org (Postfix) with ESMTP id 1E25C14D16 for ; Sat, 22 Jan 2000 23:53:05 -0800 (PST) (envelope-from assar@assaris.sics.se) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id IAA75875; Sun, 23 Jan 2000 08:52:52 +0100 (CET) (envelope-from assar) To: Greg Lehey Cc: Robert Watson , Brian Beattie , fs@FreeBSD.ORG Subject: Re: UDF, userfs References: <20000122131656.C391@mojave.worldwide.lemis.com> From: Assar Westerlund Date: 23 Jan 2000 08:52:52 +0100 In-Reply-To: Greg Lehey's message of "Sat, 22 Jan 2000 13:16:57 +0800" Message-ID: <5liu0l1ay3.fsf@assaris.sics.se> Lines: 14 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Greg Lehey writes: > Hmm. A kld runs in kernel context, not user context. Sure, it's > easier to load than rebuilding a kernel, and I believe klds are the > correct approach to added kernel functionality, but it doesn't offer > one of the prime advantages of userland development: if your program > crashes, your program crashes, not the system. If you're developing a > kld, a bug can crash the system. Yes, but both the Coda and the Arla kld are very simple and all the real work (and thus, the devlopment) takes part in the user space daemon. The kld is mostly there as a way of communicating with the kernel. /assar To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Jan 22 23:56:21 2000 Delivered-To: freebsd-fs@freebsd.org Received: from assaris.sics.se (assaris.sics.se [193.10.66.234]) by hub.freebsd.org (Postfix) with ESMTP id 0D64614D16 for ; Sat, 22 Jan 2000 23:56:14 -0800 (PST) (envelope-from assar@assaris.sics.se) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id IAA75883; Sun, 23 Jan 2000 08:56:18 +0100 (CET) (envelope-from assar) To: Robert Watson Cc: Brian Beattie , fs@FreeBSD.ORG Subject: Re: UDF, userfs References: From: Assar Westerlund Date: 23 Jan 2000 08:56:18 +0100 In-Reply-To: Robert Watson's message of "Fri, 21 Jan 2000 18:54:15 -0500 (EST)" Message-ID: <5l901hw7a5.fsf@assaris.sics.se> Lines: 13 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Robert Watson writes: > I know that the Arla project has put significant work into making > their code useful beyond just an AFS client with the intent in mind > that other developers could use the module for other file systems. > I built a fair portion of a caching file system based on the Arla > kernel module at one point. I would just like to add that even through we build xfs (the kernel module used by Arla) for use with Arla we did try to keep it general and I'm aware of several other file system implementors that have used or are using it as the way of implementing their file system. /assar To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message