From owner-freebsd-fs Sun Sep 13 00:02:03 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA28482 for freebsd-fs-outgoing; Sun, 13 Sep 1998 00:02:03 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA28471 for ; Sun, 13 Sep 1998 00:02:01 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id AAA01709 for ; Sun, 13 Sep 1998 00:01:44 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id AAA29180 for ; Sun, 13 Sep 1998 00:01:43 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id AAA22846 for freebsd-fs@freebsd.org; Sun, 13 Sep 1998 00:01:42 -0700 (PDT) Date: Sun, 13 Sep 1998 00:01:42 -0700 (PDT) From: Don Lewis Message-Id: <199809130701.AAA22846@salsa.gv.tsc.tdk.com> To: freebsd-fs@FreeBSD.ORG Subject: vm system interaction with nullfs Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Since the vm system keeps track of what it has in memory by (vnode, offset), how is this supposed to work when stackable filesystems are in use which create multiple vnodes for a single filesytem object, or is this broken? Unless this works right, it looks like you'll end up with multiple copies of the same disk blocks in memory and in memory copies may all be different. It would seem that in the case of nullfs and similar transparent filesytems, the vm system should always use the lowest vnode, but this doesn't seem to be implemented (though I could just be getting lost in the maze of twisty little passages). It's even messier if the layer isn't transparent, like an encryption layer. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 13 01:33:36 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id BAA02937 for freebsd-fs-outgoing; Sun, 13 Sep 1998 01:33:36 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from implode.root.com (implode.root.com [198.145.90.17]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id BAA02932 for ; Sun, 13 Sep 1998 01:33:32 -0700 (PDT) (envelope-from root@implode.root.com) Received: from implode.root.com (localhost [127.0.0.1]) by implode.root.com (8.8.5/8.8.5) with ESMTP id BAA19667; Sun, 13 Sep 1998 01:29:06 -0700 (PDT) Message-Id: <199809130829.BAA19667@implode.root.com> To: Don Lewis cc: freebsd-fs@FreeBSD.ORG Subject: Re: vm system interaction with nullfs In-reply-to: Your message of "Sun, 13 Sep 1998 00:01:42 PDT." <199809130701.AAA22846@salsa.gv.tsc.tdk.com> From: David Greenman Reply-To: dg@root.com Date: Sun, 13 Sep 1998 01:29:06 -0700 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >Since the vm system keeps track of what it has in memory by (vnode, offset), >how is this supposed to work when stackable filesystems are in use which >create multiple vnodes for a single filesytem object, or is this broken? >Unless this works right, it looks like you'll end up with multiple copies >of the same disk blocks in memory and in memory copies may all be different. > >It would seem that in the case of nullfs and similar transparent filesytems, >the vm system should always use the lowest vnode, but this doesn't seem to >be implemented (though I could just be getting lost in the maze of twisty >little passages). It's even messier if the layer isn't transparent, >like an encryption layer. Yes, that's the fundamental reason why nullfs is broken. It's been known for years that the solution is to always reference the same underlying VM object, but noone has yet gotten around to implementing that correctly. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 13 15:44:05 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA14986 for freebsd-fs-outgoing; Sun, 13 Sep 1998 15:44:05 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA14956 for ; Sun, 13 Sep 1998 15:43:59 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id PAA12862; Sun, 13 Sep 1998 15:43:45 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp03.primenet.com, id smtpd012847; Sun Sep 13 15:43:37 1998 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id PAA21022; Sun, 13 Sep 1998 15:43:34 -0700 (MST) From: Terry Lambert Message-Id: <199809132243.PAA21022@usr04.primenet.com> Subject: Re: vm system interaction with nullfs To: Don.Lewis@tsc.tdk.com (Don Lewis) Date: Sun, 13 Sep 1998 22:43:34 +0000 (GMT) Cc: freebsd-fs@FreeBSD.ORG In-Reply-To: <199809130701.AAA22846@salsa.gv.tsc.tdk.com> from "Don Lewis" at Sep 13, 98 00:01:42 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Since the vm system keeps track of what it has in memory by (vnode, offset), > how is this supposed to work when stackable filesystems are in use which > create multiple vnodes for a single filesytem object, or is this broken? Yes, this is still broken. This was the primary reason for the migration of a putpages/getpages into all "bottom-of-stack" FS's. The general fix is to create a "getfinalvp". This would allow you to page through an object, while allowing layers stacked on top to dictate layout/content. Another part of the puzzle is that the vnode locking is overly complex (VOP_LOCK). The specific remedy for this case is to move the locking code for top level VFS consumers (system calls, NFS server, etc. are all VFS consumers) into the common code; this resolves the recusion issues. For FS's that stack on top of other FS's, they must call the general locking code on the underlying vnodes. That is, it is the reponsibility of a VFS consumer to ensure the lock state before making VOP calls on the vnode(s). The last part is that operations that otherwise need specific knowledge of in core inode need to be divorced from that knowledge. For the most part, this means that advisory locks must be asserted on vnodes, not on in-core inodes. This thins down the code that you must duplicate going down, and makes it possible to proxy locks down. Again, the code coes into the VFS consumers. The typical implementation for terminal (bottom-of-stack) VFS's is to implement the code as a veto-based operation, and the terminal FS's never veto the operation. These steps, taken together, move most of the complexity of stacking into a single set of common routines, and make operations much thinner for stacking layers that don't implement them directly, but instead pass the operations down. > Unless this works right, it looks like you'll end up with multiple copies > of the same disk blocks in memory and in memory copies may all be different. It's worse than merely aliasing the the VM objects (in reality, the VM object associated with the vnode would be aliased if you used this code today, and the free order would be the inverse of the stacking order -- in other words, it would *always* get them wrong). The existance of multiple copies is less of a problem than the fact that you don't know the right backing object for a given piece of data in a stack (consider the case of a "quotafs", where quotas are implemented using a stacking layer instead of being an integrated feature of the FS). This is the point of the "getfinalvp" suggestion -- probably more properly "getbackingvp". > It would seem that in the case of nullfs and similar transparent filesytems, > the vm system should always use the lowest vnode, but this doesn't seem to > be implemented (though I could just be getting lost in the maze of twisty > little passages). It's even messier if the layer isn't transparent, > like an encryption layer. Not in all cases, actually. For a cryptographic FS (such as the one one of John Heidemann's students wrote, and John sent me), you will want a backing object for the unencrypted data, seperate from the backing object of the on-disk data. There are also cases where the in-core data and the backing data aren't the same size. For some of these (like a compression layer), you would implement this via the comperssion layer's vp's get/putpages, and not operate on the backing store directly. One could also imagine an "attributed" FS, where the files have "attribute binary" or "attribute text + character set". For example, if you were to use a Unicode representation of an NFS mounted legacy filesystem, you might want to attribute text files on the basis of the remote character set (e.g., ISO 8859-2), and do a two-for-one page expansion of teh data for it to be locally usable. In any of these cases, the "getbackingvp" would return an object higher up in the stack that the actual backing object, and it would have vm pages that it would fill from the underlying backing vp using the get/putpages of the vp for the actual backing object. Really, you should go to ftp.cs.ucla.edu, and read up on the stacking architecture. The documents in the "ficus" directory are the actual design documents for the BSD 4.4 stacking architecture. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Sep 15 03:03:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id DAA02092 for freebsd-fs-outgoing; Tue, 15 Sep 1998 03:03:34 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from wipinfo.soft.net (agni.wipinfo.soft.net [164.164.6.20]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id DAA02056 for ; Tue, 15 Sep 1998 03:02:54 -0700 (PDT) (envelope-from hiren@tagore.wipinfo.soft.net) Received: from tagore.wipinfo.soft.net by wipinfo.soft.net (SMI-8.6/SMI-SVR4) id SAA02224; Mon, 14 Sep 1998 18:46:23 -0500 Date: Mon, 14 Sep 1998 18:57:32 +0530 (IST) From: Hiren Mehta X-Sender: hiren@tagore To: freebsd-fs@FreeBSD.ORG Subject: longfile to short file name conversion algo for msdosfs Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi All, Is there any algorithm which converts UNIX longfilenames to MS-DOS short filenames (8+3 chars) instead of truncating the filename ? Thanks Hiren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Sep 15 07:54:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA23050 for freebsd-fs-outgoing; Tue, 15 Sep 1998 07:54:34 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pau-amma.whistle.com (s205m64.whistle.com [207.76.205.64]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA23026 for ; Tue, 15 Sep 1998 07:54:16 -0700 (PDT) (envelope-from dhw@whistle.com) Received: (from dhw@localhost) by pau-amma.whistle.com (8.8.8/8.8.7) id HAA04876; Tue, 15 Sep 1998 07:53:49 -0700 (PDT) (envelope-from dhw) Date: Tue, 15 Sep 1998 07:53:49 -0700 (PDT) From: David Wolfskill Message-Id: <199809151453.HAA04876@pau-amma.whistle.com> To: freebsd-fs@FreeBSD.ORG, hiren@tagore.wipinfo.soft.net Subject: Re: longfile to short file name conversion algo for msdosfs In-Reply-To: Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Date: Mon, 14 Sep 1998 18:57:32 +0530 (IST) >From: Hiren Mehta >Is there any algorithm which converts UNIX longfilenames to >MS-DOS short filenames (8+3 chars) instead of truncating the filename ? Several algorithms exist, but because of the nature of the transform, you're going to lose information. Thus, where you didn't have name collisions in UNIX, you could have them in the 8.3 world. So you need to decide how you want to handle collisions. And "truncating the filename" is a valid algorithm, as well. david -- David Wolfskill UNIX System Administrator dhw@whistle.com voice: (650) 577-7158 pager: (650) 371-4621 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Sep 15 12:59:52 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA19409 for freebsd-fs-outgoing; Tue, 15 Sep 1998 12:59:52 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA19401 for ; Tue, 15 Sep 1998 12:59:42 -0700 (PDT) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id MAA23727; Tue, 15 Sep 1998 12:59:22 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp01.primenet.com, id smtpd023687; Tue Sep 15 12:59:19 1998 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id MAA22309; Tue, 15 Sep 1998 12:59:16 -0700 (MST) From: Terry Lambert Message-Id: <199809151959.MAA22309@usr09.primenet.com> Subject: Re: longfile to short file name conversion algo for msdosfs To: hiren@tagore.wipinfo.soft.net (Hiren Mehta) Date: Tue, 15 Sep 1998 19:59:15 +0000 (GMT) Cc: freebsd-fs@FreeBSD.ORG In-Reply-To: from "Hiren Mehta" at Sep 14, 98 06:57:32 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Is there any algorithm which converts UNIX longfilenames to > MS-DOS short filenames (8+3 chars) instead of truncating the filename ? Yes. See the WIN95IFS.DOC file that came with the MSDN Level II DDK; it enumerates the full algorithm used by Microsoft's IFS code in Windows95 for VFAT/VFAT32. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Sep 17 07:08:58 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA12102 for freebsd-fs-outgoing; Thu, 17 Sep 1998 07:08:58 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from friley-186-113.res.iastate.edu (friley-186-113.res.iastate.edu [129.186.186.113]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA12097 for ; Thu, 17 Sep 1998 07:08:56 -0700 (PDT) (envelope-from mystify@friley-186-113.res.iastate.edu) Received: from friley-186-113.res.iastate.edu (loopback [127.0.0.1]) by friley-186-113.res.iastate.edu (8.9.1/8.8.8) with ESMTP id JAA27000 for ; Thu, 17 Sep 1998 09:08:31 -0500 (CDT) (envelope-from mystify@friley-186-113.res.iastate.edu) Message-Id: <199809171408.JAA27000@friley-186-113.res.iastate.edu> To: freebsd-fs@FreeBSD.ORG Subject: Alleged "read-only" partition preventing ccd Date: Thu, 17 Sep 1998 09:08:30 -0500 From: Patrick Hartling Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This summer, the boot disk on my machine died leaving me stuck for a way to boot (obviously). Since I have two other disks, I wasn't totally out of luck. The only solution I had was to break up my mirrored CCD and use a partition from it for / so I could get going again. I have since gotten a replacement for the dead disk and want to go back to having a mirrored CCD. Unfortunately, when I try to dd the existing partition to the mirroring location (raw device to raw device) in single-user mode, dd(1) only writes 512 bytes and then claims that the destination partition is read-only. However, I can write a new file system to that partition, mount it, create files, etc. I can even dd the character devices, but it takes about half an hour and then the CCD won't mount. If it would help, I can do that to get the exact error message, but I do not have time right now. Is there something that can be done to remove the read-only restriction on this partition? And for that matter, how did it get there in the first place? -Patrick Patrick L. Hartling | Research Assistant, ICEMT mystify@friley-186-113.res.iastate.edu | SE Lab - 1117 Black Engineering http://www.public.iastate.edu/~oz/ | http://www.icemt.iastate.edu/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Sep 17 09:40:23 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA08775 for freebsd-fs-outgoing; Thu, 17 Sep 1998 09:40:23 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from godzilla.zeta.org.au (godzilla.zeta.org.au [203.15.68.22]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA08708 for ; Thu, 17 Sep 1998 09:40:01 -0700 (PDT) (envelope-from bde@godzilla.zeta.org.au) Received: (from bde@localhost) by godzilla.zeta.org.au (8.8.7/8.8.7) id CAA20799; Fri, 18 Sep 1998 02:39:38 +1000 Date: Fri, 18 Sep 1998 02:39:38 +1000 From: Bruce Evans Message-Id: <199809171639.CAA20799@godzilla.zeta.org.au> To: freebsd-fs@FreeBSD.ORG, mystify@friley-186-113.res.iastate.edu Subject: Re: Alleged "read-only" partition preventing ccd Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Unfortunately, when I try to dd the existing partition to the mirroring >location (raw device to raw device) in single-user mode, dd(1) only writes >512 bytes and then claims that the destination partition is read-only. This is because the disk label at offset 512 is write protected. >However, I can write a new file system to that partition, mount it, create >files, etc. I can even dd the character devices, but it takes about half an >hour and then the CCD won't mount. If it would help, I can do that to get >the exact error message, but I do not have time right now. Correct error handling is not possible for writes to block devices. The i/o gets reblocked to a too-small block size of 2K (that's why block devices are so slow). Write errors are essentially ignored. There is a write error for the first 2K block and none of this block is copied. This probably breaks mounting later. Block devices should only be used for mounting unless you know about problems like this and either want them or don't care. >Is there something that can be done to remove the read-only restriction on >this partition? And for that matter, how did it get there in the first >place? The DIOCWLABEL ioctl controls the write protection flag. dd'ing to an unlabeled disk should work. There is no easy way to clear a label. I usually use a binary editor on the whole disk device. This exploits brokenness of label write protection on the whole disk device. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Sep 17 09:48:49 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA10612 for freebsd-fs-outgoing; Thu, 17 Sep 1998 09:48:49 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from friley-186-113.res.iastate.edu (friley-186-113.res.iastate.edu [129.186.186.113]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA10603 for ; Thu, 17 Sep 1998 09:48:43 -0700 (PDT) (envelope-from mystify@friley-186-113.res.iastate.edu) Received: from friley-186-113.res.iastate.edu (loopback [127.0.0.1]) by friley-186-113.res.iastate.edu (8.9.1/8.8.8) with ESMTP id LAA07794 for ; Thu, 17 Sep 1998 11:48:19 -0500 (CDT) (envelope-from mystify@friley-186-113.res.iastate.edu) Message-Id: <199809171648.LAA07794@friley-186-113.res.iastate.edu> cc: freebsd-fs@FreeBSD.ORG Subject: Re: Alleged "read-only" partition preventing ccd In-reply-to: Message from Patrick Hartling of "Thu, 17 Sep 1998 09:08:30 CDT." <199809171408.JAA27000@friley-186-113.res.iastate.edu> Date: Thu, 17 Sep 1998 11:48:19 -0500 From: Patrick Hartling Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Patrick Hartling wrote: } Unfortunately, when I try to dd the existing partition to the mirroring } location (raw device to raw device) in single-user mode, dd(1) only writes } 512 bytes and then claims that the destination partition is read-only. } However, I can write a new file system to that partition, mount it, create } files, etc. I can even dd the character devices, but it takes about half an ^^^^^^^^^ | Argh, this should say "block" ------+ } hour and then the CCD won't mount. If it would help, I can do that to get } the exact error message, but I do not have time right now. Patrick L. Hartling | Research Assistant, ICEMT mystify@friley-186-113.res.iastate.edu | SE Lab - 1117 Black Engineering http://www.public.iastate.edu/~oz/ | http://www.icemt.iastate.edu/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message