From owner-freebsd-fs Tue Feb 27 14:14:41 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id OAA11359 for fs-outgoing; Tue, 27 Feb 1996 14:14:41 -0800 (PST) Received: from seraglio.staidan.qld.edu.au (staidans.client.uq.edu.au [130.102.39.106]) by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id OAA11334 for ; Tue, 27 Feb 1996 14:14:28 -0800 (PST) Received: from aidan.staidan.qld.edu.au (aidan.staidan.qld.edu.au [203.12.39.2]) by seraglio.staidan.qld.edu.au (8.6.12/8.6.12) with ESMTP id IAA22892; Wed, 28 Feb 1996 08:12:22 +1000 Received: from AIDAN/SpoolDir by aidan.staidan.qld.edu.au (Mercury 1.21); 28 Feb 96 08:12:35 -1000 Received: from SpoolDir by AIDAN (Mercury 1.21); 28 Feb 96 08:12:20 -1000 From: "Peter Stubbs" Organization: St Aidan's A.G.S. To: Peter van Heusden , freebsd-fs@freebsd.org Date: Wed, 28 Feb 1996 08:12:10 -1000 Subject: Re: Compressing filesystem for FreeBSD Priority: normal X-mailer: Pegasus Mail for Windows (v2.23) Message-ID: <6BD18783D77@aidan.staidan.qld.edu.au> Sender: owner-fs@freebsd.org Precedence: bulk On 24 Feb 96, Peter van Heusden wrote: > > Does anyone know if anyone has done any work on writing a > compressing filesystem for FreeBSD? I've been thinking about this, > and the strategies to do it, and I'd like to know if there is any > previous work out there I can refer to... > This isn't much of an answer, but since nobody else has said anything I'll point out that the the nearest thing that's here now is the pseudo-device gzip which allows gziped a.outs to be run. Perhaps all that is needed is to extend that device to other files? I'd hate to see us go the same direction as MS-DOG and create a single file with all the other files compressed into it. Perhaps the netware 4 technique of compressing files that haven't been access for a certain time using a (very) niced daemon? Cheers, Peter Peter Stubbs, St Aidan's AGS. ph +61-07-3379-9911, fax +61-07-3379-9432 From owner-freebsd-fs Tue Feb 27 18:20:15 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id SAA06684 for fs-outgoing; Tue, 27 Feb 1996 18:20:15 -0800 (PST) Received: from virginia.edu (mars.itc.Virginia.EDU [128.143.2.9]) by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id SAA06522 for ; Tue, 27 Feb 1996 18:19:55 -0800 (PST) Received: from archive.cs.virginia.edu by mail.virginia.edu id aa27636; 27 Feb 96 21:08 EST Received: from stretch.cs.Virginia.edu (atf3r@stretch-fo.cs.Virginia.EDU [128.143.136.14]) by archive.cs.Virginia.EDU (8.7.1/8.6.6) with SMTP id VAA14870; Tue, 27 Feb 1996 21:07:50 -0500 (EST) Received: by stretch.cs.Virginia.edu (4.1/SMI-2.0) id AA29786; Tue, 27 Feb 96 21:07:48 EST Date: Tue, 27 Feb 1996 21:07:45 -0500 (EST) From: "Adrian T. Filipi-Martin" Reply-To: adrian@virginia.edu To: Peter Stubbs Cc: Peter van Heusden , freebsd-fs@freebsd.org Subject: Re: Compressing filesystem for FreeBSD In-Reply-To: <6BD18783D77@aidan.staidan.qld.edu.au> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-fs@freebsd.org Precedence: bulk On Wed, 28 Feb 1996, Peter Stubbs wrote: > On 24 Feb 96, Peter van Heusden wrote: > > > > > Does anyone know if anyone has done any work on writing a > > compressing filesystem for FreeBSD? I've been thinking about this, > > and the strategies to do it, and I'd like to know if there is any > > previous work out there I can refer to... > > > This isn't much of an answer, but since nobody else has said > anything I'll point out that the the nearest thing that's here now is the > pseudo-device gzip which allows gziped a.outs to be run. Perhaps all > that is needed is to extend that device to other files? > > I'd hate to see us go the same direction as MS-DOG and create a > single file with all the other files compressed into it. Perhaps the > netware 4 technique of compressing files that haven't been access for > a certain time using a (very) niced daemon? The real problem with compressing a file system that is often forgotten is that you are increasing the cost in CPU cycles for a disk I/O. While I have had dos/linux enthusiasts go on about how compressing your filesystem can both improve I/O bandwidth and increase available disk space, the cost is often fogotten. On a multi-user system, the CPU is not idle during disk I/O. I think there would also be problems with paging and reads in general. Which block of the disk contains the nth byte? In short, I don't see it being worth the effort. Adrian System Administrator for the NVL, NIIMS and Telemedicine labs adrian@virginia.edu ---->>>>| Support your local programmer, http://www.cs.virginia.edu/~atf3r/ --->>>| STOP Software Patent Abuses NOW! Member: The League for -->>| For an application and information Programming Freedom ->| see: http://www.lpf.org/ From owner-freebsd-fs Tue Feb 27 21:39:08 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id VAA12929 for fs-outgoing; Tue, 27 Feb 1996 21:39:08 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [192.216.222.3]) by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id VAA12924 for ; Tue, 27 Feb 1996 21:39:06 -0800 (PST) Received: from rocky.sri.MT.net ([204.182.243.10]) by who.cdrom.com (8.6.12/8.6.11) with ESMTP id VAA22237 for ; Tue, 27 Feb 1996 21:39:04 -0800 Received: (from nate@localhost) by rocky.sri.MT.net (8.6.12/8.6.12) id WAA06665; Tue, 27 Feb 1996 22:40:23 -0700 Date: Tue, 27 Feb 1996 22:40:23 -0700 From: Nate Williams Message-Id: <199602280540.WAA06665@rocky.sri.MT.net> To: adrian@virginia.edu Cc: Peter Stubbs , Peter van Heusden , freebsd-fs@freebsd.org Subject: Re: Compressing filesystem for FreeBSD In-Reply-To: References: <6BD18783D77@aidan.staidan.qld.edu.au> Sender: owner-fs@freebsd.org Precedence: bulk > > The real problem with compressing a file system that is often > forgotten is that you are increasing the cost in CPU cycles for a disk > I/O. I don't think it's at all forgotten. I think that folks are willing to take the performance hit for the decrease in disk space for files that are rarely accessed. > While I have had dos/linux enthusiasts go on about how compressing > your filesystem can both improve I/O bandwidth and increase available disk > space, the cost is often fogotten. While I do know that compressed FS often do increase I/O under DOS due to stupid disk layouts, I haven't heard many 'compression' proponents claim that it increases I/O on real FS. > On a multi-user system, the CPU is not idle during disk I/O. Depends. On my SCSI system it might be, depending on the load. > I think there would also be problems with paging and reads in general. > Which block of the disk contains the nth byte? In short, I don't see > it being worth the effort. On my laptop it would *certainly* be worth the effort. I've got cycles to burn since I'm the only user, and I'm more than willing to get free disk space for my cycles. On my multi-user system, I'm more than willing to take the performance hit for our local archives which are rarely accessed by take up a huge amount of space. I think most folks who are serious about this understand the costs, and are willing to pay them for certain applications. Nate From owner-freebsd-fs Tue Feb 27 21:41:16 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id VAA12990 for fs-outgoing; Tue, 27 Feb 1996 21:41:16 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [192.216.222.3]) by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id VAA12985 for ; Tue, 27 Feb 1996 21:41:13 -0800 (PST) Received: from crh.cl.msu.edu ([35.8.1.24]) by who.cdrom.com (8.6.12/8.6.11) with ESMTP id VAA22275 for ; Tue, 27 Feb 1996 21:41:13 -0800 Received: (from henrich@localhost) by crh.cl.msu.edu (8.6.12/8.6.12) id AAA02027; Wed, 28 Feb 1996 00:39:50 -0500 Date: Wed, 28 Feb 1996 00:39:50 -0500 From: Charles Henrich Message-Id: <199602280539.AAA02027@crh.cl.msu.edu> To: atf3r@stretch.cs.virginia.edu, freebsd-fs@freebsd.org Subject: Re: Compressing filesystem for FreeBSD Newsgroups: lists.freebsd.fs References: <4h0fhq$c40@msunews.cl.msu.edu> X-Newsreader: NN version 6.5.0 #3 (NOV) Sender: owner-fs@freebsd.org Precedence: bulk In lists.freebsd.fs you write: > The real problem with compressing a file system that is often >forgotten is that you are increasing the cost in CPU cycles for a disk >I/O. While I have had dos/linux enthusiasts go on about how compressing >your filesystem can both improve I/O bandwidth and increase available disk >space, the cost is often fogotten. On a multi-user system, the CPU is not >idle during disk I/O. I think there would also be problems with paging >and reads in general. Which block of the disk contains the nth byte? In >short, I don't see it being worth the effort. In real life however, huge numbers of peoples CPU's are idle 99% of the time. I know for one that my P100 is idling along doing not a damn thing for over 50% of the time. If I can use that CPU for something productive, Im all for it. -Crh -- Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From owner-freebsd-fs Thu Feb 29 19:19:10 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id TAA06969 for fs-outgoing; Thu, 29 Feb 1996 19:19:10 -0800 (PST) Received: from ucthpx.uct.ac.za ([137.158.128.1]) by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id TAA06918 for ; Thu, 29 Feb 1996 19:18:43 -0800 (PST) Received: by ucthpx.uct.ac.za (Smail3.1.28.1 #47) id m0tsLMN-000KMYC; Fri, 1 Mar 96 05:18 SAST Received: from leftside by leftside.its.uct.ac.za with smtp (Smail3.1.29.1 #1) id m0tsBrC-00004HC; Thu, 29 Feb 96 19:09 SAT Date: Thu, 29 Feb 1996 19:09:33 +0200 (SAT) From: Peter van Heusden X-Sender: pvh@leftside To: Peter Stubbs cc: freebsd-fs@freebsd.org Subject: Re: Compressing filesystem for FreeBSD In-Reply-To: <6BD18783D77@aidan.staidan.qld.edu.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-fs@freebsd.org Precedence: bulk On Wed, 28 Feb 1996, Peter Stubbs wrote: > On 24 Feb 96, Peter van Heusden wrote: > > > > > Does anyone know if anyone has done any work on writing a > > compressing filesystem for FreeBSD? I've been thinking about this, > > and the strategies to do it, and I'd like to know if there is any > > previous work out there I can refer to... > > > This isn't much of an answer, but since nobody else has said > anything I'll point out that the the nearest thing that's here now is the > pseudo-device gzip which allows gziped a.outs to be run. Perhaps all > that is needed is to extend that device to other files? Hm. I'll have a look at it... > > I'd hate to see us go the same direction as MS-DOG and create a > single file with all the other files compressed into it. Perhaps the > netware 4 technique of compressing files that haven't been access for > a certain time using a (very) niced daemon? That's what I was thinking. Basically, you have two places where you can interface, imo: 1) At the block level, when a block is written/read from disk. Now, someone on hackers just mentioned that that might cause problems with efficiency - as I recall, down there things are meant to work on nice aligned boundaries, and things. 2) At the file level, when a file is read/written. Netware 4, as you mention, does a mark 'n sweepish compress, and then a decompress when you access a compressed file. This looks kinda simpler. I haven't looked deep into the FS code to decide whether it is simple or not, though. As for the MSDOG idea of keeping things in a single file - that's what we got filesystems for, no? My initial idea for a compressing FS was that I would like /usr/src to be a compressing FS, but /usr/bin shouldn't be. Since you're working with a real OS with a real FS, you don't need to wonder about the psuedo-FS garbage MSDOG loads you with... *but* one thing to watch out for on a compressing FS is the integrity of your data. You don't want to lose a whole file because a single block got screwed or something. Hm. Anyway, it sounds like I should got the drawing board for the while and see if anything comes up. Peter From owner-freebsd-fs Sat Mar 2 15:57:51 1996 Return-Path: owner-fs Received: (from root@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id PAA26102 for fs-outgoing; Sat, 2 Mar 1996 15:57:51 -0800 (PST) Received: (from hsu@localhost) by freefall.freebsd.org (8.7.3/8.7.3) id PAA26095 for freebsd-fs; Sat, 2 Mar 1996 15:57:49 -0800 (PST) Date: Sat, 2 Mar 1996 15:57:49 -0800 (PST) From: Jeffrey Hsu Message-Id: <199603022357.PAA26095@freefall.freebsd.org> To: freebsd-fs Subject: cryptographic filesystem Sender: owner-fs@FreeBSD.ORG Precedence: bulk Here's a pointer to CFS in case someone is interested. There's a BSDI port, so it shouldn't be too hard to get it working under FreeBSD. ------------------------------------------------------------------- >From cfs@research.att.com Sat Mar 2 01:01:57 PST 1996 Article: 18068 of comp.archives From: Matt Blaze Subject: [sci.crypt] New release (v.1.3.2) of CFS Unix encrypting file system available Date: 30 Dec 1995 19:00:33 GMT Organization: {AT&T | Bell} Laboratories Research Lines: 125 Reply-To: cfs@research.att.com Source code for the latest version (release 1.3.2) of CFS, the Cryptographic File System, is now available upon request for research and experimental use in the US and Canada. CFS pushes encryption services into the Unix(tm) file system. It supports secure storage at the system level through a standard Unix file system interface to encrypted files. Users associate a cryptographic key with the directories they wish to protect. Files in these directories (as well as their pathname components) are transparently encrypted and decrypted with the specified key without further user intervention; cleartext is never stored on a disk or sent to a remote file server. CFS employs a novel combination of DES stream and codebook cipher modes to provide high security with good performance on a modern workstation. CFS can use any available file system for its underlying storage without modification, including remote file servers such as NFS. System management functions, such as file backup, work in a normal manner and without knowledge of the key. CFS runs under SunOS and several other BSD-derived systems with NFS. It is implemented entirely at user level, as a local NFS server running on the client machine's "loopback" interface. It consists of about 5000 lines of code and supporting documentation. You must have "root" access to install CFS. CFS was first mentioned at the work-in-progress session at the Winter '93 USENIX Conference and was more fully detailed in: Matt Blaze. "A Cryptographic File System for Unix", Proc. 1st ACM Conference on Computer and Communications Security, Fairfax, VA, November 1993. (PostScript available by anonymous ftp from research.att.com in the file dist/mab/cfs.ps.) and in Matt Blaze. "Key Management in an Encrypting File System", Proc. Summer '94 USENIX Tech. Conference, Boston, MA, June 1994. (PostScript available by anonymous ftp from research.att.com in the file dist/mab/cfskey.ps.) Version 1.3 of CFS also includes ESM, the Encrypting Session Manager. ESM provides shell-to-shell encrypted sessions across insecure links and requires no OS or network support. It is useful for typing cfs passphrases when logged in over the network. ESM needs RSAREF 2.0 to compile and is tested only on SunOS and BSDI. ESM is the first released part of a suite of session encryption tools that are described in Matt Blaze and Steve Bellovin. "Session-layer Encryption." Proc. 1995 USENIX Security Workshop, Salt Lake City, June 1995. (PostScript is available from ftp://research.att.com/dist/mab/sesscrypt.ps) The new version of CFS differs from the version described in the papers in a few ways: * The DES-based encryption scheme has been strengthened, and now provides greater security but with the online latency of only single-DES. * Support for the smartcard-based key management system is not included and a few of the tools are not included. * An impoved key management scheme now allows chaning the passphrase associated with a directory. * The performance has been improved. * The security of the system against certain non-cryptanalytic attacks has been improved somewhat. * User-contributed ports to a number of additional platforms. * Hooks for adding new ciphers. * 3-DES, MacGuffin, and SAFER-SK128 encryption options. * Timeout options allow automatic detach of encrypted directories after a set time or period of inactivity. CFS is distributed as a research prototype; it is COMPLETELY UNSUPPORTED software. No warranty of any kind is provided. We will not be responsible if the system deletes all your files and emails the cleartext directly to the NSA or your mother. Also, we do not have the resources to port the software to other platforms, although you are welcome to do this yourself. The software was developed under SunOS and BSDI, and there are also unsupported user-contributed ports available for AIX, HP/UX, Irix, Linux, Solaris and Ultrix. We really can't promise to provide any technical support at all, beyond the source code itself. We also maintain a mailing list for CFS users and developers; subscription information is included with the source code. Because of export restrictions on cryptographic software, we are only able to make the software available within the US and Canada to US and Canadian citizens and permanent residents. Unfortunately, we cannot make it available for general anonymous ftp or other uncontrolled access, nor can we allow others to do so. Sorry. Legal stuff from the README file: * Copyright (c) 1992, 1993, 1994, 1995 by AT&T. * Permission to use, copy, and modify this software without fee * is hereby granted, provided that this entire notice is included in * all copies of any software which is or includes a copy or * modification of this software and in all copies of the supporting * documentation for such software. * * This software is subject to United States export controls. * * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR AT&T MAKE ANY * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. If you would like a copy of the CFS source code, please read to the end of this message and then send email to: cfs@research.att.com DO NOT REPLY DIRECTLY TO THIS MESSAGE. You must include a statement that you are in the US or Canada, are a citizen or legal permanent resident of the US or Canada, and have read and understand the license conditions stated above. Be sure to include an email address in a US- or Canada-registered domain. The code will be sent to you via email in a "shar" shell archive (a little over 300K bytes long).