From owner-freebsd-afs Sun Jun 28 14:20:54 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA10402 for freebsd-afs-outgoing; Sun, 28 Jun 1998 14:20:54 -0700 (PDT) (envelope-from owner-freebsd-afs@FreeBSD.ORG) Received: from fledge.watson.org (root@COPLAND.CODA.CS.CMU.EDU [128.2.222.48]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA10397 for ; Sun, 28 Jun 1998 14:20:52 -0700 (PDT) (envelope-from robert@cyrus.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.8.8/8.8.8) with SMTP id RAA05286 for ; Sun, 28 Jun 1998 17:20:51 -0400 (EDT) Date: Sun, 28 Jun 1998 17:20:51 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org Reply-To: Robert Watson To: freebsd-afs@FreeBSD.ORG Subject: Coda Network Computer (fwd) Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-afs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org It can be done under Linux -- I am pretty sure this is doable under FreeBSD, but have not had time to try it. Given the recent mention of Coda in the FreeBSD newsletter, etc, I think this would prove an interesting (and hopefully gratifying) project for anyone out there who is interested. Also, on the AFS front -- I recently posted an announcement to freebsd-security and freebsd-hackers concerning a lkm PAG implementation under FreeBSD. This is certainly useful in the context of AFS in all the environments I work in (non-FreeBSD) -- perhaps this would be useful for any FreeBSD port? Robert N Watson Carnegie Mellon University http://www.cmu.edu/ TIS Labs at Network Associates, Inc. http://www.tis.com/ SafePort Network Services http://www.safeport.com/ robert@fledge.watson.org http://www.watson.org/~robert/ ---------- Forwarded message ---------- Date: Sun, 28 Jun 1998 17:07:57 -0400 (EDT) From: Michael Callahan Reply-To: linux-coda@TELEMANN.coda.cs.cmu.edu To: linux-coda@TELEMANN.coda.cs.cmu.edu Cc: sanm@corelcomputer.com Subject: Coda Network Computer Resent-Date: Sun, 28 Jun 1998 17:08:31 -0400 Resent-From: linux-coda@TELEMANN.coda.cs.cmu.edu The Coda Network Computer: A Guide for Manufacturers ---------------------------------------------------- Peter and I decided to waste some time on a fun hack we've been talking about: booting a Linux machine with root in /coda! The idea is that one could have a machine that had an ext2 root filesystem that contained just enough to start up networking, start venus, and then chroot into /coda. To be precise, all of this should be done within a shell script running as init (pid 1); the last step would be for this shell script to do exec chroot /coda/linuxroot /sbin/init The result would be that you could put a massive venus cache on the client, and so have the benefit of local-disk access to all your system binaries, yet have the Coda benefit of having essentially no local data, instead having all systems' complete configurations in Coda. Ultimately, of course, the idea would be that we could clone root directories within /coda, so that the process of setting up a new machine would consist simply of loading this small venus-supporting root filesystem, rebooting, and letting a hoard walk pull system binaries into cache. We're not going to be able to complete this project, but we've done some crucial steps and hope that someone would like to take over. What we've done: 1) We did a minimal Red Hat 5.1 install into a small partition on one of our machines. Then we copied this entire heirarchy into /coda/linuxroot. This copy completes successfully, but it does strip out the device files, which are not currently supported by the Coda filesystem (see below!). This was about 100MB of stuff--minimum install possible on Red Hat 5.1 without selecting packages individually; frankly, a bit large for our taste! It does take a while to copy. 2) The first fun thing to do once one has this is chroot /coda/linuxroot /bin/bash This first chroots into the /coda filesystem, and then execs the bin/bash found there (/coda/linuxroot/bin/bash). You must do this as root. Meanwhile, if you run codacon in another window, you'll see the various libraries that bash uses (there are a lot!) loading into your minicache. Eventually, you will get a shell prompt. You can now try typing ls, vi, perl, etc -- all of which will be running entirely using files within the /coda/linuxroot root filesystem! This all worked fine for us. 3) The next issue to tackle is how to make device files available to our chrooted environment. Recall again that Coda does not currently allow you to do mknod within /coda. Until Coda has this support, the solution will be to make /coda/linuxroot/dev a mount point for another, non-Coda filesystem that holds the device files. Our approach was to use a loopback block device with an ext2 filesystem image. To create the image, we did this in a shell that is not chrooted: dd if=/dev/zero of=/tmp/img bs=4k count=1000 losetup /dev/loop0 /tmp/img mke2fs -i 1024 /dev/loop0 mount /dev/loop0 /mnt/loop cd /dev find . | cpio -p -m -d --verbose /mnt/loop umount /mnt/loop Now we can try mounting this image within the Coda filesystem: mount /dev/loop0 /coda/linuxroot/dev This definitely doesn't work under 2.0 kernels; there, it returns "device busy". This is due to an interaction between the Coda minicache and the mount system call. However, on 2.1, this command worked right away. (Does it work on BSD? We don't know.) So for this project, you should definitely use a 2.1 kernel (we recommend Coda 4.6.0 (linux-coda-4.6.0.tgz) with Linux 2.1.107). We thought this all looked very encouraging, and we think it would be a lot of fun to go the rest of the way, but this is as far as we could get in the very short amount of time we had to play with it. Here's what we envisage as the next steps: 4) The next step is tackle booting. The target here is to have a system that can get to the point of starting to run /coda/linuxroot/sbin/init as process ID 1. Achieving this will be a major milestone, even if the init process immediately gets confused! What you're aiming for is a shell script that can run as /sbin/init in the "hard disk root" (the one that will become invisible after the real system gets going from /coda/linuxroot with its init). This script will need to get the system going to the point where venus can run, and then starting the init from within /coda/linuxroot. Essentially, this means: - fscking/remounting the hard disk / filesystem in read-write mode, - setting up networking, - starting venus, - waiting for venus to mount /coda, - mounting /coda/linuxroot/dev, - (maybe) mouting a local tmp partition on /coda/linuxroot/tmp, and finally - running exec chroot /coda/linuxroot/sbin/init (Waiting for venus to mount /coda will probably require looping, polling /etc/mtab.) To get started on this, we recommend setting up a small partition with a minimal install of Linux (we would re-use the install that we copied into /coda). First, change this system so that it runs /bin/bash as its /sbin/init: i.e., it boots to a shell. (Note that booting in single-user mode does a lot more already than this, so it's not equivalent.) Now you can analyze /etc/inittab and the rc scripts to figure out what are the key things that your script will need to do to make venus work. We've given some of this list above, but we could have missed something. Using the boot-up shell, you can try setting things up manually, gradually copying commands into a shell script. Bear in mind that you won't have update running, so it will be vital to use 'sync' to keep the disk up-to-date. Actually, the issue of what to do about update is quite important and needs investigation. One argument is that the hard disk /sbin/init should start update so that, if venus crashes and the boot does not complete, you at least have the venus cache files synced to disk because update is, at least, already running. On the other hand, the /coda init will, unless you change it, expect to run update itself, and this may fail if there is an update process already running. For debugging, we strongly recommend creating a couple virtual consoles, one with a bash running in the hard disk universe, one with codacon running. 5) Once you can get /coda/linuxroot/sbin/init to start, all that remains is to fix all problems that develop in running all programs that people run in Linux, under Coda! In particular, there are several issues we think may arise: - suid: Coda supports suid files, but this capability hasn't been tested or used for a long time. Does it work? - named pipes and other non-file filesystem objects: various programs (lpd, syslog, X) use non-file objects that appear in the filesystem, like named pipes, to do IPC. Coda currently does not support this. - Coda protection model: the Coda protection model is based on acls on directories and tokens for users. It will require a bit of tinkering to permit root to have the correct access tokens to permit it to do what it will want to do without jeopardizing the security of the installation on the server. - hardlinks crossing directories: Coda does support multiple hardlinks to a file provided all the hardlinks are in a single directory. It does not support hardlinks that cross directories. Is this a problem? - locking: probably /var/mail should not be shared across multiple Coda clients right now, because Coda's locking support is at best limitted to locking on one client. 6) Finally, of course, the real dream is to have many clients share essentially the same root filesystems except, perhaps, for a small number of config files. Doing this surely requires a lot of careful thought (which probably isn't very Coda-specific), but the specific advantage of doing this within Coda is that potentially you could support large clusters of workstations which would cache everything they need, providing local-disk performance with single-system manageability. The holy grail! We'd love to hear of any efforts to pursue this program and would be very happy to provide feedback. Michael Callahan Peter Braam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-afs" in the body of the message From owner-freebsd-afs Mon Jun 29 14:46:52 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA15894 for freebsd-afs-outgoing; Mon, 29 Jun 1998 14:46:52 -0700 (PDT) (envelope-from owner-freebsd-afs@FreeBSD.ORG) Received: from TELOS.ODYSSEY.CS.CMU.EDU (TELOS.ODYSSEY.CS.CMU.EDU [128.2.185.175]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA15874 for ; Mon, 29 Jun 1998 14:46:44 -0700 (PDT) (envelope-from jaharkes@TELOS.ODYSSEY.CS.CMU.EDU) Received: from TELOS.ODYSSEY.CS.CMU.EDU (localhost [127.0.0.1]) by TELOS.ODYSSEY.CS.CMU.EDU (8.8.7/8.8.7) with ESMTP id RAA01672; Mon, 29 Jun 1998 17:46:27 -0400 Message-Id: <199806292146.RAA01672@TELOS.ODYSSEY.CS.CMU.EDU> X-Mailer: exmh version 2.0zeta 7/24/97 To: linux-kernel@vger.rutgers.edu, freebsd-afs@FreeBSD.ORG Subject: Coda version 4.6.0 available Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 29 Jun 1998 17:46:26 -0300 From: jaharkes@cs.cmu.edu Sender: owner-freebsd-afs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Coda distributed file system version 4.6.0 now available http://www.coda.cs.cmu.edu Coda is an advanced experimental distributed file system with features such as server replication and disconnected operation for laptops (see the WWW site). Release 4.6.0 is mainly a bug fix release with changes since 4.4.4 detailed in the ChangeLog file available on the WWW site and in the root of the sources. The major improvements in this release are: - no more server hangs on first time startup of a server - RPC2 packet buffers are now checked for sanity and should not cause segfaults when bad - many thread stack overflows fixed - linux 2.1 kernel code is now MUCH more stable (many changes) - a coda-linux kernel package for easy module building - Linux Debian compilation fixed - major improvements in debugging features - preliminary support for NT & 95 Coda servers - very preliminary support for Win 95 clients (see below) - initial support for NetBSD packages Stability: we believe this release is a major improvement over the previous one. Coda 4.6.0 on Linux, FreeBSD and NetBSD is very usable as an experimental file system. The development of Coda is largely done within /coda and many of us keep email and other vital files in /coda. We have confusing situations, but have not lost any files. Platforms: Linux (i86 and sparc): Kernel 2.0 -- kernel code included in coda-linux package; this code is not much changed from 4.4.4: it is stable in the sense of not crashing your system, but is known to have odd bugs due to an incomplete implementation of the Coda semantics Kernel 2.1 -- kernel code is included in the Linux kernel, HOWEVER, that code lags our current version, so you should use the kernel code provided in the 4.6.0 coda-kernel package instead (we are holding off submitting patches to avoid overloading Linus). This code closely follows the Coda semantics and is in constant use by most of us, but is less extensively exercised than the 2.0 code and may still, rarely, crash systems. User-level issues: we no longer support libc5; it may still work just fine. The Coda tarball compiles under Debian and there will be a Debian maintainer for Coda at some point (details to be announced). Packages for Red Hat 5.0 are available and appear to work fine on Red Hat 5.1. NetBSD 1.3: NetBSD benefits from the general bugfixes and is working very well both as a server and as a client. NetBSD 1.3 packages are available. FreeBSD: FreeBSD mostly works very well; however, there is a bug which occasionally crashes the client cache manager. Tarballs for 2.2.5 are available and packages will be available. We apologize that for project reasons we will not be able to adequately test 2.2.6 versions until August although 4.4.4 was known to work just as well on 2.2.5 and 2.2.6. Win32: Preliminary support for running Coda servers on Windows NT and Windows 95. In this release, the Win32 port is only suitable for developers to experiment with. No kernel-mode code is needed for the Coda server. All building is done with cross-compilation using Cygwin32 from Red Hat Linux (cross-build RPMs are available); see README.nt in the root of the source tree for details. Windows 95: Very preliminary support for a client on Windows 95 is included in 4.6.0. To use this, you need to have the entire Microsoft toolchain necessary to build VxDs for the kernel-level components of Coda. In addition, you need a special cross-compilation environment for the user-level components which is available for Red Hat Linux from our website. Note on Windows ports: we hope that by making this code available we will encourage developers to explore it and perhaps join in development. It is not usable as a filesystem yet. We thank many net contributors for patches, but above all Michael Callahan for developing and contributing the Windows 95 port as well as continuing to be an invaluable resource for our group. Please join our bazaar! Signed: The CMU Coda team Peter J. Braam development leader Shafeeq Sinnamohideen extensive networking code improvements Jan Harkes improvements to the client and RVM Bob Baron BSD support Henry Pierce release engineering Satya head of the group vital for past, present and future To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-afs" in the body of the message From owner-freebsd-afs Fri Jul 3 17:49:41 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA15826 for freebsd-afs-outgoing; Fri, 3 Jul 1998 17:49:41 -0700 (PDT) (envelope-from owner-freebsd-afs@FreeBSD.ORG) Received: from TELOS.ODYSSEY.CS.CMU.EDU (TELOS.ODYSSEY.CS.CMU.EDU [128.2.185.175]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id RAA15821; Fri, 3 Jul 1998 17:49:38 -0700 (PDT) (envelope-from jaharkes@TELOS.ODYSSEY.CS.CMU.EDU) Received: from TELOS.ODYSSEY.CS.CMU.EDU (localhost [127.0.0.1]) by TELOS.ODYSSEY.CS.CMU.EDU (8.8.7/8.8.7) with ESMTP id UAA01805; Fri, 3 Jul 1998 20:49:32 -0400 Message-Id: <199807040049.UAA01805@TELOS.ODYSSEY.CS.CMU.EDU> X-Mailer: exmh version 2.0zeta 7/24/97 To: coda-announce@TELEMANN.coda.cs.cmu.edu, linux-coda@TELEMANN.coda.cs.cmu.edu, linux-fsdevel@vger.rutgers.edu, linux-kernel@vger.rutgers.edu, freebsd-hackers@FreeBSD.ORG, freebsd-afs@FreeBSD.ORG, netbsd-announce@netbsd.org, netbsd-current@netbsd.org Subject: Coda version 4.6.1 available Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 03 Jul 1998 20:49:32 -0300 From: jaharkes@cs.cmu.edu Sender: owner-freebsd-afs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Coda distributed file system version 4.6.1 now available http://www.coda.cs.cmu.edu Coda is an advanced experimental distributed file system with features such as server replication and disconnected operation for laptops (see the WWW site). Release 4.6.1 is mainly a bug fix release with changes since 4.4.4 detailed in the ChangeLog file available on the WWW site and in the root of the sources. The major improvements in this release are: Since 4.6.0: - an uninitialized variable error has been found which in combination with a lookup of a symlink of a particlar length could cause an internal fd to be closed which would crash venus. (so only upgrading coda-client-4.6.1 is necessary for people who are already running 4.6.0) Since 4.4.4: - no more server hangs on first time startup of a server - RPC2 packet buffers are now checked for sanity and should not cause segfaults when bad - many thread stack overflows fixed - linux 2.1 kernel code is now MUCH more stable (many changes) - a coda-linux kernel package for easy module building - Linux Debian compilation fixed - major improvements in debugging features - preliminary support for NT & 95 Coda servers - very preliminary support for Win 95 clients (see below) - initial support for FreeBSD & NetBSD packages Stability: we believe this release is a major improvement over the previous one. Coda 4.6.1 on Linux, FreeBSD and NetBSD is very usable as an experimental file system. The development of Coda is largely done within /coda and many of us keep email and other vital files in /coda. We have confusing situations, but have not lost any files. Platforms: Linux (i86 and sparc): Kernel 2.0 -- kernel code included in coda-linux package; this code is not much changed from 4.4.4: it is stable in the sense of not crashing your system, but is known to have odd bugs due to an incomplete implementation of the Coda semantics Kernel 2.1 -- kernel code is included in the Linux kernel, HOWEVER, that code lags our current version, so you should use the kernel code provided in the 4.6.1 coda-kernel package instead (we are holding off submitting patches to avoid overloading Linus). This code closely follows the Coda semantics and is in constant use by most of us, but is less extensively exercised than the 2.0 code and may still, rarely, crash systems. User-level issues: we no longer support libc5; it may still work just fine. The Coda tarball compiles under Debian and there will be a Debian maintainer for Coda at some point (details to be announced). Packages for Red Hat 5.0 are available and appear to work fine on Red Hat 5.1. NetBSD 1.3 x86: NetBSD benefits from the general bugfixes and is working very well both as a server and as a client. Coda for NetBSD 1.3 on x86 is available as NetBSD packages. We will be more than happy to help others port Coda to other NetBSD platforms; we just don't have the resources or hardware to do this ourselves. FreeBSD 2.2.5 & 2.2.6: FreeBSD mostly works very well; It is now released as FreeBSD packages. We have also released our internal "ports" collection for the Coda Client and Server to build the FreeBSD packages with. We apologize that for project reasons we will not be able to adequately test 2.2.6 versions until August. It does appear to work reasonably well on both 2.2.5 and 2.2.6. Win32: Preliminary support for running Coda servers on Windows NT and Windows 95. In this release, the Win32 port is only suitable for developers to experiment with. No kernel-mode code is needed for the Coda server. All building is done with cross-compilation using Cygwin32 from Red Hat Linux (cross-build RPMs are available); see README.nt in the root of the source tree for details. Windows 95: Very preliminary support for a client on Windows 95 is included since 4.6.0. To use this, you need to have the entire Microsoft toolchain necessary to build VxDs for the kernel-level components of Coda. In addition, you need a special cross-compilation environment for the user-level components which is available for Red Hat Linux from our website. Note on Windows ports: we hope that by making this code available we will encourage developers to explore it and perhaps join in development. It is not usable as a filesystem yet. We thank many net contributors for patches, but above all Michael Callahan for developing and contributing the Windows 95 port as well as continuing to be an invaluable resource for our group. Please join our bazaar! Signed: The CMU Coda team Peter J. Braam development leader Shafeeq Sinnamohideen extensive networking code improvements Jan Harkes improvements to the client and RVM Bob Baron BSD support Henry Pierce release engineering Satya head of the group vital for past, present and future development To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-afs" in the body of the message