From owner-freebsd-current@FreeBSD.ORG Sat Apr 14 09:48:32 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 27ADD16A406 for ; Sat, 14 Apr 2007 09:48:32 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mail.atsec.com (mail.atsec.com [195.30.252.105]) by mx1.freebsd.org (Postfix) with ESMTP id 6E12813C46A for ; Sat, 14 Apr 2007 09:48:30 +0000 (UTC) (envelope-from se@freebsd.org) Received: (qmail 14379 invoked by uid 10125); 14 Apr 2007 09:21:49 -0000 X-SpaceNet-Virusscan: Sophos Version: 4.13; Last IDE Update: 2007-04-14 04:30 no information about results Received: from p5087934c.dip0.t-ipconnect.de (HELO ?192.168.0.12?) (80.135.147.76) by mail.atsec.com with SMTP; 14 Apr 2007 09:21:49 -0000 X-SpaceNet-Authentification: SMTP AUTH verified Message-ID: <46209D21.2010704@FreeBSD.org> Date: Sat, 14 Apr 2007 11:21:37 +0200 From: Stefan Esser User-Agent: Thunderbird 2.0.0.0 (Windows/20070326) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> In-Reply-To: <20070409094319.GB76673@garage.freebsd.pl> X-Enigmail-Version: 0.94.1.2.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2007 09:48:32 -0000 Pawel Jakub Dawidek wrote: > On Mon, Apr 09, 2007 at 03:17:23AM +0200, Pawel Jakub Dawidek wrote: Hi Pawel, great to see ZFS support committed to -current! It is amazingly simple to get a test setup going and it worked fine in my initial simple test cases. But now I've run into problems that probably are not technical but caused by a lack of understanding ... >> 3. It is now possible to have root file system on ZFS. You would still >> need UFS for your /boot/ file system. Hmmm, I yesterday bought a new disk for playing with ZFS and while it worked fine in most of my tests, I could not get the kernel to mount the ZFS based root partition. Maybe there is something wrong in the way I tried it (I did not strictly follow the method you suggested, because of special requirements). > Let me explain how this suppose to work. > > You have ad0 disk. Create one slice covering entire disk: > > # fdisk -BI /dev/ad0 In my case I have a ad0s1a (boot file system) and ad0s1b (swap for core dumps) and planned to have ad0s2 for ZFS. DOS and BSD partitions were created from, within sysinstall (and I have verified with CLI tools that the result looks OK). > Initialize BSDlabel: > > # bsdlabel -wB /dev/ad0s1 > > Edit your label and create small (like 256MB-512MB) 'a' partition and > use the rest for 'd' partition: > > # bsdlabel -e /dev/ad0s1 > > 'd' partition will be used for ZFS: In my case 1GB for ad0s1a (256MB) and ad0s1b (768MB) and 300GB for ad0s2. > # zpool create tank ad0s1d zpool create test ad0s2 > Create UFS file system on /dev/ad0s1a and copy /boot/ directory in > there: > > # newfs /dev/ad0s1a > # mount /dev/ad0s1a /mnt/tmp > # cp -Rp /boot/* /mnt/tmp/ I have copied /boot and /rescue to ad0s1a (and later, to simplify further testing, also /etc, /bin, /sbin, /lib, /libexec, since I wanted to be able to run recovery tools from ad0s1a). > Note that there is no /boot/ directory on ad0s1a yet. This is one of the > two possibilities. You now need to create symlink: > > # cd /mnt/tmp > # ln -s . boot > > From what I checked our loader should handle symlinks just fine. > This will allow us to mount /dev/ad0s1a on /boot directory and use it as > usual. > > Another option is to: > > # cp -Rp /boot /mnt/tmp/ > > and in the future mount /dev/ad0s1a on eg. /bootdisk and create symlink: > > # ln -s bootdisk/boot /boot I decided to habe a /.ufs file system and mount ad0s1a there, with symlinks from /boot and /rescue to /.ufs/boot and /.ufs/rescue ... > All in all, you should see your kernel when you do: > > # ls -l /mnt/tmp/boot/kernel > > Now don't forget to add zfs_load="YES" to /mnt/tmp/boot/loader.conf. > > Ok, you also need to tell your loader where your root file system is. > You can do it by adding: > > vfs.root.mountfrom="zfs:tank" > > to /mnt/tmp/boot/loader.conf or you can create /mnt/tmp/etc/fstab file > with one entry only: > > tank / zfs rw 0 0 Neither of these make the kernel accept the ZFS root file system in my case. (I currently have both, but even when I enter zfs:test at the prompt displayed by the kernel after failure to mount a root fs, it is not accepted). I do then enter ufs:ad0s1a to boot my UFS file system for recovery (with /bin, /sbin, /lib, and /etc besides /boot and /rescue). Then I have access to "zfs" and "zpool" commands, which let me mount the ZFS file system, even overloading the UFS root. I only have to make sure, that I manually mount DEVFS into the ZFS root. But in most of my tests, I need to "export" and "import -f" the ZFS pool, since it is displayed as "FAULTED" at first. This is true even if I manage to mount the UFS partition in such a way, that /boot/zfs/zpool.cache is correctly updated. I assume that file is used to control the automatic ZFS root mounting? Do I need to set the "alternate root" flag on the pool during import to have it accepted as a root FS during boot??? (This will be my next test, I think ...) > On your ZFS file system, your /etc/fstab should contains the line above > and: > > /dev/ad0s1a /boot ufs rw 0 0 I use: /dev/ad0s1a /.ufs ufs rw 1 2 > (and everything else, ie. your swap and other file systems) Hmmm, there are a few points that I do not fully understand: It seems that ZFS "legacy" mounts are not supported under FreeBSD, is this correct? (E.g. if I enter "zfs set mountpoint=legacy test" then "test" can not be mounted with "zfs mount test" and there is no other way to mount it since we do not have a "mount_zfs", yet?) I tried to set the mountpoint of my to-be root file system to "/" with "zfs set mountpoint=/ test" but I'm under the impression that this does not really work. Setting it to "//" does appear to have the desired effect, though, but may lead to a panic during shutdown. (Sorry, I've got no core-dumps but could try producing one later if there is interest. The panic is because of a ref count becoming negative but I did not write down the message.) I decided to have multiple zfs file systems (test/var, test/usr ...) and can see them with zfs list. What is the correct way to get them mounted automatically? (Assuming I get the problem to have the kernel automatically mount the ZFS root solved ...) Do I need fstab entries for for ZFS file systems (e.g. "test/usr") or does ZFS mount them automatically when the pool "test" is mounted? Or do I need a fstab line for each of them? What's supposed to go into /etc/zfs, besides the ZFS exports file? Regards, STefan