Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 21 Nov 2011 10:14:56 -0500
From:      "Daniel Staal" <>
Subject:   Re: Setting up ZFS - Filesystem Properties and Installing on Root
Message-ID:  <>
In-Reply-To: <>
References:  <>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help

On Sun, November 20, 2011 10:34 pm, APseudoUtopia wrote:
> Hello,
> I'll be setting up a server with ZFS on 9.0-RELEASE (when it's
> released...). I've never used ZFS before, and although I've been
> reading quite a bit about it, I have some questions.
> My plan is to use RAID-Z1 across 4 disks. I'll be using GPT, and I
> would like the root to be ZFS as well. I found a guide:
> In step #4, it has you create boot, swap, and zfs partitions on all 3
> (which would be 4 in my case) disks. Then, in step #5, you install the
> bootloader into all 3 (4) drives. Why do you need boot and swap
> partitions on EACH disk? It seems to me that you would only need disk
> 1 to have boot, swap, and zfs, and the other 3 disks only have one
> partition (using the entire drive) for zfs's pool. Does it have to do
> with the RAIDZ1 setup? Even then, I don't understand it because it's
> not disk mirroring, it's RAID. The BIOS is set to look on one specific
> disk for the loader, not all of them. It seems I'm not understanding
> something entirely here.

As someone else said, the BIOS will search attached drives.  And remember
that one of the points to using a ZFS raid is that you can get hotswap
drives.  If your one drive with a bootloader fails, you can hotswap it and
not notice that it was that drive, which then means your next boot will

Personally, I dedicated one small drive to the boot code, and put
everything else in ZFS.  There are varying opinions on whether that is a
good idea.  ;)

Swap in theory can be put on the ZFS volume - and I have done so - but I
would not recommend it.  It's caused lockups in some situations, which
would have been avoided by putting swap on it's own volume.

> Also, with ZFS, you can have an unlimited number of filesystems,
> correct? I've been trying to figure out the best way to create these
> filesystems with the appropriate flags (specifically: atime,
> compression, devices, exec, quota, readonly, and setuid). If, for
> example, I set devices=off and suid=off on the tank/var filesystem, it
> is applied to the children filesystem, such as, /var/log, /var/db, and
> so on? The flags/properties can be changed on-the-fly, correct? If,
> for example, I set a filesystem noexec, but later realize I need exec,
> I can change it without issue?

Yep.  As long as you didn't set the volume that has the zfs tools to
noexec.  ;)

> Does anyone with zfs experience have any tips on creating a filesystem
> layout, in terms of which filesystems to create and what
> flags/properties? Would it be bad to set noatime, nosuid, nodev, and
> noexec all on the tank, then allow each property appropriately for
> each directory as necessary? As in, set the whole tank noexec, but
> allow exec for /bin, /usr/home, /usr/local/bin, etc.?

Well, you already linked to a decent guide to the filesystem layout. 
Following it won't get you too far wrong.  A couple of thoughts though:

One thing that might have been useful in my experience is not having / be
/zroot.  If I'd set /zroot/root to / (and built everything else below
that) I'd be able to play around a bit more, by setting up other
filesystems under /zroot. (Including things like making a new root volume,
or excluding volumes from snapshots.)  Note that would cause changes in
the bootloader code from the wiki example.

And don't be to worried about getting it 'right the first time': Because
ZFS allows you to create filesystems so easily, you can play around with
it.  It's not too hard to move a directory tree to a new filesystem, even
on a live box.  ;)  You can create copies, try things out, and then use
whatever's the best system for you.

As for noexec on /...  I'd be hesitant.  Without being an absolute master
on what is run from where under all circumstances in FreeBSD, it would
worry me.  (Does /etc need exec?   How about /root?  /boot?  etc.)  I'd
rather err on the side of turning it off where I know I don't need it,
instead of turning it on where I do.  If I forget one, it's less likely to
break my system.  ;)

The one other place where I'd disagree with the wiki page you linked would
be on compression: If you have a fairly recent processor in the box, I'd
recommend gzip for just about everywhere.  In most cases it'll actually
speed up I/O.

Oh, and if you are thinking of using dedup, you'll need sha256 for the
checksum.  But I think it'll do that automatically if you turn on dedup.

Daniel T. Staal

This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.

Want to link to this message? Use this URL: <>