Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 22 Nov 2011 09:52:41 +0000
From:      krad <>
To:        Matthew Seaman <>
Subject:   Re: Setting up ZFS - Filesystem Properties and Installing on Root
Message-ID:  <>
In-Reply-To: <>
References:  <> <> <> <>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
> It seems to me that you would only need disk
> 1 to have boot, swap, and zfs, and the other 3 disks only have one
> partition (using the entire drive) for zfs's pool.

As other have mentioned redundancy,  but also you will nver see the befit
as the zfs vdev (like any other raid system) size will defined by the
smallest unit in the group. ie if you have 4 x 1tb drives and you have 3x
1tb slices and 950GB available on your boot drive then all the storage you
will get is 4 x 950 - the parity data. Therefore make all you drives
layouts identical and mirror any boot partitions across them all, or just 2
and use the other 2 for swap or a combination of the 2.

Another way to do it is boot off usb stick although you should be able to
boot off a native raidz these days without to much hassle. If you do run
into issues with booting of zfs though try these recompiled boot blocks as
I never have issues with them.

If you are using 4k disks which there is a fairly good chance you are make
sure you create the pool with ashift=12 using the gnop trick. Otherwise you
may experiance bad disk performance.

WIth regards to dedup, unless you have bucket loads of ram (32+Gigs) and/or
an ssd dedicated to l2arc stay away from it as you will almost certainly
find that very quickly the DDT wont fit into ram, and when that happens the
performance of the pool takes a serious performance dive do to every write
incuring many many reads to retrieve the ddt information. Also it may not
be worth it with your dataset. To test what you might achieve do a zdb -S
<pool> to see your expected dedup ratio.

in terms of disk layout this is fairly arbitary and you have a lot of
choice. This is what i use, and a loosly based it on opensolaris

system-4k/be                              26.6G   207G   252K  /system-4k/be
system-4k/be/root20110930                 1.73G   207G  1.31G  legacy
system-4k/be/root20111011                 2.03G   207G  1.69G  legacy
system-4k/be/root20111023                 1.98G   207G  1.68G
system-4k/be/root20111028                 2.00G   207G  1.68G
system-4k/be/root20111112                 2.08G   207G  1.76G
system-4k/be/tmp                           360K   209G   360K  /tmp
system-4k/be/usr-local                    3.30G   207G  3.30G  /usr/local/
system-4k/be/usr-obj                       728M   207G   728M  /usr/obj
system-4k/be/usr-ports                    2.05G   207G  1.51G  /usr/ports
system-4k/be/usr-ports/distfiles           547M   207G   547M
system-4k/be/usr-src                       705M   207G   705M  /usr/src
system-4k/be/var                          2.04G   213G   816M  /var
system-4k/be/var/log                      1.21G   213G  1.21G  /var/log
system-4k/be/var/mysql                    34.0M   213G  34.0M  /var/db/mysql

everytime I do a make installword and installkernel I create a new root fs.
This way I can easily flip flop back and two between different os builds if
i want to. I use this simple script to set it up for me. Its not perfect
but it works well enough

$ cat /usr/local/scripts/install_world

if [ $UID != 0 ] ; then
  echo your not root !! ; exit 1

date=`date '+%Y%m%d'`
oroot=`grep "vfs.root.mountfrom=\"zfs:system-4k/" /boot/loader.conf | sed
-e "s#^.*\"zfs:system-4k/be/##" -e "s#\"##"`

export DESTDIR=/$zpool/be/$nroot

if [ "$oroot" =  "$nroot" ] ; then
 echo "i cant update twice in one day"; exit 1

echo building in $zpool/be/$nroot

zfs snapshot $zpool/be/$oroot@$snap &&
zfs send $zpool/be/$oroot@$snap | mbuffer -m 500M | zfs receive -vv
$zpool/be/$nroot    &&
cd /usr/src &&
make installkernel &&
mount_nullfs /var $DESTDIR/var &&
mergemaster -p -D $DESTDIR &&
make installworld &&
mergemaster -D $DESTDIR &&
sed -i -e "s#$zpool/be/$oroot#$zpool/be/$nroot#" $DESTDIR/boot/loader.conf
&& \
echo "Installing boot records.." &&
zpool status system-4k | grep -A 2 mirror | grep ad | sed -e "s/p[0-9]//"
        while read a b; do
                gpart bootcode -b /zfsboot/pmbr -p /zfsboot/gptzfsboot -i 1
        done &&
cp -v /zfsboot/zfsloader $DESTDIR/boot/. &&
echo -en "\n\nNow run these two commands to make the changes live, and
 zfs set mountpoint=legacy $zpool/be/$nroot
 zpool set bootfs=$zpool/be/$nroot $zpool\n\n"

Want to link to this message? Use this URL: <>