From owner-freebsd-questions@FreeBSD.ORG Tue Nov 22 02:09:47 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78302106566B for ; Tue, 22 Nov 2011 02:09:47 +0000 (UTC) (envelope-from apseudoutopia@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0CA308FC17 for ; Tue, 22 Nov 2011 02:09:46 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so9749697bkb.13 for ; Mon, 21 Nov 2011 18:09:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=xFj3taxBwfPN4A1VOzSLy+pecyAI6qm1daANnZp6QXg=; b=dgiUimyWs5gKLYbMutjKPFhohPZqF3t/blVQdBolPycs0xOAWHeliwn6Tt6u7zZyrE 7RFN/Tme+9Pslnk1jTjkgLFULpJe0PmGQgOfKHs7Y4h+pf6nHxeB04W0OpZ8ciPCbe4y PLW3FUefSE3rLlZ+1rsPWOCJD7H6PnfcGHKyw= MIME-Version: 1.0 Received: by 10.205.112.6 with SMTP id eq6mr16971888bkc.16.1321927785899; Mon, 21 Nov 2011 18:09:45 -0800 (PST) Received: by 10.204.133.215 with HTTP; Mon, 21 Nov 2011 18:09:45 -0800 (PST) In-Reply-To: <88f3d8e819b3420f8e61723bee90ba5e.squirrel@www.magehandbook.com> References: <88f3d8e819b3420f8e61723bee90ba5e.squirrel@www.magehandbook.com> Date: Mon, 21 Nov 2011 21:09:45 -0500 Message-ID: From: APseudoUtopia To: Daniel Staal Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-questions@freebsd.org Subject: Re: Setting up ZFS - Filesystem Properties and Installing on Root X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 02:09:47 -0000 On Mon, Nov 21, 2011 at 10:14 AM, Daniel Staal wrote: > > On Sun, November 20, 2011 10:34 pm, APseudoUtopia wrote: >> Hello, >> >> I'll be setting up a server with ZFS on 9.0-RELEASE (when it's >> released...). I've never used ZFS before, and although I've been >> reading quite a bit about it, I have some questions. >> >> My plan is to use RAID-Z1 across 4 disks. I'll be using GPT, and I >> would like the root to be ZFS as well. I found a guide: >> http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ1 >> >> In step #4, it has you create boot, swap, and zfs partitions on all 3 >> (which would be 4 in my case) disks. Then, in step #5, you install the >> bootloader into all 3 (4) drives. Why do you need boot and swap >> partitions on EACH disk? It seems to me that you would only need disk >> 1 to have boot, swap, and zfs, and the other 3 disks only have one >> partition (using the entire drive) for zfs's pool. Does it have to do >> with the RAIDZ1 setup? Even then, I don't understand it because it's >> not disk mirroring, it's RAID. The BIOS is set to look on one specific >> disk for the loader, not all of them. It seems I'm not understanding >> something entirely here. > > As someone else said, the BIOS will search attached drives. =C2=A0And rem= ember > that one of the points to using a ZFS raid is that you can get hotswap > drives. =C2=A0If your one drive with a bootloader fails, you can hotswap = it and > not notice that it was that drive, which then means your next boot will > fail. > > Personally, I dedicated one small drive to the boot code, and put > everything else in ZFS. =C2=A0There are varying opinions on whether that = is a > good idea. =C2=A0;) > > Swap in theory can be put on the ZFS volume - and I have done so - but I > would not recommend it. =C2=A0It's caused lockups in some situations, whi= ch > would have been avoided by putting swap on it's own volume. > >> Also, with ZFS, you can have an unlimited number of filesystems, >> correct? I've been trying to figure out the best way to create these >> filesystems with the appropriate flags (specifically: atime, >> compression, devices, exec, quota, readonly, and setuid). If, for >> example, I set devices=3Doff and suid=3Doff on the tank/var filesystem, = it >> is applied to the children filesystem, such as, /var/log, /var/db, and >> so on? The flags/properties can be changed on-the-fly, correct? If, >> for example, I set a filesystem noexec, but later realize I need exec, >> I can change it without issue? > > Yep. =C2=A0As long as you didn't set the volume that has the zfs tools to > noexec. =C2=A0;) > >> Does anyone with zfs experience have any tips on creating a filesystem >> layout, in terms of which filesystems to create and what >> flags/properties? Would it be bad to set noatime, nosuid, nodev, and >> noexec all on the tank, then allow each property appropriately for >> each directory as necessary? As in, set the whole tank noexec, but >> allow exec for /bin, /usr/home, /usr/local/bin, etc.? > > Well, you already linked to a decent guide to the filesystem layout. > Following it won't get you too far wrong. =C2=A0A couple of thoughts thou= gh: > > One thing that might have been useful in my experience is not having / be > /zroot. =C2=A0If I'd set /zroot/root to / (and built everything else belo= w > that) I'd be able to play around a bit more, by setting up other > filesystems under /zroot. (Including things like making a new root volume= , > or excluding volumes from snapshots.) =C2=A0Note that would cause changes= in > the bootloader code from the wiki example. > > And don't be to worried about getting it 'right the first time': Because > ZFS allows you to create filesystems so easily, you can play around with > it. =C2=A0It's not too hard to move a directory tree to a new filesystem,= even > on a live box. =C2=A0;) =C2=A0You can create copies, try things out, and = then use > whatever's the best system for you. > > As for noexec on /... =C2=A0I'd be hesitant. =C2=A0Without being an absol= ute master > on what is run from where under all circumstances in FreeBSD, it would > worry me. =C2=A0(Does /etc need exec? =C2=A0 How about /root? =C2=A0/boot= ? =C2=A0etc.) =C2=A0I'd > rather err on the side of turning it off where I know I don't need it, > instead of turning it on where I do. =C2=A0If I forget one, it's less lik= ely to > break my system. =C2=A0;) > > The one other place where I'd disagree with the wiki page you linked woul= d > be on compression: If you have a fairly recent processor in the box, I'd > recommend gzip for just about everywhere. =C2=A0In most cases it'll actua= lly > speed up I/O. > > Oh, and if you are thinking of using dedup, you'll need sha256 for the > checksum. =C2=A0But I think it'll do that automatically if you turn on de= dup. > > Daniel T. Staal > > --------------------------------------------------------------- > This email copyright the author. =C2=A0Unless otherwise noted, you > are expressly allowed to retransmit, quote, or otherwise use > the contents for non-commercial purposes. =C2=A0This copyright will > expire 5 years after the author's death, or in 30 years, > whichever is longer, unless such a period is in excess of > local copyright law. > --------------------------------------------------------------- > Ok, thank you for the advice. It's on a quad-core xeon, so, in reference to your suggestion, I'll turn on compression for the entire pool. I did read that having swap in zfs may cause problems, so I'll be sticking to the traditional freebsd-swap partition. Another quick question about swap: If I have 4 drives, with 512MB swap, the system uses all 4 swap partitions, correct? So it's not like it'd be going to waste? I'd have a total of 2 GB swap? Thanks!