Date: Mon, 27 Jan 2014 12:20:00 +0100 (CET) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> To: Kaya Saman <kayasaman@gmail.com> Cc: freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: ZFS confusion Message-ID: <alpine.BSF.2.00.1401271149160.4811@mail.fig.ol.no> In-Reply-To: <52E62DFF.3010600@gmail.com> References: <52E40C82.7050302@gmail.com> <alpine.BSF.2.00.1401270944100.4811@mail.fig.ol.no> <52E62DFF.3010600@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 27 Jan 2014 09:59-0000, Kaya Saman wrote: > Many thanks Trond for the response.... > > I was getting worried for a sec there :-) > > On 01/27/2014 09:08 AM, Trond Endrestøl wrote: > > On Sat, 25 Jan 2014 19:12-0000, Kaya Saman wrote: > > > > [...] > > No one's answered this, so I'll just give you my 2 cents. > > > > Triple parity means you're using storage capacity equivalent of three > > drives for parity alone. If you use five drives in total, this gives > > you 2 drives worth of real data and 3 drives worth of parity. In other > > words, you should really consider using a lot more drives when using > > triple parity, say nine drives. > > So, basically if I have (just as silly example); 5x drives of 100MB capacity, > this would equal 500MB in total. > > Meaning that only 200MB would be usable..... Also remember a couple of things: ZFS uses dynamically stripe widths, not fixed ones as HW RAID. A 37K file will occupy 37K, rounded up to the nearest integer of disk blocks, plus the space needed for parity and metadata. Without a dedicated ZIL, ZFS will also use some of the available space for housekeeping, e.g. ZIL. You shouldn't fill a pool in production completely with data, leaving no room for any COW operations. > Having just done a quick experiment: > > zpool create test_pool raidz3 /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4 > /tmp/disk5 > > I get: > > test_pool 476M 304K 476M 0% 1.00x ONLINE - > > then writing a 300M file to pool: > > dd if=/dev/zero of=/test_pool/file bs=300M count=1 > dd: /test_pool/file: No space left on device > > test_pool 476M 396M 79.8M 83% 1.00x ONLINE - > > du -sch /test_pool/file > 157M /test_pool/file > 157M total > > So I guess, now I have to ask; when building a disk pool, what's the best > option to go for between capacity and redundancy? If you value your data and your data grows, as it usually does, go for both. I hope you're adding some form of backup to the mix, as ZFS is no substitute for doing proper backups to independent media. > In my case I gradually want to expand my storage when I need it, so to > maximize capacity RAID0 would be the easiest. This doesn't give me any > redundancy however, unless I use RAID1+0 but then I loose physical capacity > due to the mirrors. > > > Would it be better to create a raidz1 pool then just add raidz1 pools to the > master as time goes by? > > As in: > > zpool create test_pool raidz1 /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4 > /tmp/disk5 > > > zpool add test_pool raidz1 /tmp/disk6 /tmp/disk7 /tmp/disk8 /tmp/disk9 > /tmp/disk10 > > > dd if=/dev/zero of=/test_pool/file bs=600M count=1 1+0 > records in > 1+0 records out > 629145600 bytes transferred in 190.069986 secs (3310073 bytes/sec) > > > Then running a 'zpool list' gets: > > test_pool 952M 751M 201M 78% 1.00x ONLINE - > > > So virtually out of: > > > NAME STATE READ WRITE CKSUM > test_pool ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > /tmp/disk1 ONLINE 0 0 0 > /tmp/disk2 ONLINE 0 0 0 > /tmp/disk3 ONLINE 0 0 0 > /tmp/disk4 ONLINE 0 0 0 > /tmp/disk5 ONLINE 0 0 0 > raidz1-1 ONLINE 0 0 0 > /tmp/disk6 ONLINE 0 0 0 > /tmp/disk7 ONLINE 0 0 0 > /tmp/disk8 ONLINE 0 0 0 > /tmp/disk9 ONLINE 0 0 0 > /tmp/disk10 ONLINE 0 0 0 > > errors: No known data errors > > > I have 8 disks usable with 2x disks for parity.... if my understanding is > correct. Correct. One "parity drive" evenly (or close to evenly) spread over the members in each raidz1 vdev. I just realized having multiple zraidN vdevs reduces the time needed for resilvering. Just don't make each vdev too small, i.e. with too few drives. As other's have told you, consider raidz2 or raidz3, as drives from the same batch and with equal wear and tear, might fail more or less simultaneously. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1401271149160.4811>