Date: Wed, 14 Jan 2015 10:03:36 -0800 From: Freddie Cash <fjwcash@gmail.com> To: =?UTF-8?Q?Karli_Sj=C3=B6berg?= <karli.sjoberg@slu.se> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: How many ram... Message-ID: <CAOjFWZ4x5p04BeM1%2BKLgce7pMMc5X4Ni5%2BL3Px=soTY8teX%2Bxg@mail.gmail.com> In-Reply-To: <6a3129720b4a439994841c28df676cd1@exch2-4.slu.se> References: <6a3129720b4a439994841c28df676cd1@exch2-4.slu.se>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 14, 2015 at 9:47 AM, Karli Sj=C3=B6berg <karli.sjoberg@slu.se> = wrote: > Den 14 jan 2015 18:28 skrev Freddie Cash <fjwcash@gmail.com>: > > One of them has dedupe enabled (yeah, yeah, we know, we're moving away > from > > it, it's actually the last one with it enabled), > > But what about all of the savings you were benefitting from? Wasn't it > like 10x dedup savings or something, I know I've asked before at the foru= ms > but a person forgets... What's made you change your mind? > =E2=80=8BOriginally, we were getting great disk space savings =E2=80=8Bthat= made it worthwhile (4x was our lowest, I think our highest was around 8x). However, then we started backing up our mail server with millions of tiny files ... and performance tanked (backups wouldn't complete overnight), especially when deleting old snapshots. We moved the mail server off to it's own backups box without dedupe (it's one of the multi-JBOD storage systems as 1 year of daily backups is 46 TB) and performance went back to usable. Then we started getting issues with resilvers taking 3+weeks to replace disks, monthly scrubs just barely completing before the next one starts, and running out of RAM a lot. When hardware died and killed the pool, we rebuilt it without dedupe and things are running much smoother now. We didn't lose any data as we had it replicated off-site. :) We have 4 storage systems running ZFS: - admin site backups using dedupe with 64 GB of RAM and 16 harddrives - school site backups using compression only, with 64 GB of RAM and 24 harddrives - mail server backups using compression only, with 128 GB of RAM and 90 harddrives - offsite backups storage using dedupe, with 128 GB of RAM and 90 harddrives The long-term goal is to have only the off-site backups storage system using dedupe. And to try and keep it at 90 harddrives. To help with that, we'll be getting another off-site backups storage system for the mail server backups, which will remove the bulk of the data out of the deduped pool. =E2=80=8BWhen we started with ZFS, back in the FreeBSD 7 days, 500 GB serve= r-class harddrives were around $100-200 CDN, and anything over 1 TB was out of our price range, so dedupe was worthwhile (we started with server-class drives attached to 3Ware RAID controllers). Then 2 TB desktop-class drives dropped down around the $120 CDN range and we started replacing them (and using LSI SATA controllers). And dedupe started losing it's awesomeness. Now, we get 2 TB drives in bulk for $80 CDN, so there's no point suffering through the pain points that =E2=80=8Bcome with dedupe on ZFS. --=20 Freddie Cash fjwcash@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ4x5p04BeM1%2BKLgce7pMMc5X4Ni5%2BL3Px=soTY8teX%2Bxg>