Date: Sat, 19 Sep 2015 09:04:48 -0500 (CDT) From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> To: Sami Halabi <sodynet1@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup Message-ID: <alpine.GSO.2.01.1509190843040.1673@freddy.simplesystems.org> In-Reply-To: <CAEW%2BogbPswfOWQzbwNZR5qyMrCEfrcSP4Q7%2By4zuKVVD=KNuUA@mail.gmail.com> References: <CAEW%2BogbPswfOWQzbwNZR5qyMrCEfrcSP4Q7%2By4zuKVVD=KNuUA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 19 Sep 2015, Sami Halabi wrote: > Hi everyone, > I've been searching the documentation, wiki.. etc. but found no rule of > thumb to CPU requirements to run ZFS fs. for memory there are few rules > according to using dedup or not. > > rules of thumb I concluded are so far: > 1. use compression lz4. > 2. use checksum. > 3. disable atime. > > from what i read the status of dedup is not that clear and seems there are > bugs and better to avoid it? I am not so sure about dedup bugs. Usually the problem encountered with dedup is unrealistic expectations about how it will behave. Dedup requires a large amount of RAM and/or L2ARC device which are appropriately sized for the amount of storage and the amount of redundancy encountered. The problem typically encountered is that things work "fine" with dedup until one day they do not and when they do not, it is like going off a cliff. Don't enable dedup unless you have carefully researched it, decided that its value outweighs its cost, and make sure that you have enough RAM or SSD-based L2ARC (preferably RAM) so that the tables it needs are all readily available without accessing rotating media. Depending on what you were planning to use dedup for, there may be other zfs tricks (e.g. cloning) which are amost as effective, but without the risks. > so according to 1-3 above what cpu requirements i need? is ATOM cpu like > supermicro c2750/3/5/8 enough to run system of 20TB /40TB with 1-3 above? > if dedup IS enabled would it still work fine? CPU usage is rarely a problem for zfs except for when compression is involved. Lz4 is supposed to be CPU-efficient on Intel CPUs. Don't use gzip compression if you don't have a powerful CPU. Compression can be changed at any time, although a given file will remain compressed with the compression setting in effect when it was created. CPU use is driven by I/O requirements and selected compression algorithms and not by the size of the storage pool. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.GSO.2.01.1509190843040.1673>