Date: Tue, 8 Jul 2008 22:56:45 -0700 From: Jeremy Chadwick <koitsu@FreeBSD.org> To: CZUCZY Gergely <phoemix@harmless.hu> Cc: freebsd-fs@freebsd.org, Kris Kennaway <kris@FreeBSD.org> Subject: Re: Thinking of using ZFS/FBSD for a backup system Message-ID: <20080709055645.GA40076@eos.sc1.parodius.com> In-Reply-To: <20080709074420.24df3be4@mort.in.publishing.hu> References: <bd9320b30807072315x105cf058tf9f952f0f5bb2a6a@mail.gmail.com> <20080708100701.57031cda@twoflower.in.publishing.hu> <bd9320b30807080131j5e0e02a4y3231d7bfa1738517@mail.gmail.com> <4873C4FA.2020004@FreeBSD.org> <20080708221327.5c1d0e92@mort.in.publishing.hu> <4873CF6C.7000205@FreeBSD.org> <20080708225449.1070252d@mort.in.publishing.hu> <4873F4E9.3040203@FreeBSD.org> <20080709074420.24df3be4@mort.in.publishing.hu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jul 09, 2008 at 07:44:20AM +0200, CZUCZY Gergely wrote: > On Wed, 09 Jul 2008 01:14:49 +0200 > Kris Kennaway <kris@FreeBSD.org> wrote: > > > CZUCZY Gergely wrote: > > I don't know; empirically my setup is an upper bound. How large was > > "as large as it was allowed" for you? > Well, we cannot buy "upper bounds" all over, just because some > developer is unable to figure out things. I think you can't expect > FreeBSD users to spend as much money as possible, just because the devs > can't tell how much is enough... > It seems more like a twilight zone then a stable feature now ;) > > It was exactly as much as an amd64 installation would allow with 2GB of > physical memory. We've dismissed the setup around february, and I don't > have the configs anymore. It was an amd64 setup with 2GB of physical > memory. The bottom line here is that i386 and amd64 both have a kmem_size limit of 2GB. You can throw 32GB of RAM into an amd64 box, but FreeBSD will only utilise up to 2GB of that for kmem. That is purely a FreeBSD limitation, and is being dealt with in HEAD by Alan Cox. I believe he has a patch, or it may have been committed -- I don't follow HEAD. I can point people to a mailing list URL, if needed. This is one of the limitations Gergely is referring to. Since ZFS is incredibly memory-hungry, you're forced to tune ZFS to try and get it to "play nice" with that 2GB limit on STABLE/RELEASE systems. You also need to keep in mind that you can't just set kmem_size and kmem_size_max to 2048M, because the kernel needs memory for other things. The tuning parameters I use on my 2GB amd64 and 4GB amd64 boxes are: vm.kmem_size="1536M" vm.kmem_size_max="1536M" vfs.zfs.arc_min="16M" vfs.zfs.arc_max="64M" If you set kmem_size and kmem_size_max any higher than that, the machine will panic on boot, stating (indirectly) that there isn't enough memory available for the kernel to allocate for other things. Until I added the arc_min and arc_max setting, I could occasionally panic the machines under very heavy load (heavy zpool I/O), caused by kmem exhaustion. Since adding the arc_* tunings, I've tried very hard to crash the machines, and I cannot. But there's absolutely no guarantee those tuning parameters above will ensure FreeBSD won't panic due to kmem exhaustion. I believe this is the point Gergely is making about the "stability" of the whole thing. Now, with regards to prefetch_disable, folks can disable that if they want. I disable it on my above systems because for what they do, the overall performance appears better with prefetching disabled. I hope this helps shed some light here... -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080709055645.GA40076>