Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 5 Mar 2011 09:42:47 -0500
From:      Andriy Bakay <andriy@irbisnet.com>
To:        Jeremy Chadwick <freebsd@jdc.parodius.com>
Cc:        "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, "pjd@FreeBSD.org" <pjd@FreeBSD.org>
Subject:   Re: kmem_map too small with ZFS and 8.2-RELEASE
Message-ID:  <625F7D8A-78F5-426B-B2C3-A1B4D289FD5F@irbisnet.com>
In-Reply-To: <20110304143801.GA1734@icarus.home.lan>
References:  <1299232133.18671.3.camel@pc286.embl.fr> <4D70E7D1.5030205@egr.msu.edu> <20110304143801.GA1734@icarus.home.lan>

next in thread | previous in thread | raw e-mail | index | archive | help
On 2011-03-04, at 9:38, Jeremy Chadwick <freebsd@jdc.parodius.com> wrote:

> On Fri, Mar 04, 2011 at 08:23:29AM -0500, Adam McDougall wrote:
>> On 03/04/11 04:48, Micka=C3=ABl Can=C3=A9vet wrote:
>>>> I'd use vm.kmem_size=3D"32G" (i.e. twice your RAM) and that's it.
>>>=20
>>> Should I also increase vfs.zfs.arc_max ?
>>>=20
>>> Do you have any idea why the kernel panicked at only 8GB allocated ?
>>>=20
>>> Thank you
>>=20
>> I believe ARC allocations in kmem can become fragmented, so when it
>> is searching for a place to store a new contiguous segment of
>> memory, the remaining fragmented free spaces may all be too small.
>> I also set
>> vm.kmem_size to about twice the amount of ram to help it avoid this
>> issue.  I suspect if kmem is badly fragmented then performance of
>> ZFS can downgrade, so that is another reason to keep kmem bigger.
>=20
> My findings on 8.2-RELEASE indicate that doing this results in very
> unexpected behaviour regarding the ARC maximum.  As such, I cannot
> recommend this model.
>=20
> For example, on an amd64 system with 8GB physical RAM and these two
> settings in /boot/loader.conf:
>=20
> vm.kmem_size=3D"8192M"
> vfs.zfs.arc_max=3D"6144M"
>=20
> kstat.zfs.misc.arcstats.size tops out at around 6240986896, with Wired
> in top(1) showing ~6.3GB.  This is expected behaviour and fits (I
> think) what people expect.
>=20
> However, on the exact same system with these two settings:
>=20
> vm.kmem_size=3D"16384M"
> vfs.zfs.arc_max=3D"6144M"
>=20
> The above ARC numbers are exactly *half* that amount.  This is easily
> reproducible.
>=20
> Can someone 1) justify the "2x the amount of RAM for vm.kmem_size"
> setting, and 2) explain in detail the above behaviour?
>=20
> --=20
> | Jeremy Chadwick                                   jdc@parodius.com |
> | Parodius Networking                       http://www.parodius.com/ |
> | UNIX Systems Administrator                  Mountain View, CA, USA |
> | Making life hard for others since 1977.               PGP 4BD6C0CB |
>=20
> _______________________________________________
> freebsd-fs@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"

I guess Pawel can:

vm.kmem_size=3D=E2=80=9D6G=E2=80=9D # This should be 150% of your RAM.

http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-confi=
guration/



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?625F7D8A-78F5-426B-B2C3-A1B4D289FD5F>