Date: Fri, 2 Oct 2009 17:59:36 -0700 From: Artem Belevich <fbsdlist@src.cx> To: Pawel Jakub Dawidek <pjd@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: ARC size constantly shrinks, then ZFS slows down extremely Message-ID: <ed91d4a80910021759h4a3d3b9csde86c59075795eeb@mail.gmail.com> In-Reply-To: <20091003000909.GD1660@garage.freebsd.pl> References: <4AC1E540.9070001@fsn.hu> <4AC5B2C7.2000200@fsn.hu> <20091002184526.GA1660@garage.freebsd.pl> <ed91d4a80910021638r72213389uf0767b9bb523b7d9@mail.gmail.com> <20091003000909.GD1660@garage.freebsd.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
> Before we start debuging pathological cases, could you try the patch > with defaul settings? Eventually with vm.kmem_size set to the amount of > RAM you have. System runs stable/8 r197716 on 4-core amd64 with 8G of RAM. With default /boot/loader.conf. Kernel comes up with following parameters: vm.kmem_size: 2764533760 vfs.zfs.arc_min: 215979200 vfs.zfs.arc_max: 1727833600 Under load ARC size reaches ~1.7G. At that time top reports: Mem: 47M Active, 11M Inact, 2158M Wired, 268K Cache, 21M Buf, 5693M Free However, as the FS load continues, ARC size, stays at 1.7G for couple of minutes, then shrinks down to 1.2G, then slowly grows to 1.7G, stays there for a little and then the shrink/grow cycle repeats. Throughout the test there's always ~5G of *free* memory. =============================================================== Now, the same experiment, with vm.kmem_size=8G vm.kmem_size: 8589934592 vfs.zfs.arc_min: 939524096 vfs.zfs.arc_max: 7516192768 ARC grows to 6.2G: Mem: 47M Active, 13M Inact, 7376M Wired, 31M Buf, 473M Free Then it quickly shrinks to 4.6G and grows to 6.2G again, shrinks again, etc.. What's different from the previous case is that after a while ZFS adjusts target size (kstat.zfs.misc.arcstats.c) down to ~5.8G and after that ZFS size oscillates between 4.2G and 5.6G. Another observation -- ARC shrinking happens when system is left with ~512M of free memory. Yet another observation is that even with ARC peak of ~5.8G, system has about 7.5G wired. Where did almost 2G of difference go? Fragmentation? I've tried both experiments with and without L2ARC -- behavior seems to be the same. --Artem
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ed91d4a80910021759h4a3d3b9csde86c59075795eeb>