Date: Sat, 04 Sep 2010 10:53:25 -0400 From: jhell <jhell@DataIX.net> To: Artem Belevich <fbsdlist@src.cx> Cc: freebsd-fs@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? Message-ID: <4C825D65.3040004@DataIX.net> In-Reply-To: <AANLkTi=6bta-Obrh2ejLCHENEbhV5stbMsvfek3Ki4ba@mail.gmail.com> References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk> <AANLkTi=6bta-Obrh2ejLCHENEbhV5stbMsvfek3Ki4ba@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------000703040900060801010300 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 09/03/2010 21:09, Artem Belevich wrote: >> For reference top shows:- >> Mem: 42M Active, 3129M Inact, 565M Wired, 3188K Cache, 19M Buf, 203M Free > > Here's your problem -- inactive list got all your memory and starved ZFS ARC. > Easy workaround is to set vfs.zfs.arc_min to a value that would > guarantee that ARC does not give up too much memory. Let's say - 2GB. > Be warned that it would effectively make those 2GB unavailable to > applications. > > Long term, though, there were number of patches posted on > freebsd-current and freebsd-hackers recently that do improve that > particular issue with ZFS. This patch in particular may help you: > http://lists.freebsd.org/pipermail/freebsd-hackers/2010-August/032731.html > >> Swap: 4096M Total, 1180K Used, 4095M Free >> >> So what are we missing and how to we get zfs to perform like ufs >> and use all free ram as cache? > Attached is the needfree patch mentioned in the URL alongside a local system patch to adjust kern.maxusers to no more than 512 on systems that can support it. Modern systems would always raise to a value of 512 or more so I do not see a need to keep it limited to 384. This setting also effects the outcome of other values in the system. Not saying that these will directly help your case in anyway shape or form but if you would like to give them a shot then they are here for the taking. Apply with: cd /usr/src patch -p1 </path/to/patch Regards, - -- jhell,v -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMgl1kAAoJEJBXh4mJ2FR+oggH/2Rq2C2vakQkUr9tGraeR+BN 9aIjTPjqAaUIZ/uwrb0q8UBVbb/5pgPDknKlGQUs3S84UwbS1bZ91NbQwoadqqwx MzUFmvETCCKa1GuU0KE6ozd61J9IN7uEm2ZaZhHtcfqsAe5oRMBYhSKNAvOi19H4 4ePWQI0Cgk3lhzAesbG5YD23dh9mOe7EU2cAR8mbwGA8C56bfLK3VSz1mjq2VfVf rkjerZoIkPtrswtXqhdC3G1Zwe+PkPIUfR5IvVWauaLVlt++nywVoB6gZ8JzoWMR kECtVn+/TBbr+69p0XcL6R31DDhtM/QhdisMdpuf2gLakHRXtm4EMBWSTgGon7Y= =YpPh -----END PGP SIGNATURE----- --------------000703040900060801010300 Content-Type: text/plain; name="maxusers.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="maxusers.patch" changeset: 181:deccbf75a766 user: J. Hellenthal <jhell@DataIX.net> date: Fri Sep 03 12:50:37 2010 -0400 files: sys/kern/subr_param.c description: Increase maxusers to 512 by default if it equals 512 or higher. diff -r 627cf9b49e0f -r deccbf75a766 sys/kern/subr_param.c --- a/sys/kern/subr_param.c Fri Sep 03 12:42:00 2010 -0400 +++ b/sys/kern/subr_param.c Fri Sep 03 12:50:37 2010 -0400 @@ -257,8 +257,8 @@ maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE); if (maxusers < 32) maxusers = 32; - if (maxusers > 384) - maxusers = 384; + if (maxusers > 512) + maxusers = 512; } /* --------------000703040900060801010300 Content-Type: text/plain; name="needfree.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="needfree.patch" changeset: 188:dafc39874132 user: J. Hellenthal <jhell@DataIX.net> date: Sat Sep 04 07:34:28 2010 -0400 files: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c description: PFC: r211762 arc.c needfree patch ------------------------------------------------------------------------ r211762 | avg | 2010-08-24 13:48:22 -0400 (Tue, 24 Aug 2010) | 20 lines zfs arc_reclaim_thread: no need to call arc_reclaim_needed when resetting needfree needfree is checked at the very start of arc_reclaim_needed. This change makes code easier to follow and maintain in face of potential changed in arc_reclaim_needed. Also, put the whole sub-block under _KERNEL because needfree can be set only in kernel code. To do: rename needfree to something else to aovid confusion with OpenSolaris global variable of the same name which is used in the same code, but has different meaning (page deficit). Note: I have an impression that locking around accesses to this variable as well as mutual notifications between arc_reclaim_thread and arc_lowmem are not proper. MFC after: 1 week ------------------------------------------------------------------------ diff -r 34a823aff8f6 -r dafc39874132 sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Sep 04 07:23:09 2010 -0400 +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Sep 04 07:34:28 2010 -0400 @@ -2317,12 +2317,12 @@ if (arc_eviction_list != NULL) arc_do_user_evicts(); - if (arc_reclaim_needed()) { +#ifdef _KERNEL + if (needfree) { needfree = 0; -#ifdef _KERNEL wakeup(&needfree); + } #endif - } /* block until needed, or one second, whichever is shorter */ CALLB_CPR_SAFE_BEGIN(&cpr); --------------000703040900060801010300 Content-Type: application/octet-stream; name="maxusers.patch.sig" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="maxusers.patch.sig" iQEcBAABAgAGBQJMgl1kAAoJEJBXh4mJ2FR+ztMIAJL+5MwxxHsSRmrTjt3RCtWvay7uRhnC 9HpEPJkzPNRzIwFx4ckMIuH5tMZTXHlRf6Xt0dTIYf28ExQFQyefVarMuFeK1XfESISt5PEm wDLPITr9AnEACE7IsPEpRZV0kuiPyThDybF+2eh682rPeFkHhvcNCFEkO2jpljXYdTXGbyc+ Csw2dIaMd5fFCWtXwNSCah7DoA0gsnTKRgcFJgan0/GMDLGd1N4QWTlqX0buyOGxCSDhrHZK 6vJRoMmDpJz7RzXNigiayzN8y3PgU6P2JbbHGws8edmQqWpbXHP35wmT6P5/+LhwbyplNu0U S/1U8IHFtRxTmxSL0/wGFjo= --------------000703040900060801010300 Content-Type: application/octet-stream; name="needfree.patch.sig" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="needfree.patch.sig" iQEcBAABAgAGBQJMgl1kAAoJEJBXh4mJ2FR+5ggIAJ41d2boBSnDtIXfrrWQyzvIaTV45naA Es5n94Vcx5DBkbQo915SzhOPko0l26Lkr11+oUwRpi3Cwf5ik1PaVS0CNFjZ+eCWXRBay55V AlnthIGC9K3/EbRrnuPMlKpFWbVptAOBSIRnaZVo2LU13TkkBNqFlg6AkBijcT0SCFeRVCOm xPDP/qRvOj8OEBXsbSfqC4YQajBXRy5nl7mFmK1vPfcwURG7/LUhV/jpUaCbj5QW8S58S7Zq /LxQ45DgRu5P3SK3IldeC+KJFgfxYW52zFQ9bavEuzake18OHbEFs6Qj6YiAcR0cnzA0/w9p wM6awh9TPOZ6XYaFaMgs3Ds= --------------000703040900060801010300--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C825D65.3040004>