Date: Tue, 7 Jan 2014 19:20:25 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Alexander Motin <mav@FreeBSD.org> Cc: hackers@freebsd.org Subject: Re: UMA caches draining Message-ID: <20140107172025.GP59496@kib.kiev.ua> In-Reply-To: <52CBCC4F.8020900@FreeBSD.org> References: <52CB4F7D.2080909@FreeBSD.org> <20140107054825.GI59496@kib.kiev.ua> <52CBCC4F.8020900@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--PRJsLXMT40XvOKNo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 07, 2014 at 11:43:43AM +0200, Alexander Motin wrote: > On 07.01.2014 07:48, Konstantin Belousov wrote: > > On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: > >> I have some questions about memory allocation. At this moment our UMA > >> never returns freed memory back to the system until it is explicitly > >> asked by pageout daemon via uma_reclaim() call, that happens only when > >> system is quite low on memory. How does that coexist with buffer cache > >> and other consumers? Will, for example, buffer cache allocate buffers > >> and be functional when most of system's memory uselessly consumed by U= MA > >> caches? Is there some design how that supposed to work? > > Allocation of the pages which consitute a new buffer creates the pressu= re > > and causes pagedaemon wakeup if amount of free pages is too low. Look > > at the vm_page_grab() call in allocbuf(). Also note that buffer cache > > is not shrinked in response to the low memory events, and buffers pages > > are excluded from the page daemon scans since pages are wired. >=20 > Thanks. I indeed can't see respective vm_lowmem handler. But how does it= =20 > adapt then? It should have some sort of back pressure. And since it=20 > can't know about UMA internals, it should probably just see that system= =20 > is getting low on physical memory. Won't it shrink itself first in such= =20 > case before pagedaemon start its reclaimage? Buffer cache only caches buffers, it is not supposed to provide the file content cache, at least for VMIO. The buffer cache size is capped during system configuration, the algorithm to calculate the cache size is not easy to re-type, but look at the vfs_bio.c:kern_vfs_bio_buffer_alloc(). On the modern machines with 512MB of RAM of more, it is essentially 10% of the RAM which is dedicated to buffer cache. >=20 > When vm_lowmem condition finally fire, it will purge different data from= =20 > different subsystems, potentially still usable. UMA caches though have=20 > no valid data, only an allocation optimization. Shouldn't they be freed= =20 > first somehow, at least an unused part, as in my patch? Also I guess=20 > having more really free memory should make M_NOWAIT allocations to fail= =20 > less often. IMO this is not a right direction. My opinion is that M_NOWAIT allocation should be mosty banned from the top-level of the kernel, and then interrupt threads and i/o path should try hard to avoid allocations at all. Purging UMA caches on first sign of low memory condition would make UMA slower, possibly much slower for many workloads which are routinely handled now. Our code is accustomed to fast allocators, look at how many allocations typical syscall makes for temp buffers. Such change requires profiling of varying workloads to prove that it does not cause regressions. I suspect that what you do is tailored for single (ab)user of UMA. You might try to split UMA low memory handler into two, one for abuser, and one for the rest of caches. >=20 > >> I've made an experimental patch for UMA > >> (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20 > >> seconds return back to the system cached memory, unused for the last 20 > >> seconds. Algorithm is quite simple and patch seems like working, but I > >> am not sure whether I am approaching problem from the right side. Any > >> thoughts? >=20 > --=20 > Alexander Motin --PRJsLXMT40XvOKNo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSzDdYAAoJEJDCuSvBvK1B2jUP/3oSrPY1OWe65n/zdGXHTlu7 hw932AnxNCqLlUFsw8jAXp8x2tHf3mIZjCbVSFbz9Axjz5Q3G7cAlDLJJDGaY4qL FVZPi7VVU5/YODNkOAImIIZZtjjVNH4gGdLu22k+8mRSaA2/T2lLt0TdLf8aphIY iIGVkxaQOVymDQkoutHk3Jzn3gzAfc/c0gPY8ZwgIfzKi0YPFWHEvuB2SX4bdkes W846N53qKovjB6mNeIC2m20QL6vYWIzs/3B9BLRLNyvWpJIeDZx+WVtdWMN25Q/+ 934nFuZYGEItNafoTj90CsgXjWdwoQA9WO9QDxYhbiZ6TlhsPNjw7DAobY5OiOpJ TITx7k6bMrSkvrSAfVva3pBLK1aAmgxxaBTy+KXZcsCt658Afl3BsUJhv9J+IEyh sW8i2mDFsyq2cca1EiVmpTUpPDFZl9oc0kiN8Zlq5OET+NyeiPfVerauKSSmesSp 7+pWRPJae7veJvf2VAL8/zlt7MDaXHxYbqZpGoRsFW+xjXsOl9MDt8B0DdhVFO71 gE8KpRvJfbjRtRx3smGuDtlFb/QSUIsJo8bS7i2MtIFOeYto8JrAlJdVO7zpCadJ LcZWAtfXiM5xLbKkpof+9q95tE8w9VobtDvTJh7xNlACS1xOlICYS17nv0gCGMp0 YhLD5j3XgjN0lLUci2Zn =H1Fo -----END PGP SIGNATURE----- --PRJsLXMT40XvOKNo--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140107172025.GP59496>