Date: Mon, 22 Apr 2013 13:17:10 -0400 From: John Baldwin <jhb@freebsd.org> To: Freddie Cash <fjwcash@gmail.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>, Robert Waksmundzki <waksmundzki@gmail.com> Subject: Re: NUMA, cpuset and malloc Message-ID: <201304221317.10722.jhb@freebsd.org> In-Reply-To: <CAOjFWZ4inzabFpO4OH-8U%2B5PpXzxO1mVyCnig0T8ghgsx0PH%2BA@mail.gmail.com> References: <D2C5ECBF-1D71-4E58-93D8-E670CD55E27D@gmail.com> <201304221132.08194.jhb@freebsd.org> <CAOjFWZ4inzabFpO4OH-8U%2B5PpXzxO1mVyCnig0T8ghgsx0PH%2BA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday, April 22, 2013 12:29:29 pm Freddie Cash wrote: > On Mon, Apr 22, 2013 at 8:32 AM, John Baldwin <jhb@freebsd.org> wrote: > > > On Saturday, April 20, 2013 6:43:26 pm Robert Waksmundzki wrote: > > > On NUMA systems allocated memory is striped across local and non-local > > banks > > in order to have consistent performance in case the task is rescheduled to > > a > > different CPU socket. > > > When a process is pinned to a single CPU socket with cpuset having the > > memory allocator prefer local banks would probably improve performance. > > Default system behavior would stay the same and the optimization would > > only be > > triggered on big multi socket systems when administrator used cpuset > > (command > > mostly used for performance optimization anyway). > > > > > > Is this something currently implemented in FreeBSD? Is this even a good > > idea? > > > > You can get something sort of like this by enabling NUMA in your kernel > > (9.0 > > and later) and always pinning your processes with cpuset. (The simple NUMA > > bits always allocate memory in the memory domain the current thread is > > running in at the time of the fault.) > > > > How does one enable NUMA? > > A "grep -i numa /usr/src/sys/conf/NOTES /usr/src/sys/amd64/conf/NOTES" > turns up 0 hits for both 9-STABLE r248547 and 10-CURRENT (April 11, used > svnup so no way to get the exact revision number, that I know of). > > Or, is it enabled automatically? You have to chagne the VM_NDOMAIN setting. In recent HEAD and 9-stable you can do it in the kernel config (options VM_NDOMAIN=4 for example). In older HEAD and 9 you have to edit sys/amd64/include/vmparam.h or sys/i386/include/vmparam.h and change VM_NDOMAIN before building your kernel. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201304221317.10722.jhb>