Date: Wed, 27 Mar 2013 19:52:45 +0100 From: "Ronald Klop" <ronald-freebsd8@klop.yi.org> To: "Ian Lepore" <ian@freebsd.org>, Unga <unga888@yahoo.com> Cc: "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org> Subject: Re: FreeBSD 9.1 excessive memory allocations [SOLVED] Message-ID: <op.wumb17fn8527sy@212-182-167-131.ip.telfort.nl> In-Reply-To: <1364409226.37379.YahooMailNeo@web161906.mail.bf1.yahoo.com> References: <1364322902.78474.YahooMailNeo@web161904.mail.bf1.yahoo.com> <1364393170.36972.49.camel@revolution.hippie.lan> <1364409226.37379.YahooMailNeo@web161906.mail.bf1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 27 Mar 2013 19:33:46 +0100, Unga <unga888@yahoo.com> wrote: > > > ----- Original Message ----- > >> From: Ian Lepore <ian@FreeBSD.org> >> To: Unga <unga888@yahoo.com> >> Cc: "freebsd-stable@freebsd.org" <freebsd-stable@FreeBSD.org> >> Sent: Wednesday, March 27, 2013 2:06 PM >> Subject: Re: FreeBSD 9.1 excessive memory allocations >> >> On Tue, 2013-03-26 at 11:35 -0700, Unga wrote: >>> Hi all >>> >>> >>> I have a heavily threaded C application, developed on an Intel Core i5 >> laptop (2 cores) running FreeBSD 8.1-RELEASE. >>> >>> When this application compile and run on another Intel Core i7 laptop >>> (4 >> cores) running FreeBSD 9.1-RELEASE, this application immediately starts >> grabbing >> memory by over 100MB per second and soon exit with not enough RAM. >>> >>> >>> Both laptops having 4GB RAM. >>> >>> All malloc and free are mutex locked. >>> >>> Very rarely this problem happens on the i5 (2 cores) laptop too, but >>> on the >> i7 laptop, it happens every time. >>> >>> Appreciate any feedback to identify and fix this issue. >>> >>> Best regards >>> Unga >>> >> >> Too many moving parts, you need to partition the problem. Is it the >> change in OS (and especially libraries) that causes the problem, or the >> change in the number of cores (more concurrent threads) is exposing some >> sort of application-side race condition? Given the fact that it does >> occur on 2 cores + freebsd 8.1, even if more rarely, it's almost surely >> an application problem. >> Perhaps you could use a tool such as valgrind to help track down the >> runaway allocations? >> >> Another way to expose thread race problems is to force more thread >> context switches. A blunt tool for doing so is to set hz=5000 or even >> higher in /boot/loader.conf. I've never done that on a desktop or >> laptop system, only on embedded systems, but it should still work okay. >> If changing the application code is easier, you can get a similar effect >> by creating a thread whose only job is to preempt other threads, by >> using rtprio_thread() to set it to real time priority, then just have it >> sleep for short random intervals (microseconds), all it does is wakes up >> and goes right back to sleep. >> >> Also, FYI, there's no need to use a mutex in your application to protect >> allocations. The memory allocator in libc is thread-safe, and in fact >> is particularly designed for efficient multi-threaded allocation. >> >> -- Ian >> > > Dear Tony, Alexander, Jan, Ian and Jeremy > > Thank you very much for your very valuable comments. > > Problem seems to be solved. But require more testing. > > 1. Fixed an application-level crucial bug. This is nearly a 7000 lines C > app. It was really hard to see as the application is designed with 8 > dedicated threads. > > 2. At run-time, this application shoots up to about 400 dynamic threads > on the i7 machine, whereas on the i5 machine, it only shoots up 57 > dynamic threads. It was bit scaring, therefore, made a hard limit on > total number of threads to 64. > > Regarding mutex locks on allocations, as per the malloc(3), it says > small and medium size allocations are done from per thread caches, > therefore, thread-safe. My allocations are large in nature, about 5-7MB. The per thread caches are for speed. Not for thread-safety. Ronald.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.wumb17fn8527sy>