Date: Thu, 24 Nov 2011 22:29:45 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Maxim Konovalov <maxim@freebsd.org> Cc: arch@freebsd.org, current@freebsd.org, igor@sysoev.ru, Maxim Dounin <mdounin@mdounin.ru> Subject: Re: RLIMIT_DATA and malloc(3) use of mmap(2) Message-ID: <20111124202945.GR50300@deviant.kiev.zoral.com.ua> In-Reply-To: <alpine.BSF.2.00.1111242056080.69998@mp2.macomnet.net> References: <20111122124410.GP50300@deviant.kiev.zoral.com.ua> <20111122154357.GI95664@mdounin.ru> <20111122154935.GR50300@deviant.kiev.zoral.com.ua> <alpine.BSF.2.00.1111231121260.69998@mp2.macomnet.net> <alpine.BSF.2.00.1111242056080.69998@mp2.macomnet.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--865IE93Xpm46K7pT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 24, 2011 at 09:38:19PM +0400, Maxim Konovalov wrote: > Hi Kostik, >=20 > On Wed, 23 Nov 2011, 11:22+0400, Maxim Konovalov wrote: >=20 > > [...] > > > Anyway, the patch needs testers before I will push it forward. > > > > > [igor's email was corrected] > > > > We will test it in out environment and let you know. > > > It seems I don't understand how it works. >=20 > Here is a test program: http://maxim.int.ru/stuff/malloc_test.c.txt >=20 > It allocates successfully 64x1MB chunks by malloc(), reallocf() and > realloc() with the following command line: >=20 > MALLOC_OPTIONS=3DJM limits -d 10m ./malloc_test >=20 > When we add L flag to the MALLOC_OPTIONS it starts to work strange: >=20 > It allocates 64MB via malloc() but fails to allocate more than 3MB by > realloc() and reallocf(). The non-failing 64MB allocation is easily explained by a bug. I forgot to account for the case when existing map entry was expanded, instead of new entry created due to mmap(2). The other part, in particular, the failure after 3MB, is in fact the correct behaviour. Jemalloc() caches allocation of the pages, and it allocates more then asked in the request. ktrace(1) shows the whole history. Malloc() first allocated 4MB for the needs of libc etc. Then, it allocated 4MB chunk which was used for satisfying the requests of 1-3MB. When the 4MB request came in, the allocation for 8MB was attempted, and failed, since 4MB (libc etc) + 8MB =3D 12MB > data limit. >=20 > More funny, the result varies from time to time: I have no explanation for this, and cannot reproduce the issue. Look at the ktrace. Overall, the test is quite curious but absolutely unrealistic. Fixed patch is available at http://people.freebsd.org/~kib/misc/map_datalimit.2.patch --865IE93Xpm46K7pT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7OqTkACgkQC3+MBN1Mb4hVFACginFQ/syGlHkbuL2ZtY20QdAX mqgAoKgirRQKnX1phG9DhOMUwQHm/B+C =5PsP -----END PGP SIGNATURE----- --865IE93Xpm46K7pT--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111124202945.GR50300>