Date: Wed, 9 Nov 2016 18:42:30 +0000 (UTC) From: Jason Evans <jasone@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r308473 - in head: contrib/jemalloc contrib/jemalloc/doc contrib/jemalloc/include/jemalloc contrib/jemalloc/include/jemalloc/internal contrib/jemalloc/src lib/libc/stdlib/jemalloc Message-ID: <201611091842.uA9IgUet012596@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: jasone Date: Wed Nov 9 18:42:30 2016 New Revision: 308473 URL: https://svnweb.freebsd.org/changeset/base/308473 Log: Update jemalloc to 4.3.1. Added: head/contrib/jemalloc/include/jemalloc/internal/spin.h (contents, props changed) head/contrib/jemalloc/src/spin.c (contents, props changed) Modified: head/contrib/jemalloc/ChangeLog head/contrib/jemalloc/FREEBSD-Xlist head/contrib/jemalloc/FREEBSD-diffs head/contrib/jemalloc/VERSION head/contrib/jemalloc/doc/jemalloc.3 head/contrib/jemalloc/include/jemalloc/internal/arena.h head/contrib/jemalloc/include/jemalloc/internal/chunk.h head/contrib/jemalloc/include/jemalloc/internal/chunk_dss.h head/contrib/jemalloc/include/jemalloc/internal/ckh.h head/contrib/jemalloc/include/jemalloc/internal/huge.h head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal.h head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_decls.h head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_defs.h head/contrib/jemalloc/include/jemalloc/internal/mb.h head/contrib/jemalloc/include/jemalloc/internal/mutex.h head/contrib/jemalloc/include/jemalloc/internal/nstime.h head/contrib/jemalloc/include/jemalloc/internal/private_namespace.h head/contrib/jemalloc/include/jemalloc/internal/prng.h head/contrib/jemalloc/include/jemalloc/internal/prof.h head/contrib/jemalloc/include/jemalloc/internal/size_classes.h head/contrib/jemalloc/include/jemalloc/internal/tcache.h head/contrib/jemalloc/include/jemalloc/internal/tsd.h head/contrib/jemalloc/include/jemalloc/internal/util.h head/contrib/jemalloc/include/jemalloc/internal/witness.h head/contrib/jemalloc/include/jemalloc/jemalloc.h head/contrib/jemalloc/src/arena.c head/contrib/jemalloc/src/chunk.c head/contrib/jemalloc/src/chunk_dss.c head/contrib/jemalloc/src/ckh.c head/contrib/jemalloc/src/ctl.c head/contrib/jemalloc/src/huge.c head/contrib/jemalloc/src/jemalloc.c head/contrib/jemalloc/src/mutex.c head/contrib/jemalloc/src/nstime.c head/contrib/jemalloc/src/pages.c head/contrib/jemalloc/src/prof.c head/contrib/jemalloc/src/rtree.c head/contrib/jemalloc/src/stats.c head/contrib/jemalloc/src/tcache.c head/contrib/jemalloc/src/tsd.c head/contrib/jemalloc/src/util.c head/lib/libc/stdlib/jemalloc/Makefile.inc Modified: head/contrib/jemalloc/ChangeLog ============================================================================== --- head/contrib/jemalloc/ChangeLog Wed Nov 9 18:00:50 2016 (r308472) +++ head/contrib/jemalloc/ChangeLog Wed Nov 9 18:42:30 2016 (r308473) @@ -4,6 +4,59 @@ brevity. Much more detail can be found https://github.com/jemalloc/jemalloc +* 4.3.1 (November 7, 2016) + + Bug fixes: + - Fix a severe virtual memory leak. This regression was first released in + 4.3.0. (@interwq, @jasone) + - Refactor atomic and prng APIs to restore support for 32-bit platforms that + use pre-C11 toolchains, e.g. FreeBSD's mips. (@jasone) + +* 4.3.0 (November 4, 2016) + + This is the first release that passes the test suite for multiple Windows + configurations, thanks in large part to @glandium setting up continuous + integration via AppVeyor (and Travis CI for Linux and OS X). + + New features: + - Add "J" (JSON) support to malloc_stats_print(). (@jasone) + - Add Cray compiler support. (@ronawho) + + Optimizations: + - Add/use adaptive spinning for bootstrapping and radix tree node + initialization. (@jasone) + + Bug fixes: + - Fix large allocation to search starting in the optimal size class heap, + which can substantially reduce virtual memory churn and fragmentation. This + regression was first released in 4.0.0. (@mjp41, @jasone) + - Fix stats.arenas.<i>.nthreads accounting. (@interwq) + - Fix and simplify decay-based purging. (@jasone) + - Make DSS (sbrk(2)-related) operations lockless, which resolves potential + deadlocks during thread exit. (@jasone) + - Fix over-sized allocation of radix tree leaf nodes. (@mjp41, @ogaun, + @jasone) + - Fix over-sized allocation of arena_t (plus associated stats) data + structures. (@jasone, @interwq) + - Fix EXTRA_CFLAGS to not affect configuration. (@jasone) + - Fix a Valgrind integration bug. (@ronawho) + - Disallow 0x5a junk filling when running in Valgrind. (@jasone) + - Fix a file descriptor leak on Linux. This regression was first released in + 4.2.0. (@vsarunas, @jasone) + - Fix static linking of jemalloc with glibc. (@djwatson) + - Use syscall(2) rather than {open,read,close}(2) during boot on Linux. This + works around other libraries' system call wrappers performing reentrant + allocation. (@kspinka, @Whissi, @jasone) + - Fix OS X default zone replacement to work with OS X 10.12. (@glandium, + @jasone) + - Fix cached memory management to avoid needless commit/decommit operations + during purging, which resolves permanent virtual memory map fragmentation + issues on Windows. (@mjp41, @jasone) + - Fix TSD fetches to avoid (recursive) allocation. This is relevant to + non-TLS and Windows configurations. (@jasone) + - Fix malloc_conf overriding to work on Windows. (@jasone) + - Forcibly disable lazy-lock on Windows (was forcibly *enabled*). (@jasone) + * 4.2.1 (June 8, 2016) Bug fixes: Modified: head/contrib/jemalloc/FREEBSD-Xlist ============================================================================== --- head/contrib/jemalloc/FREEBSD-Xlist Wed Nov 9 18:00:50 2016 (r308472) +++ head/contrib/jemalloc/FREEBSD-Xlist Wed Nov 9 18:42:30 2016 (r308473) @@ -1,6 +1,8 @@ $FreeBSD$ +.appveyor.yml .autom4te.cfg .git* +.travis.yml FREEBSD-* INSTALL Makefile* Modified: head/contrib/jemalloc/FREEBSD-diffs ============================================================================== --- head/contrib/jemalloc/FREEBSD-diffs Wed Nov 9 18:00:50 2016 (r308472) +++ head/contrib/jemalloc/FREEBSD-diffs Wed Nov 9 18:42:30 2016 (r308473) @@ -1,11 +1,11 @@ diff --git a/doc/jemalloc.xml.in b/doc/jemalloc.xml.in -index c4a44e3..4626e9b 100644 +index 3d2e721..b361db2 100644 --- a/doc/jemalloc.xml.in +++ b/doc/jemalloc.xml.in @@ -53,11 +53,23 @@ <para>This manual describes jemalloc @jemalloc_version@. More information can be found at the <ulink - url="http://www.canonware.com/jemalloc/">jemalloc website</ulink>.</para> + url="http://jemalloc.net/">jemalloc website</ulink>.</para> + + <para>The following configuration options are enabled in libc's built-in + jemalloc: <option>--enable-fill</option>, @@ -27,30 +27,30 @@ index c4a44e3..4626e9b 100644 <refsect2> <title>Standard API</title> <funcprototype> -@@ -2961,4 +2973,18 @@ malloc_conf = "lg_chunk:24";]]></programlisting></para> - <para>The <function>posix_memalign<parameter/></function> function conforms - to IEEE Std 1003.1-2001 (“POSIX.1”).</para> +@@ -2963,4 +2975,18 @@ malloc_conf = "lg_chunk:24";]]></programlisting></para> + <para>The <function>posix_memalign()</function> function conforms + to IEEE Std 1003.1-2001 (<quote>POSIX.1</quote>).</para> </refsect1> + <refsect1 id="history"> + <title>HISTORY</title> -+ <para>The <function>malloc_usable_size<parameter/></function> and -+ <function>posix_memalign<parameter/></function> functions first appeared in -+ FreeBSD 7.0.</para> -+ -+ <para>The <function>aligned_alloc<parameter/></function>, -+ <function>malloc_stats_print<parameter/></function>, and -+ <function>mallctl*<parameter/></function> functions first appeared in -+ FreeBSD 10.0.</para> ++ <para>The <function>malloc_usable_size()</function> and ++ <function>posix_memalign()</function> functions first appeared in FreeBSD ++ 7.0.</para> ++ ++ <para>The <function>aligned_alloc()</function>, ++ <function>malloc_stats_print()</function>, and ++ <function>mallctl*()</function> functions first appeared in FreeBSD ++ 10.0.</para> + -+ <para>The <function>*allocx<parameter/></function> functions first appeared -+ in FreeBSD 11.0.</para> ++ <para>The <function>*allocx()</function> functions first appeared in FreeBSD ++ 11.0.</para> + </refsect1> </refentry> diff --git a/include/jemalloc/internal/arena.h b/include/jemalloc/internal/arena.h -index b1de2b6..da6b6d2 100644 +index f39ce54..a3ba55d 100644 --- a/include/jemalloc/internal/arena.h +++ b/include/jemalloc/internal/arena.h -@@ -718,8 +718,13 @@ arena_miscelm_get_mutable(arena_chunk_t *chunk, size_t pageind) +@@ -719,8 +719,13 @@ arena_miscelm_get_mutable(arena_chunk_t *chunk, size_t pageind) JEMALLOC_ALWAYS_INLINE const arena_chunk_map_misc_t * arena_miscelm_get_const(const arena_chunk_t *chunk, size_t pageind) { @@ -64,7 +64,7 @@ index b1de2b6..da6b6d2 100644 } JEMALLOC_ALWAYS_INLINE size_t -@@ -778,8 +783,13 @@ arena_mapbitsp_get_mutable(arena_chunk_t *chunk, size_t pageind) +@@ -779,8 +784,13 @@ arena_mapbitsp_get_mutable(arena_chunk_t *chunk, size_t pageind) JEMALLOC_ALWAYS_INLINE const size_t * arena_mapbitsp_get_const(const arena_chunk_t *chunk, size_t pageind) { @@ -79,7 +79,7 @@ index b1de2b6..da6b6d2 100644 JEMALLOC_ALWAYS_INLINE size_t diff --git a/include/jemalloc/internal/jemalloc_internal.h.in b/include/jemalloc/internal/jemalloc_internal.h.in -index 8f82edd..78e2df2 100644 +index fdc8fef..56a35a4 100644 --- a/include/jemalloc/internal/jemalloc_internal.h.in +++ b/include/jemalloc/internal/jemalloc_internal.h.in @@ -8,6 +8,9 @@ @@ -108,7 +108,7 @@ index 8f82edd..78e2df2 100644 static const bool config_prof = #ifdef JEMALLOC_PROF diff --git a/include/jemalloc/internal/jemalloc_internal_decls.h b/include/jemalloc/internal/jemalloc_internal_decls.h -index 2b8ca5d..42d97f2 100644 +index c907d91..4626632 100644 --- a/include/jemalloc/internal/jemalloc_internal_decls.h +++ b/include/jemalloc/internal/jemalloc_internal_decls.h @@ -1,6 +1,9 @@ @@ -122,10 +122,10 @@ index 2b8ca5d..42d97f2 100644 #ifdef _WIN32 # include <windows.h> diff --git a/include/jemalloc/internal/mutex.h b/include/jemalloc/internal/mutex.h -index 5221799..60ab041 100644 +index b442d2d..76518db 100644 --- a/include/jemalloc/internal/mutex.h +++ b/include/jemalloc/internal/mutex.h -@@ -52,9 +52,6 @@ struct malloc_mutex_s { +@@ -57,9 +57,6 @@ struct malloc_mutex_s { #ifdef JEMALLOC_LAZY_LOCK extern bool isthreaded; @@ -135,7 +135,7 @@ index 5221799..60ab041 100644 #endif bool malloc_mutex_init(malloc_mutex_t *mutex, const char *name, -@@ -62,6 +59,7 @@ bool malloc_mutex_init(malloc_mutex_t *mutex, const char *name, +@@ -67,6 +64,7 @@ bool malloc_mutex_init(malloc_mutex_t *mutex, const char *name, void malloc_mutex_prefork(tsdn_t *tsdn, malloc_mutex_t *mutex); void malloc_mutex_postfork_parent(tsdn_t *tsdn, malloc_mutex_t *mutex); void malloc_mutex_postfork_child(tsdn_t *tsdn, malloc_mutex_t *mutex); @@ -144,10 +144,10 @@ index 5221799..60ab041 100644 #endif /* JEMALLOC_H_EXTERNS */ diff --git a/include/jemalloc/internal/private_symbols.txt b/include/jemalloc/internal/private_symbols.txt -index f2b6a55..69369c9 100644 +index 87c8c9b..df576f6 100644 --- a/include/jemalloc/internal/private_symbols.txt +++ b/include/jemalloc/internal/private_symbols.txt -@@ -311,7 +311,6 @@ iralloct_realign +@@ -307,7 +307,6 @@ iralloct_realign isalloc isdalloct isqalloc @@ -335,7 +335,7 @@ index f943891..47d032c 100755 +#include "jemalloc_FreeBSD.h" EOF diff --git a/src/jemalloc.c b/src/jemalloc.c -index 5d1f493..46dd1d1 100644 +index 38650ff..f659b55 100644 --- a/src/jemalloc.c +++ b/src/jemalloc.c @@ -4,6 +4,10 @@ @@ -347,9 +347,9 @@ index 5d1f493..46dd1d1 100644 +__sym_compat(_malloc_options, __malloc_options_1_0, FBSD_1.0); + /* Runtime configuration options. */ - const char *je_malloc_conf JEMALLOC_ATTR(weak); - bool opt_abort = -@@ -2673,6 +2677,107 @@ je_malloc_usable_size(JEMALLOC_USABLE_SIZE_CONST void *ptr) + const char *je_malloc_conf + #ifndef _WIN32 +@@ -2756,6 +2760,107 @@ je_malloc_usable_size(JEMALLOC_USABLE_SIZE_CONST void *ptr) */ /******************************************************************************/ /* @@ -457,7 +457,7 @@ index 5d1f493..46dd1d1 100644 * The following functions are used by threading libraries for protection of * malloc during fork(). */ -@@ -2814,4 +2919,11 @@ jemalloc_postfork_child(void) +@@ -2894,4 +2999,11 @@ jemalloc_postfork_child(void) ctl_postfork_child(tsd_tsdn(tsd)); } @@ -470,7 +470,7 @@ index 5d1f493..46dd1d1 100644 + /******************************************************************************/ diff --git a/src/mutex.c b/src/mutex.c -index a1fac34..a24e420 100644 +index 6333e73..13f8d79 100644 --- a/src/mutex.c +++ b/src/mutex.c @@ -66,6 +66,17 @@ pthread_create(pthread_t *__restrict thread, @@ -491,7 +491,7 @@ index a1fac34..a24e420 100644 #endif bool -@@ -140,7 +151,7 @@ malloc_mutex_postfork_child(tsdn_t *tsdn, malloc_mutex_t *mutex) +@@ -142,7 +153,7 @@ malloc_mutex_postfork_child(tsdn_t *tsdn, malloc_mutex_t *mutex) } bool @@ -500,7 +500,7 @@ index a1fac34..a24e420 100644 { #ifdef JEMALLOC_MUTEX_INIT_CB -@@ -154,3 +165,14 @@ malloc_mutex_boot(void) +@@ -156,3 +167,14 @@ malloc_mutex_boot(void) #endif return (false); } @@ -516,7 +516,7 @@ index a1fac34..a24e420 100644 +#endif +} diff --git a/src/util.c b/src/util.c -index a1c4a2a..04f9153 100644 +index 7905267..bee1c77 100644 --- a/src/util.c +++ b/src/util.c @@ -67,6 +67,22 @@ wrtmessage(void *cbopaque, const char *s) Modified: head/contrib/jemalloc/VERSION ============================================================================== --- head/contrib/jemalloc/VERSION Wed Nov 9 18:00:50 2016 (r308472) +++ head/contrib/jemalloc/VERSION Wed Nov 9 18:42:30 2016 (r308473) @@ -1 +1 @@ -4.2.1-0-g3de035335255d553bdb344c32ffdb603816195d8 +4.3.1-0-g0110fa8451af905affd77c3bea0d545fee2251b2 Modified: head/contrib/jemalloc/doc/jemalloc.3 ============================================================================== --- head/contrib/jemalloc/doc/jemalloc.3 Wed Nov 9 18:00:50 2016 (r308472) +++ head/contrib/jemalloc/doc/jemalloc.3 Wed Nov 9 18:42:30 2016 (r308473) @@ -2,12 +2,12 @@ .\" Title: JEMALLOC .\" Author: Jason Evans .\" Generator: DocBook XSL Stylesheets v1.76.1 <http://docbook.sf.net/>; -.\" Date: 06/08/2016 +.\" Date: 11/08/2016 .\" Manual: User Manual -.\" Source: jemalloc 4.2.1-0-g3de035335255d553bdb344c32ffdb603816195d8 +.\" Source: jemalloc 4.3.1-0-g0110fa8451af905affd77c3bea0d545fee2251b2 .\" Language: English .\" -.TH "JEMALLOC" "3" "06/08/2016" "jemalloc 4.2.1-0-g3de035335255" "User Manual" +.TH "JEMALLOC" "3" "11/08/2016" "jemalloc 4.3.1-0-g0110fa8451af" "User Manual" .\" ----------------------------------------------------------------- .\" * Define some portability stuff .\" ----------------------------------------------------------------- @@ -31,7 +31,7 @@ jemalloc \- general purpose memory allocation functions .SH "LIBRARY" .PP -This manual describes jemalloc 4\&.2\&.1\-0\-g3de035335255d553bdb344c32ffdb603816195d8\&. More information can be found at the +This manual describes jemalloc 4\&.3\&.1\-0\-g0110fa8451af905affd77c3bea0d545fee2251b2\&. More information can be found at the \m[blue]\fBjemalloc website\fR\m[]\&\s-2\u[1]\d\s+2\&. .PP The following configuration options are enabled in libc\*(Aqs built\-in jemalloc: @@ -101,26 +101,26 @@ const char *\fImalloc_conf\fR; .SS "Standard API" .PP The -\fBmalloc\fR\fB\fR +malloc() function allocates \fIsize\fR bytes of uninitialized memory\&. The allocated space is suitably aligned (after possible pointer coercion) for storage of any type of object\&. .PP The -\fBcalloc\fR\fB\fR +calloc() function allocates space for \fInumber\fR objects, each \fIsize\fR bytes in length\&. The result is identical to calling -\fBmalloc\fR\fB\fR +malloc() with an argument of \fInumber\fR * \fIsize\fR, with the exception that the allocated memory is explicitly initialized to zero bytes\&. .PP The -\fBposix_memalign\fR\fB\fR +posix_memalign() function allocates \fIsize\fR bytes of memory such that the allocation\*(Aqs base address is a multiple of @@ -131,7 +131,7 @@ must be a power of 2 at least as large a sizeof(\fBvoid *\fR)\&. .PP The -\fBaligned_alloc\fR\fB\fR +aligned_alloc() function allocates \fIsize\fR bytes of memory such that the allocation\*(Aqs base address is a multiple of @@ -143,7 +143,7 @@ is not an integral multiple of \fIalignment\fR\&. .PP The -\fBrealloc\fR\fB\fR +realloc() function changes the size of the previously allocated memory referenced by \fIptr\fR to @@ -151,19 +151,19 @@ to bytes\&. The contents of the memory are unchanged up to the lesser of the new and old sizes\&. If the new size is larger, the contents of the newly allocated portion of the memory are undefined\&. Upon success, the memory referenced by \fIptr\fR is freed and a pointer to the newly allocated memory is returned\&. Note that -\fBrealloc\fR\fB\fR +realloc() may move the memory allocation, resulting in a different return value than \fIptr\fR\&. If \fIptr\fR is \fBNULL\fR, the -\fBrealloc\fR\fB\fR +realloc() function behaves identically to -\fBmalloc\fR\fB\fR +malloc() for the specified size\&. .PP The -\fBfree\fR\fB\fR +free() function causes the allocated memory referenced by \fIptr\fR to be made available for future allocations\&. If @@ -173,13 +173,13 @@ is .SS "Non\-standard API" .PP The -\fBmallocx\fR\fB\fR, -\fBrallocx\fR\fB\fR, -\fBxallocx\fR\fB\fR, -\fBsallocx\fR\fB\fR, -\fBdallocx\fR\fB\fR, -\fBsdallocx\fR\fB\fR, and -\fBnallocx\fR\fB\fR +mallocx(), +rallocx(), +xallocx(), +sallocx(), +dallocx(), +sdallocx(), and +nallocx() functions all have a \fIflags\fR argument that can be used to specify options\&. The functions only check the options that are contextually relevant\&. Use bitwise or (|) operations to specify one or more of the following: @@ -211,7 +211,7 @@ Initialize newly allocated memory to con .RS 4 Use the thread\-specific cache (tcache) specified by the identifier \fItc\fR, which must have been acquired via the -"tcache\&.create" +tcache\&.create mallctl\&. This macro does not validate that \fItc\fR specifies a valid identifier\&. @@ -238,7 +238,7 @@ specifies an arena index in the valid ra .RE .PP The -\fBmallocx\fR\fB\fR +mallocx() function allocates at least \fIsize\fR bytes of memory, and returns a pointer to the base address of the allocation\&. Behavior is undefined if @@ -247,7 +247,7 @@ is \fB0\fR\&. .PP The -\fBrallocx\fR\fB\fR +rallocx() function resizes the allocation at \fIptr\fR to be at least @@ -258,7 +258,7 @@ is \fB0\fR\&. .PP The -\fBxallocx\fR\fB\fR +xallocx() function resizes the allocation at \fIptr\fR in place to be at least @@ -274,33 +274,33 @@ is (\fIsize\fR + \fIextra\fR > \fBSIZE_T_MAX\fR)\&. .PP The -\fBsallocx\fR\fB\fR +sallocx() function returns the real size of the allocation at \fIptr\fR\&. .PP The -\fBdallocx\fR\fB\fR +dallocx() function causes the memory referenced by \fIptr\fR to be made available for future allocations\&. .PP The -\fBsdallocx\fR\fB\fR +sdallocx() function is an extension of -\fBdallocx\fR\fB\fR +dallocx() with a \fIsize\fR parameter to allow the caller to pass in the allocation size as an optimization\&. The minimum valid input size is the original requested size of the allocation, and the maximum valid input size is the corresponding value returned by -\fBnallocx\fR\fB\fR +nallocx() or -\fBsallocx\fR\fB\fR\&. +sallocx()\&. .PP The -\fBnallocx\fR\fB\fR +nallocx() function allocates no memory, but it performs the same size computation as the -\fBmallocx\fR\fB\fR +mallocx() function, and returns the real size of the allocation that would result from the equivalent -\fBmallocx\fR\fB\fR +mallocx() function call, or \fB0\fR if the inputs exceed the maximum supported size class and/or alignment\&. Behavior is undefined if @@ -309,7 +309,7 @@ is \fB0\fR\&. .PP The -\fBmallctl\fR\fB\fR +mallctl() function provides a general interface for introspecting the memory allocator, as well as setting modifiable parameters and triggering actions\&. The period\-separated \fIname\fR argument specifies a location in a tree\-structured namespace; see the @@ -328,10 +328,12 @@ and \fB0\fR\&. .PP The -\fBmallctlnametomib\fR\fB\fR -function provides a way to avoid repeated name lookups for applications that repeatedly query the same portion of the namespace, by translating a name to a \(lqManagement Information Base\(rq (MIB) that can be passed repeatedly to -\fBmallctlbymib\fR\fB\fR\&. Upon successful return from -\fBmallctlnametomib\fR\fB\fR, +mallctlnametomib() +function provides a way to avoid repeated name lookups for applications that repeatedly query the same portion of the namespace, by translating a name to a +\(lqManagement Information Base\(rq +(MIB) that can be passed repeatedly to +mallctlbymib()\&. Upon successful return from +mallctlnametomib(), \fImibp\fR contains an array of \fI*miblenp\fR @@ -343,7 +345,7 @@ and the input value of \fI*miblenp\fR\&. Thus it is possible to pass a \fI*miblenp\fR that is smaller than the number of period\-separated name components, which results in a partial MIB that can be used as the basis for constructing a complete MIB\&. For name components that are integers (e\&.g\&. the 2 in -"arenas\&.bin\&.2\&.size"), the corresponding MIB component will always be that integer\&. Therefore, it is legitimate to construct code like the following: +arenas\&.bin\&.2\&.size), the corresponding MIB component will always be that integer\&. Therefore, it is legitimate to construct code like the following: .sp .if n \{\ .RS 4 @@ -372,35 +374,50 @@ for (i = 0; i < nbins; i++) { .\} .PP The -\fBmalloc_stats_print\fR\fB\fR -function writes human\-readable summary statistics via the +malloc_stats_print() +function writes summary statistics via the \fIwrite_cb\fR callback function pointer and \fIcbopaque\fR data passed to \fIwrite_cb\fR, or -\fBmalloc_message\fR\fB\fR +malloc_message() if \fIwrite_cb\fR is -\fBNULL\fR\&. This function can be called repeatedly\&. General information that never changes during execution can be omitted by specifying "g" as a character within the +\fBNULL\fR\&. The statistics are presented in human\-readable form unless +\(lqJ\(rq +is specified as a character within the +\fIopts\fR +string, in which case the statistics are presented in +\m[blue]\fBJSON format\fR\m[]\&\s-2\u[2]\d\s+2\&. This function can be called repeatedly\&. General information that never changes during execution can be omitted by specifying +\(lqg\(rq +as a character within the \fIopts\fR string\&. Note that -\fBmalloc_message\fR\fB\fR +malloc_message() uses the -\fBmallctl*\fR\fB\fR +mallctl*() functions internally, so inconsistent statistics can be reported if multiple threads use these functions simultaneously\&. If \fB\-\-enable\-stats\fR -is specified during configuration, \(lqm\(rq and \(lqa\(rq can be specified to omit merged arena and per arena statistics, respectively; \(lqb\(rq, \(lql\(rq, and \(lqh\(rq can be specified to omit per size class statistics for bins, large objects, and huge objects, respectively\&. Unrecognized characters are silently ignored\&. Note that thread caching may prevent some statistics from being completely up to date, since extra locking would be required to merge counters that track thread cache operations\&. +is specified during configuration, +\(lqm\(rq +and +\(lqa\(rq +can be specified to omit merged arena and per arena statistics, respectively; +\(lqb\(rq, +\(lql\(rq, and +\(lqh\(rq +can be specified to omit per size class statistics for bins, large objects, and huge objects, respectively\&. Unrecognized characters are silently ignored\&. Note that thread caching may prevent some statistics from being completely up to date, since extra locking would be required to merge counters that track thread cache operations\&. .PP The -\fBmalloc_usable_size\fR\fB\fR +malloc_usable_size() function returns the usable size of the allocation pointed to by \fIptr\fR\&. The return value may be larger than the size that was requested during allocation\&. The -\fBmalloc_usable_size\fR\fB\fR +malloc_usable_size() function is not a mechanism for in\-place -\fBrealloc\fR\fB\fR; rather it is provided solely as a tool for introspection purposes\&. Any discrepancy between the requested allocation size and the size reported by -\fBmalloc_usable_size\fR\fB\fR +realloc(); rather it is provided solely as a tool for introspection purposes\&. Any discrepancy between the requested allocation size and the size reported by +malloc_usable_size() should not be depended on, since such behavior is entirely implementation\-dependent\&. .SH "TUNING" .PP @@ -408,12 +425,14 @@ Once, when the first call is made to one .PP The string specified via \fB\-\-with\-malloc\-conf\fR, the string pointed to by the global variable -\fImalloc_conf\fR, the \(lqname\(rq of the file referenced by the symbolic link named +\fImalloc_conf\fR, the +\(lqname\(rq +of the file referenced by the symbolic link named /etc/malloc\&.conf, and the value of the environment variable \fBMALLOC_CONF\fR, will be interpreted, in that order, from left to right as options\&. Note that \fImalloc_conf\fR may be read before -\fBmain\fR\fB\fR +main() is entered, so the declaration of \fImalloc_conf\fR should specify an initializer that contains the final value to be read by jemalloc\&. @@ -427,15 +446,15 @@ and can be safely set any time prior to program invocation\&. .PP An options string is a comma\-separated list of option:value pairs\&. There is one key corresponding to each -"opt\&.*" +opt\&.* mallctl (see the MALLCTL NAMESPACE section for options documentation)\&. For example, abort:true,narenas:1 sets the -"opt\&.abort" +opt\&.abort and -"opt\&.narenas" +opt\&.narenas options\&. Some options have boolean values (true/false), others have integer values (base 8, 10, or 16, depending on prefix), and yet others have raw string values\&. .SH "IMPLEMENTATION NOTES" .PP @@ -460,22 +479,22 @@ Memory is conceptually broken into equal .PP Small objects are managed in groups by page runs\&. Each run maintains a bitmap to track which regions are in use\&. Allocation requests that are no more than half the quantum (8 or 16, depending on architecture) are rounded up to the nearest power of two that is at least sizeof(\fBdouble\fR)\&. All other object size classes are multiples of the quantum, spaced such that there are four size classes for each doubling in size, which limits internal fragmentation to approximately 20% for all but the smallest size classes\&. Small size classes are smaller than four times the page size, large size classes are smaller than the chunk size (see the -"opt\&.lg_chunk" +opt\&.lg_chunk option), and huge size classes extend from the chunk size up to the largest size class that does not exceed \fBPTRDIFF_MAX\fR\&. .PP Allocations are packed tightly together, which can be an issue for multi\-threaded applications\&. If you need to assure that allocations do not suffer from cacheline sharing, round your allocation requests up to the nearest multiple of the cacheline size, or specify cacheline alignment when allocating\&. .PP The -\fBrealloc\fR\fB\fR, -\fBrallocx\fR\fB\fR, and -\fBxallocx\fR\fB\fR +realloc(), +rallocx(), and +xallocx() functions may resize allocations without moving them under limited circumstances\&. Unlike the -\fB*allocx\fR\fB\fR +*allocx() API, the standard API does not officially round up the usable size of an allocation to the nearest size class, so technically it is necessary to call -\fBrealloc\fR\fB\fR +realloc() to grow e\&.g\&. a 9\-byte allocation to 16 bytes, or shrink a 16\-byte allocation to 9 bytes\&. Growth and shrinkage trivially succeeds in place as long as the pre\-size and post\-size both round up to the same size class\&. No other API guarantees are made regarding in\-place resizing, but the current implementation also tries to resize large and huge allocations in place, as long as the pre\-size and post\-size are both large or both huge\&. In such cases shrinkage always succeeds for large size classes, but for huge size classes the chunk allocator must support splitting (see -"arena\&.<i>\&.chunk_hooks")\&. Growth only succeeds if the trailing memory is currently available, and additionally for huge size classes the chunk allocator must support merging\&. +arena\&.<i>\&.chunk_hooks)\&. Growth only succeeds if the trailing memory is currently available, and additionally for huge size classes the chunk allocator must support merging\&. .PP Assuming 2 MiB chunks, 4 KiB pages, and a 16\-byte quantum on a 64\-bit system, the size classes in each category are as shown in Table 1\&. @@ -663,7 +682,7 @@ T} .SH "MALLCTL NAMESPACE" .PP The following names are defined in the namespace accessible via the -\fBmallctl*\fR\fB\fR +mallctl*() functions\&. Value types are specified in parentheses, their readable/writable statuses are encoded as rw, r\-, @@ -673,118 +692,118 @@ r\-, or <j> indicates an integer component, where the integer varies from 0 to some upper value that must be determined via introspection\&. In the case of -"stats\&.arenas\&.<i>\&.*", +stats\&.arenas\&.<i>\&.*, <i> equal to -"arenas\&.narenas" +arenas\&.narenas can be used to access the summation of statistics from all arenas\&. Take special note of the -"epoch" +epoch mallctl, which controls refreshing of cached dynamic statistics\&. .PP -"version" (\fBconst char *\fR) r\- +version (\fBconst char *\fR) r\- .RS 4 Return the jemalloc version string\&. .RE .PP -"epoch" (\fBuint64_t\fR) rw +epoch (\fBuint64_t\fR) rw .RS 4 If a value is passed in, refresh the data from which the -\fBmallctl*\fR\fB\fR +mallctl*() functions report values, and increment the epoch\&. Return the current epoch\&. This is useful for detecting whether another thread caused a refresh\&. .RE .PP -"config\&.cache_oblivious" (\fBbool\fR) r\- +config\&.cache_oblivious (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-cache\-oblivious\fR was specified during build configuration\&. .RE .PP -"config\&.debug" (\fBbool\fR) r\- +config\&.debug (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-debug\fR was specified during build configuration\&. .RE .PP -"config\&.fill" (\fBbool\fR) r\- +config\&.fill (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-fill\fR was specified during build configuration\&. .RE .PP -"config\&.lazy_lock" (\fBbool\fR) r\- +config\&.lazy_lock (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-lazy\-lock\fR was specified during build configuration\&. .RE .PP -"config\&.malloc_conf" (\fBconst char *\fR) r\- +config\&.malloc_conf (\fBconst char *\fR) r\- .RS 4 Embedded configure\-time\-specified run\-time options string, empty unless \fB\-\-with\-malloc\-conf\fR was specified during build configuration\&. .RE .PP -"config\&.munmap" (\fBbool\fR) r\- +config\&.munmap (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-munmap\fR was specified during build configuration\&. .RE .PP -"config\&.prof" (\fBbool\fR) r\- +config\&.prof (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-prof\fR was specified during build configuration\&. .RE .PP -"config\&.prof_libgcc" (\fBbool\fR) r\- +config\&.prof_libgcc (\fBbool\fR) r\- .RS 4 \fB\-\-disable\-prof\-libgcc\fR was not specified during build configuration\&. .RE .PP -"config\&.prof_libunwind" (\fBbool\fR) r\- +config\&.prof_libunwind (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-prof\-libunwind\fR was specified during build configuration\&. .RE .PP -"config\&.stats" (\fBbool\fR) r\- +config\&.stats (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-stats\fR was specified during build configuration\&. .RE .PP -"config\&.tcache" (\fBbool\fR) r\- +config\&.tcache (\fBbool\fR) r\- .RS 4 \fB\-\-disable\-tcache\fR was not specified during build configuration\&. .RE .PP -"config\&.tls" (\fBbool\fR) r\- +config\&.tls (\fBbool\fR) r\- .RS 4 \fB\-\-disable\-tls\fR was not specified during build configuration\&. .RE .PP -"config\&.utrace" (\fBbool\fR) r\- +config\&.utrace (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-utrace\fR was specified during build configuration\&. .RE .PP -"config\&.valgrind" (\fBbool\fR) r\- +config\&.valgrind (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-valgrind\fR was specified during build configuration\&. .RE .PP -"config\&.xmalloc" (\fBbool\fR) r\- +config\&.xmalloc (\fBbool\fR) r\- .RS 4 \fB\-\-enable\-xmalloc\fR was specified during build configuration\&. .RE .PP -"opt\&.abort" (\fBbool\fR) r\- +opt\&.abort (\fBbool\fR) r\- .RS 4 Abort\-on\-warning enabled/disabled\&. If true, most warnings are fatal\&. The process will call \fBabort\fR(3) @@ -793,115 +812,132 @@ in these cases\&. This option is disable is specified during configuration, in which case it is enabled by default\&. .RE .PP -"opt\&.dss" (\fBconst char *\fR) r\- +opt\&.dss (\fBconst char *\fR) r\- .RS 4 dss (\fBsbrk\fR(2)) allocation precedence as related to \fBmmap\fR(2) allocation\&. The following settings are supported if \fBsbrk\fR(2) -is supported by the operating system: \(lqdisabled\(rq, \(lqprimary\(rq, and \(lqsecondary\(rq; otherwise only \(lqdisabled\(rq is supported\&. The default is \(lqsecondary\(rq if +is supported by the operating system: +\(lqdisabled\(rq, +\(lqprimary\(rq, and +\(lqsecondary\(rq; otherwise only +\(lqdisabled\(rq +is supported\&. The default is +\(lqsecondary\(rq +if \fBsbrk\fR(2) -is supported by the operating system; \(lqdisabled\(rq otherwise\&. +is supported by the operating system; +\(lqdisabled\(rq +otherwise\&. .RE .PP -"opt\&.lg_chunk" (\fBsize_t\fR) r\- +opt\&.lg_chunk (\fBsize_t\fR) r\- .RS 4 Virtual memory chunk size (log base 2)\&. If a chunk size outside the supported size range is specified, the size is silently clipped to the minimum/maximum supported size\&. The default chunk size is 2 MiB (2^21)\&. .RE .PP -"opt\&.narenas" (\fBunsigned\fR) r\- +opt\&.narenas (\fBunsigned\fR) r\- .RS 4 Maximum number of arenas to use for automatic multiplexing of threads and arenas\&. The default is four times the number of CPUs, or one if there is a single CPU\&. .RE .PP -"opt\&.purge" (\fBconst char *\fR) r\- +opt\&.purge (\fBconst char *\fR) r\- .RS 4 Purge mode is \(lqratio\(rq (default) or \(lqdecay\(rq\&. See -"opt\&.lg_dirty_mult" +opt\&.lg_dirty_mult for details of the ratio mode\&. See -"opt\&.decay_time" +opt\&.decay_time for details of the decay mode\&. .RE .PP -"opt\&.lg_dirty_mult" (\fBssize_t\fR) r\- +opt\&.lg_dirty_mult (\fBssize_t\fR) r\- .RS 4 Per\-arena minimum ratio (log base 2) of active to dirty pages\&. Some dirty unused pages may be allowed to accumulate, within the limit set by the ratio (or one chunk worth of dirty pages, whichever is greater), before informing the kernel about some of those pages via \fBmadvise\fR(2) or a similar system call\&. This provides the kernel with sufficient information to recycle dirty pages if physical memory becomes scarce and the pages remain unused\&. The default minimum ratio is 8:1 (2^3:1); an option value of \-1 will disable dirty page purging\&. See -"arenas\&.lg_dirty_mult" +arenas\&.lg_dirty_mult and -"arena\&.<i>\&.lg_dirty_mult" +arena\&.<i>\&.lg_dirty_mult for related dynamic control options\&. .RE .PP -"opt\&.decay_time" (\fBssize_t\fR) r\- +opt\&.decay_time (\fBssize_t\fR) r\- .RS 4 Approximate time in seconds from the creation of a set of unused dirty pages until an equivalent set of unused dirty pages is purged and/or reused\&. The pages are incrementally purged according to a sigmoidal decay curve that starts and ends with zero purge rate\&. A decay time of 0 causes all unused dirty pages to be purged immediately upon creation\&. A decay time of \-1 disables purging\&. The default decay time is 10 seconds\&. See -"arenas\&.decay_time" +arenas\&.decay_time and -"arena\&.<i>\&.decay_time" +arena\&.<i>\&.decay_time for related dynamic control options\&. .RE .PP -"opt\&.stats_print" (\fBbool\fR) r\- +opt\&.stats_print (\fBbool\fR) r\- .RS 4 Enable/disable statistics printing at exit\&. If enabled, the -\fBmalloc_stats_print\fR\fB\fR +malloc_stats_print() function is called at program exit via an \fBatexit\fR(3) function\&. If \fB\-\-enable\-stats\fR is specified during configuration, this has the potential to cause deadlock for a multi\-threaded process that exits while one or more threads are executing in the memory allocation functions\&. Furthermore, -\fBatexit\fR\fB\fR +atexit() may allocate memory during application initialization and then deadlock internally when jemalloc in turn calls -\fBatexit\fR\fB\fR, so this option is not universally usable (though the application can register its own -\fBatexit\fR\fB\fR +atexit(), so this option is not universally usable (though the application can register its own +atexit() function with equivalent functionality)\&. Therefore, this option should only be used with care; it is primarily intended as a performance tuning aid during application development\&. This option is disabled by default\&. .RE .PP -"opt\&.junk" (\fBconst char *\fR) r\- [\fB\-\-enable\-fill\fR] +opt\&.junk (\fBconst char *\fR) r\- [\fB\-\-enable\-fill\fR] .RS 4 -Junk filling\&. If set to "alloc", each byte of uninitialized allocated memory will be initialized to -0xa5\&. If set to "free", all deallocated memory will be initialized to -0x5a\&. If set to "true", both allocated and deallocated memory will be initialized, and if set to "false", junk filling be disabled entirely\&. This is intended for debugging and will impact performance negatively\&. This option is "false" by default unless +Junk filling\&. If set to +\(lqalloc\(rq, each byte of uninitialized allocated memory will be initialized to +0xa5\&. If set to +\(lqfree\(rq, all deallocated memory will be initialized to +0x5a\&. If set to +\(lqtrue\(rq, both allocated and deallocated memory will be initialized, and if set to +\(lqfalse\(rq, junk filling be disabled entirely\&. This is intended for debugging and will impact performance negatively\&. This option is +\(lqfalse\(rq +by default unless \fB\-\-enable\-debug\fR -is specified during configuration, in which case it is "true" by default unless running inside -\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2\&. +is specified during configuration, in which case it is +\(lqtrue\(rq +by default unless running inside +\m[blue]\fBValgrind\fR\m[]\&\s-2\u[3]\d\s+2\&. .RE .PP -"opt\&.quarantine" (\fBsize_t\fR) r\- [\fB\-\-enable\-fill\fR] +opt\&.quarantine (\fBsize_t\fR) r\- [\fB\-\-enable\-fill\fR] .RS 4 Per thread quarantine size in bytes\&. If non\-zero, each thread maintains a FIFO object quarantine that stores up to the specified number of bytes of memory\&. The quarantined memory is not freed until it is released from quarantine, though it is immediately junk\-filled if the -"opt\&.junk" +opt\&.junk option is enabled\&. This feature is of particular use in combination with -\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2, which can detect attempts to access quarantined objects\&. This is intended for debugging and will impact performance negatively\&. The default quarantine size is 0 unless running inside Valgrind, in which case the default is 16 MiB\&. +\m[blue]\fBValgrind\fR\m[]\&\s-2\u[3]\d\s+2, which can detect attempts to access quarantined objects\&. This is intended for debugging and will impact performance negatively\&. The default quarantine size is 0 unless running inside Valgrind, in which case the default is 16 MiB\&. .RE .PP -"opt\&.redzone" (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR] +opt\&.redzone (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR] .RS 4 Redzones enabled/disabled\&. If enabled, small allocations have redzones before and after them\&. Furthermore, if the -"opt\&.junk" +opt\&.junk option is enabled, the redzones are checked for corruption during deallocation\&. However, the primary intended purpose of this feature is to be used in combination with -\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2, which needs redzones in order to do effective buffer overflow/underflow detection\&. This option is intended for debugging and will impact performance negatively\&. This option is disabled by default unless running inside Valgrind\&. +\m[blue]\fBValgrind\fR\m[]\&\s-2\u[3]\d\s+2, which needs redzones in order to do effective buffer overflow/underflow detection\&. This option is intended for debugging and will impact performance negatively\&. This option is disabled by default unless running inside Valgrind\&. .RE .PP -"opt\&.zero" (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR] +opt\&.zero (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR] .RS 4 Zero filling enabled/disabled\&. If enabled, each byte of uninitialized allocated memory will be initialized to 0\&. Note that this initialization only happens once for each byte, so -\fBrealloc\fR\fB\fR +realloc() and -\fBrallocx\fR\fB\fR +rallocx() calls do not zero memory that was previously allocated\&. This is intended for debugging and will impact performance negatively\&. This option is disabled by default\&. .RE .PP -"opt\&.utrace" (\fBbool\fR) r\- [\fB\-\-enable\-utrace\fR] +opt\&.utrace (\fBbool\fR) r\- [\fB\-\-enable\-utrace\fR] .RS 4 Allocation tracing based on \fButrace\fR(2) enabled/disabled\&. This option is disabled by default\&. .RE .PP -"opt\&.xmalloc" (\fBbool\fR) r\- [\fB\-\-enable\-xmalloc\fR] +opt\&.xmalloc (\fBbool\fR) r\- [\fB\-\-enable\-xmalloc\fR] .RS 4 Abort\-on\-out\-of\-memory enabled/disabled\&. If enabled, rather than returning failure for any allocation function, display a diagnostic message on \fBSTDERR_FILENO\fR @@ -921,94 +957,94 @@ malloc_conf = "xmalloc:true"; This option is disabled by default\&. .RE .PP -"opt\&.tcache" (\fBbool\fR) r\- [\fB\-\-enable\-tcache\fR] +opt\&.tcache (\fBbool\fR) r\- [\fB\-\-enable\-tcache\fR] .RS 4 Thread\-specific caching (tcache) enabled/disabled\&. When there are multiple threads, each thread uses a tcache for objects up to a certain size\&. Thread\-specific caching allows many allocations to be satisfied without performing any thread synchronization, at the cost of increased memory use\&. See the -"opt\&.lg_tcache_max" +opt\&.lg_tcache_max option for related tuning information\&. This option is enabled by default unless running inside -\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2, in which case it is forcefully disabled\&. +\m[blue]\fBValgrind\fR\m[]\&\s-2\u[3]\d\s+2, in which case it is forcefully disabled\&. .RE .PP -"opt\&.lg_tcache_max" (\fBsize_t\fR) r\- [\fB\-\-enable\-tcache\fR] +opt\&.lg_tcache_max (\fBsize_t\fR) r\- [\fB\-\-enable\-tcache\fR] .RS 4 Maximum size class (log base 2) to cache in the thread\-specific cache (tcache)\&. At a minimum, all small size classes are cached, and at a maximum all large size classes are cached\&. The default maximum is 32 KiB (2^15)\&. .RE .PP -"opt\&.prof" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR] +opt\&.prof (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR] .RS 4 Memory profiling enabled/disabled\&. If enabled, profile memory allocation activity\&. See the -"opt\&.prof_active" +opt\&.prof_active option for on\-the\-fly activation/deactivation\&. See the -"opt\&.lg_prof_sample" +opt\&.lg_prof_sample option for probabilistic sampling control\&. See the -"opt\&.prof_accum" +opt\&.prof_accum option for control of cumulative sample reporting\&. See the -"opt\&.lg_prof_interval" +opt\&.lg_prof_interval option for information on interval\-triggered profile dumping, the -"opt\&.prof_gdump" +opt\&.prof_gdump option for information on high\-water\-triggered profile dumping, and the -"opt\&.prof_final" +opt\&.prof_final option for final profile dumping\&. Profile output is compatible with the \fBjeprof\fR command, which is based on the \fBpprof\fR that is developed as part of the -\m[blue]\fBgperftools package\fR\m[]\&\s-2\u[3]\d\s+2\&. See +\m[blue]\fBgperftools package\fR\m[]\&\s-2\u[4]\d\s+2\&. See *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201611091842.uA9IgUet012596>