From owner-svn-src-all@FreeBSD.ORG Thu Jan 23 02:47:39 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 904B6E46; Thu, 23 Jan 2014 02:47:39 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 78CB6111D; Thu, 23 Jan 2014 02:47:39 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id s0N2ld8J098562; Thu, 23 Jan 2014 02:47:39 GMT (envelope-from jasone@svn.freebsd.org) Received: (from jasone@localhost) by svn.freebsd.org (8.14.7/8.14.7/Submit) id s0N2lbkU098546; Thu, 23 Jan 2014 02:47:37 GMT (envelope-from jasone@svn.freebsd.org) Message-Id: <201401230247.s0N2lbkU098546@svn.freebsd.org> From: Jason Evans Date: Thu, 23 Jan 2014 02:47:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r261071 - in head: contrib/jemalloc contrib/jemalloc/doc contrib/jemalloc/include/jemalloc contrib/jemalloc/include/jemalloc/internal contrib/jemalloc/src include lib/libc/gen lib/libc/... X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 02:47:39 -0000 Author: jasone Date: Thu Jan 23 02:47:36 2014 New Revision: 261071 URL: http://svnweb.freebsd.org/changeset/base/261071 Log: Update jemalloc to version 3.5.0. Added: head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_defs.h (contents, props changed) head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_macros.h (contents, props changed) head/contrib/jemalloc/include/jemalloc/internal/public_namespace.h (contents, props changed) Deleted: head/contrib/jemalloc/include/jemalloc/jemalloc_defs.h Modified: head/contrib/jemalloc/COPYING head/contrib/jemalloc/ChangeLog head/contrib/jemalloc/FREEBSD-Xlist head/contrib/jemalloc/FREEBSD-diffs head/contrib/jemalloc/FREEBSD-upgrade head/contrib/jemalloc/VERSION head/contrib/jemalloc/doc/jemalloc.3 head/contrib/jemalloc/include/jemalloc/internal/arena.h head/contrib/jemalloc/include/jemalloc/internal/chunk_dss.h head/contrib/jemalloc/include/jemalloc/internal/ckh.h head/contrib/jemalloc/include/jemalloc/internal/hash.h head/contrib/jemalloc/include/jemalloc/internal/huge.h head/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal.h head/contrib/jemalloc/include/jemalloc/internal/private_namespace.h head/contrib/jemalloc/include/jemalloc/internal/prng.h head/contrib/jemalloc/include/jemalloc/internal/prof.h head/contrib/jemalloc/include/jemalloc/internal/ql.h head/contrib/jemalloc/include/jemalloc/internal/qr.h head/contrib/jemalloc/include/jemalloc/internal/rb.h head/contrib/jemalloc/include/jemalloc/internal/rtree.h head/contrib/jemalloc/include/jemalloc/internal/tcache.h head/contrib/jemalloc/include/jemalloc/internal/tsd.h head/contrib/jemalloc/include/jemalloc/internal/util.h head/contrib/jemalloc/include/jemalloc/jemalloc.h head/contrib/jemalloc/include/jemalloc/jemalloc_FreeBSD.h head/contrib/jemalloc/src/arena.c head/contrib/jemalloc/src/bitmap.c head/contrib/jemalloc/src/chunk.c head/contrib/jemalloc/src/chunk_dss.c head/contrib/jemalloc/src/chunk_mmap.c head/contrib/jemalloc/src/ckh.c head/contrib/jemalloc/src/ctl.c head/contrib/jemalloc/src/huge.c head/contrib/jemalloc/src/jemalloc.c head/contrib/jemalloc/src/mutex.c head/contrib/jemalloc/src/prof.c head/contrib/jemalloc/src/quarantine.c head/contrib/jemalloc/src/rtree.c head/contrib/jemalloc/src/stats.c head/contrib/jemalloc/src/tcache.c head/contrib/jemalloc/src/tsd.c head/contrib/jemalloc/src/util.c head/include/malloc_np.h head/lib/libc/gen/tls.c head/lib/libc/stdlib/jemalloc/Makefile.inc head/lib/libc/stdlib/jemalloc/Symbol.map Modified: head/contrib/jemalloc/COPYING ============================================================================== --- head/contrib/jemalloc/COPYING Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/COPYING Thu Jan 23 02:47:36 2014 (r261071) @@ -1,10 +1,10 @@ Unless otherwise specified, files in the jemalloc source distribution are subject to the following license: -------------------------------------------------------------------------------- -Copyright (C) 2002-2013 Jason Evans . +Copyright (C) 2002-2014 Jason Evans . All rights reserved. Copyright (C) 2007-2012 Mozilla Foundation. All rights reserved. -Copyright (C) 2009-2013 Facebook, Inc. All rights reserved. +Copyright (C) 2009-2014 Facebook, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Modified: head/contrib/jemalloc/ChangeLog ============================================================================== --- head/contrib/jemalloc/ChangeLog Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/ChangeLog Thu Jan 23 02:47:36 2014 (r261071) @@ -6,6 +6,59 @@ found in the git revision history: http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git git://canonware.com/jemalloc.git +* 3.5.0 (January 22, 2014) + + This version focuses on refactoring and automated testing, though it also + includes some non-trivial heap profiling optimizations not mentioned below. + + New features: + - Add the *allocx() API, which is a successor to the experimental *allocm() + API. The *allocx() functions are slightly simpler to use because they have + fewer parameters, they directly return the results of primary interest, and + mallocx()/rallocx() avoid the strict aliasing pitfall that + allocm()/rallocx() share with posix_memalign(). Note that *allocm() is + slated for removal in the next non-bugfix release. + - Add support for LinuxThreads. + + Bug fixes: + - Unless heap profiling is enabled, disable floating point code and don't link + with libm. This, in combination with e.g. EXTRA_CFLAGS=-mno-sse on x64 + systems, makes it possible to completely disable floating point register + use. Some versions of glibc neglect to save/restore caller-saved floating + point registers during dynamic lazy symbol loading, and the symbol loading + code uses whatever malloc the application happens to have linked/loaded + with, the result being potential floating point register corruption. + - Report ENOMEM rather than EINVAL if an OOM occurs during heap profiling + backtrace creation in imemalign(). This bug impacted posix_memalign() and + aligned_alloc(). + - Fix a file descriptor leak in a prof_dump_maps() error path. + - Fix prof_dump() to close the dump file descriptor for all relevant error + paths. + - Fix rallocm() to use the arena specified by the ALLOCM_ARENA(s) flag for + allocation, not just deallocation. + - Fix a data race for large allocation stats counters. + - Fix a potential infinite loop during thread exit. This bug occurred on + Solaris, and could affect other platforms with similar pthreads TSD + implementations. + - Don't junk-fill reallocations unless usable size changes. This fixes a + violation of the *allocx()/*allocm() semantics. + - Fix growing large reallocation to junk fill new space. + - Fix huge deallocation to junk fill when munmap is disabled. + - Change the default private namespace prefix from empty to je_, and change + --with-private-namespace-prefix so that it prepends an additional prefix + rather than replacing je_. This reduces the likelihood of applications + which statically link jemalloc experiencing symbol name collisions. + - Add missing private namespace mangling (relevant when + --with-private-namespace is specified). + - Add and use JEMALLOC_INLINE_C so that static inline functions are marked as + static even for debug builds. + - Add a missing mutex unlock in a malloc_init_hard() error path. In practice + this error path is never executed. + - Fix numerous bugs in malloc_strotumax() error handling/reporting. These + bugs had no impact except for malformed inputs. + - Fix numerous bugs in malloc_snprintf(). These bugs were not exercised by + existing calls, so they had no impact. + * 3.4.1 (October 20, 2013) Bug fixes: Modified: head/contrib/jemalloc/FREEBSD-Xlist ============================================================================== --- head/contrib/jemalloc/FREEBSD-Xlist Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/FREEBSD-Xlist Thu Jan 23 02:47:36 2014 (r261071) @@ -10,14 +10,35 @@ autom4te.cache/ bin/ config.* configure* +coverage.sh doc/*.in doc/*.xml doc/*.xsl doc/*.html include/jemalloc/internal/jemalloc_internal.h.in +include/jemalloc/internal/jemalloc_internal_defs.h.in +include/jemalloc/internal/private_namespace.sh +include/jemalloc/internal/private_symbols.txt +include/jemalloc/internal/private_unnamespace.h +include/jemalloc/internal/private_unnamespace.sh +include/jemalloc/internal/public_namespace.sh +include/jemalloc/internal/public_symbols.txt +include/jemalloc/internal/public_unnamespace.h +include/jemalloc/internal/public_unnamespace.sh include/jemalloc/internal/size_classes.sh include/jemalloc/jemalloc.h.in +include/jemalloc/jemalloc.sh include/jemalloc/jemalloc_defs.h.in +include/jemalloc/jemalloc_macros.h +include/jemalloc/jemalloc_macros.h.in +include/jemalloc/jemalloc_mangle_jet.h +include/jemalloc/jemalloc_mangle.sh +include/jemalloc/jemalloc_mangle.h +include/jemalloc/jemalloc_protos_jet.h +include/jemalloc/jemalloc_protos.h +include/jemalloc/jemalloc_protos.h.in +include/jemalloc/jemalloc_rename.h +include/jemalloc/jemalloc_rename.sh include/msvc_compat/ install-sh src/zone.c Modified: head/contrib/jemalloc/FREEBSD-diffs ============================================================================== --- head/contrib/jemalloc/FREEBSD-diffs Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/FREEBSD-diffs Thu Jan 23 02:47:36 2014 (r261071) @@ -1,8 +1,8 @@ diff --git a/doc/jemalloc.xml.in b/doc/jemalloc.xml.in -index abd5e6f..1d7491a 100644 +index c7e2e87..2bd59f0 100644 --- a/doc/jemalloc.xml.in +++ b/doc/jemalloc.xml.in -@@ -51,12 +51,23 @@ +@@ -57,12 +57,23 @@ This manual describes jemalloc @jemalloc_version@. More information can be found at the jemalloc website. @@ -27,7 +27,7 @@ index abd5e6f..1d7491a 100644 Standard API -@@ -2180,4 +2191,16 @@ malloc_conf = "lg_chunk:24";]]> +@@ -2338,4 +2349,19 @@ malloc_conf = "lg_chunk:24";]]> The posix_memalign function conforms to IEEE Std 1003.1-2001 (“POSIX.1”). @@ -42,32 +42,35 @@ index abd5e6f..1d7491a 100644 + mallctl*, and + *allocm functions first appeared in + FreeBSD 10.0. ++ ++ The *allocx functions first appeared ++ in FreeBSD 11.0. + diff --git a/include/jemalloc/internal/jemalloc_internal.h.in b/include/jemalloc/internal/jemalloc_internal.h.in -index 53c135c..c547339 100644 +index d24a1fe..d101c3d 100644 --- a/include/jemalloc/internal/jemalloc_internal.h.in +++ b/include/jemalloc/internal/jemalloc_internal.h.in @@ -1,5 +1,8 @@ #ifndef JEMALLOC_INTERNAL_H - #define JEMALLOC_INTERNAL_H + #define JEMALLOC_INTERNAL_H +#include "libc_private.h" +#include "namespace.h" + #include #ifdef _WIN32 # include -@@ -54,6 +57,9 @@ typedef intptr_t ssize_t; +@@ -65,6 +68,9 @@ typedef intptr_t ssize_t; + #include #endif - #include +#include "un-namespace.h" +#include "libc_private.h" + #define JEMALLOC_NO_DEMANGLE - #include "../jemalloc@install_suffix@.h" - -@@ -95,13 +101,7 @@ static const bool config_fill = + #ifdef JEMALLOC_JET + # define JEMALLOC_N(n) jet_##n +@@ -99,13 +105,7 @@ static const bool config_fill = false #endif ; @@ -96,36 +99,24 @@ index de44e14..564d604 100644 #endif bool malloc_mutex_init(malloc_mutex_t *mutex); -diff --git a/include/jemalloc/internal/private_namespace.h b/include/jemalloc/internal/private_namespace.h -index cdb0b0e..2a98d1f 100644 ---- a/include/jemalloc/internal/private_namespace.h -+++ b/include/jemalloc/internal/private_namespace.h -@@ -218,7 +218,6 @@ - #define iralloc JEMALLOC_N(iralloc) - #define irallocx JEMALLOC_N(irallocx) - #define isalloc JEMALLOC_N(isalloc) --#define isthreaded JEMALLOC_N(isthreaded) - #define ivsalloc JEMALLOC_N(ivsalloc) - #define jemalloc_postfork_child JEMALLOC_N(jemalloc_postfork_child) - #define jemalloc_postfork_parent JEMALLOC_N(jemalloc_postfork_parent) -diff --git a/include/jemalloc/jemalloc.h.in b/include/jemalloc/jemalloc.h.in -index 31b1304..c3ef2f5 100644 ---- a/include/jemalloc/jemalloc.h.in -+++ b/include/jemalloc/jemalloc.h.in -@@ -15,6 +15,7 @@ extern "C" { - #define JEMALLOC_VERSION_GID "@jemalloc_version_gid@" - - #include "jemalloc_defs@install_suffix@.h" -+#include "jemalloc_FreeBSD.h" - - #ifdef JEMALLOC_EXPERIMENTAL - #define ALLOCM_LG_ALIGN(la) (la) +diff --git a/include/jemalloc/internal/private_symbols.txt b/include/jemalloc/internal/private_symbols.txt +index 1e64ed5..29ddba3 100644 +--- a/include/jemalloc/internal/private_symbols.txt ++++ b/include/jemalloc/internal/private_symbols.txt +@@ -225,7 +225,6 @@ iralloc + iralloct + iralloct_realign + isalloc +-isthreaded + ivsalloc + ixalloc + jemalloc_postfork_child diff --git a/include/jemalloc/jemalloc_FreeBSD.h b/include/jemalloc/jemalloc_FreeBSD.h new file mode 100644 -index 0000000..e6c8407 +index 0000000..94554bc --- /dev/null +++ b/include/jemalloc/jemalloc_FreeBSD.h -@@ -0,0 +1,117 @@ +@@ -0,0 +1,134 @@ +/* + * Override settings that were generated in jemalloc_defs.h as necessary. + */ @@ -202,6 +193,12 @@ index 0000000..e6c8407 +#undef je_free +#undef je_posix_memalign +#undef je_malloc_usable_size ++#undef je_mallocx ++#undef je_rallocx ++#undef je_xallocx ++#undef je_sallocx ++#undef je_dallocx ++#undef je_nallocx +#undef je_allocm +#undef je_rallocm +#undef je_sallocm @@ -213,6 +210,12 @@ index 0000000..e6c8407 +#define je_free __free +#define je_posix_memalign __posix_memalign +#define je_malloc_usable_size __malloc_usable_size ++#define je_mallocx __mallocx ++#define je_rallocx __rallocx ++#define je_xallocx __xallocx ++#define je_sallocx __sallocx ++#define je_dallocx __dallocx ++#define je_nallocx __nallocx +#define je_allocm __allocm +#define je_rallocm __rallocm +#define je_sallocm __sallocm @@ -236,15 +239,31 @@ index 0000000..e6c8407 +__weak_reference(__free, free); +__weak_reference(__posix_memalign, posix_memalign); +__weak_reference(__malloc_usable_size, malloc_usable_size); ++__weak_reference(__mallocx, mallocx); ++__weak_reference(__rallocx, rallocx); ++__weak_reference(__xallocx, xallocx); ++__weak_reference(__sallocx, sallocx); ++__weak_reference(__dallocx, dallocx); ++__weak_reference(__nallocx, nallocx); +__weak_reference(__allocm, allocm); +__weak_reference(__rallocm, rallocm); +__weak_reference(__sallocm, sallocm); +__weak_reference(__dallocm, dallocm); +__weak_reference(__nallocm, nallocm); +#endif +diff --git a/include/jemalloc/jemalloc_rename.sh b/include/jemalloc/jemalloc_rename.sh +index f943891..47d032c 100755 +--- a/include/jemalloc/jemalloc_rename.sh ++++ b/include/jemalloc/jemalloc_rename.sh +@@ -19,4 +19,6 @@ done + + cat < -.\" Date: 10/20/2013 +.\" Date: 01/22/2014 .\" Manual: User Manual -.\" Source: jemalloc 3.4.1-0-g0135fb806e4137dc9cdf152541926a2bc95e33f0 +.\" Source: jemalloc 3.5.0-0-gcc47dde16203a6ae7eb685b53e1ae501f3869bc6 .\" Language: English .\" -.TH "JEMALLOC" "3" "10/20/2013" "jemalloc 3.4.1-0-g0135fb806e41" "User Manual" +.TH "JEMALLOC" "3" "01/22/2014" "jemalloc 3.5.0-0-gcc47dde16203" "User Manual" .\" ----------------------------------------------------------------- .\" * Define some portability stuff .\" ----------------------------------------------------------------- @@ -31,7 +31,7 @@ jemalloc \- general purpose memory allocation functions .SH "LIBRARY" .PP -This manual describes jemalloc 3\&.4\&.1\-0\-g0135fb806e4137dc9cdf152541926a2bc95e33f0\&. More information can be found at the +This manual describes jemalloc 3\&.5\&.0\-0\-gcc47dde16203a6ae7eb685b53e1ae501f3869bc6\&. More information can be found at the \m[blue]\fBjemalloc website\fR\m[]\&\s-2\u[1]\d\s+2\&. .PP The following configuration options are enabled in libc\*(Aqs built\-in jemalloc: @@ -71,16 +71,28 @@ make variable)\&. .HP \w'void\ free('u .BI "void free(void\ *" "ptr" ");" .SS "Non\-standard API" -.HP \w'size_t\ malloc_usable_size('u -.BI "size_t malloc_usable_size(const\ void\ *" "ptr" ");" -.HP \w'void\ malloc_stats_print('u -.BI "void malloc_stats_print(void\ " "(*write_cb)" "\ (void\ *,\ const\ char\ *), void\ *" "cbopaque" ", const\ char\ *" "opts" ");" +.HP \w'void\ *mallocx('u +.BI "void *mallocx(size_t\ " "size" ", int\ " "flags" ");" +.HP \w'void\ *rallocx('u +.BI "void *rallocx(void\ *" "ptr" ", size_t\ " "size" ", int\ " "flags" ");" +.HP \w'size_t\ xallocx('u +.BI "size_t xallocx(void\ *" "ptr" ", size_t\ " "size" ", size_t\ " "extra" ", int\ " "flags" ");" +.HP \w'size_t\ sallocx('u +.BI "size_t sallocx(void\ *" "ptr" ", int\ " "flags" ");" +.HP \w'void\ dallocx('u +.BI "void dallocx(void\ *" "ptr" ", int\ " "flags" ");" +.HP \w'size_t\ nallocx('u +.BI "size_t nallocx(size_t\ " "size" ", int\ " "flags" ");" .HP \w'int\ mallctl('u .BI "int mallctl(const\ char\ *" "name" ", void\ *" "oldp" ", size_t\ *" "oldlenp" ", void\ *" "newp" ", size_t\ " "newlen" ");" .HP \w'int\ mallctlnametomib('u .BI "int mallctlnametomib(const\ char\ *" "name" ", size_t\ *" "mibp" ", size_t\ *" "miblenp" ");" .HP \w'int\ mallctlbymib('u .BI "int mallctlbymib(const\ size_t\ *" "mib" ", size_t\ " "miblen" ", void\ *" "oldp" ", size_t\ *" "oldlenp" ", void\ *" "newp" ", size_t\ " "newlen" ");" +.HP \w'void\ malloc_stats_print('u +.BI "void malloc_stats_print(void\ " "(*write_cb)" "\ (void\ *,\ const\ char\ *), void\ *" "cbopaque" ", const\ char\ *" "opts" ");" +.HP \w'size_t\ malloc_usable_size('u +.BI "size_t malloc_usable_size(const\ void\ *" "ptr" ");" .HP \w'void\ (*malloc_message)('u .BI "void (*malloc_message)(void\ *" "cbopaque" ", const\ char\ *" "s" ");" .PP @@ -172,36 +184,105 @@ is .SS "Non\-standard API" .PP The -\fBmalloc_usable_size\fR\fB\fR -function returns the usable size of the allocation pointed to by -\fIptr\fR\&. The return value may be larger than the size that was requested during allocation\&. The -\fBmalloc_usable_size\fR\fB\fR -function is not a mechanism for in\-place -\fBrealloc\fR\fB\fR; rather it is provided solely as a tool for introspection purposes\&. Any discrepancy between the requested allocation size and the size reported by -\fBmalloc_usable_size\fR\fB\fR -should not be depended on, since such behavior is entirely implementation\-dependent\&. +\fBmallocx\fR\fB\fR, +\fBrallocx\fR\fB\fR, +\fBxallocx\fR\fB\fR, +\fBsallocx\fR\fB\fR, +\fBdallocx\fR\fB\fR, and +\fBnallocx\fR\fB\fR +functions all have a +\fIflags\fR +argument that can be used to specify options\&. The functions only check the options that are contextually relevant\&. Use bitwise or (|) operations to specify one or more of the following: +.PP +\fBMALLOCX_LG_ALIGN(\fR\fB\fIla\fR\fR\fB) \fR +.RS 4 +Align the memory allocation to start at an address that is a multiple of +(1 << \fIla\fR)\&. This macro does not validate that +\fIla\fR +is within the valid range\&. +.RE +.PP +\fBMALLOCX_ALIGN(\fR\fB\fIa\fR\fR\fB) \fR +.RS 4 +Align the memory allocation to start at an address that is a multiple of +\fIa\fR, where +\fIa\fR +is a power of two\&. This macro does not validate that +\fIa\fR +is a power of 2\&. +.RE +.PP +\fBMALLOCX_ZERO\fR +.RS 4 +Initialize newly allocated memory to contain zero bytes\&. In the growing reallocation case, the real size prior to reallocation defines the boundary between untouched bytes and those that are initialized to contain zero bytes\&. If this macro is absent, newly allocated memory is uninitialized\&. +.RE +.PP +\fBMALLOCX_ARENA(\fR\fB\fIa\fR\fR\fB) \fR +.RS 4 +Use the arena specified by the index +\fIa\fR +(and by necessity bypass the thread cache)\&. This macro has no effect for huge regions, nor for regions that were allocated via an arena other than the one specified\&. This macro does not validate that +\fIa\fR +specifies an arena index in the valid range\&. +.RE .PP The -\fBmalloc_stats_print\fR\fB\fR -function writes human\-readable summary statistics via the -\fIwrite_cb\fR -callback function pointer and -\fIcbopaque\fR -data passed to -\fIwrite_cb\fR, or -\fBmalloc_message\fR\fB\fR -if -\fIwrite_cb\fR +\fBmallocx\fR\fB\fR +function allocates at least +\fIsize\fR +bytes of memory, and returns a pointer to the base address of the allocation\&. Behavior is undefined if +\fIsize\fR is -\fBNULL\fR\&. This function can be called repeatedly\&. General information that never changes during execution can be omitted by specifying "g" as a character within the -\fIopts\fR -string\&. Note that -\fBmalloc_message\fR\fB\fR -uses the -\fBmallctl*\fR\fB\fR -functions internally, so inconsistent statistics can be reported if multiple threads use these functions simultaneously\&. If -\fB\-\-enable\-stats\fR -is specified during configuration, \(lqm\(rq and \(lqa\(rq can be specified to omit merged arena and per arena statistics, respectively; \(lqb\(rq and \(lql\(rq can be specified to omit per size class statistics for bins and large objects, respectively\&. Unrecognized characters are silently ignored\&. Note that thread caching may prevent some statistics from being completely up to date, since extra locking would be required to merge counters that track thread cache operations\&. +\fB0\fR, or if request size overflows due to size class and/or alignment constraints\&. +.PP +The +\fBrallocx\fR\fB\fR +function resizes the allocation at +\fIptr\fR +to be at least +\fIsize\fR +bytes, and returns a pointer to the base address of the resulting allocation, which may or may not have moved from its original location\&. Behavior is undefined if +\fIsize\fR +is +\fB0\fR, or if request size overflows due to size class and/or alignment constraints\&. +.PP +The +\fBxallocx\fR\fB\fR +function resizes the allocation at +\fIptr\fR +in place to be at least +\fIsize\fR +bytes, and returns the real size of the allocation\&. If +\fIextra\fR +is non\-zero, an attempt is made to resize the allocation to be at least +(\fIsize\fR + \fIextra\fR) +bytes, though inability to allocate the extra byte(s) will not by itself result in failure to resize\&. Behavior is undefined if +\fIsize\fR +is +\fB0\fR, or if +(\fIsize\fR + \fIextra\fR > \fBSIZE_T_MAX\fR)\&. +.PP +The +\fBsallocx\fR\fB\fR +function returns the real size of the allocation at +\fIptr\fR\&. +.PP +The +\fBdallocx\fR\fB\fR +function causes the memory referenced by +\fIptr\fR +to be made available for future allocations\&. +.PP +The +\fBnallocx\fR\fB\fR +function allocates no memory, but it performs the same size computation as the +\fBmallocx\fR\fB\fR +function, and returns the real size of the allocation that would result from the equivalent +\fBmallocx\fR\fB\fR +function call\&. Behavior is undefined if +\fIsize\fR +is +\fB0\fR, or if request size overflows due to size class and/or alignment constraints\&. .PP The \fBmallctl\fR\fB\fR @@ -245,15 +326,14 @@ that is smaller than the number of perio .\} .nf unsigned nbins, i; - -int mib[4]; +size_t mib[4]; size_t len, miblen; len = sizeof(nbins); mallctl("arenas\&.nbins", &nbins, &len, NULL, 0); miblen = 4; -mallnametomib("arenas\&.bin\&.0\&.size", mib, &miblen); +mallctlnametomib("arenas\&.bin\&.0\&.size", mib, &miblen); for (i = 0; i < nbins; i++) { size_t bin_size; @@ -266,6 +346,38 @@ for (i = 0; i < nbins; i++) { .if n \{\ .RE .\} +.PP +The +\fBmalloc_stats_print\fR\fB\fR +function writes human\-readable summary statistics via the +\fIwrite_cb\fR +callback function pointer and +\fIcbopaque\fR +data passed to +\fIwrite_cb\fR, or +\fBmalloc_message\fR\fB\fR +if +\fIwrite_cb\fR +is +\fBNULL\fR\&. This function can be called repeatedly\&. General information that never changes during execution can be omitted by specifying "g" as a character within the +\fIopts\fR +string\&. Note that +\fBmalloc_message\fR\fB\fR +uses the +\fBmallctl*\fR\fB\fR +functions internally, so inconsistent statistics can be reported if multiple threads use these functions simultaneously\&. If +\fB\-\-enable\-stats\fR +is specified during configuration, \(lqm\(rq and \(lqa\(rq can be specified to omit merged arena and per arena statistics, respectively; \(lqb\(rq and \(lql\(rq can be specified to omit per size class statistics for bins and large objects, respectively\&. Unrecognized characters are silently ignored\&. Note that thread caching may prevent some statistics from being completely up to date, since extra locking would be required to merge counters that track thread cache operations\&. +.PP +The +\fBmalloc_usable_size\fR\fB\fR +function returns the usable size of the allocation pointed to by +\fIptr\fR\&. The return value may be larger than the size that was requested during allocation\&. The +\fBmalloc_usable_size\fR\fB\fR +function is not a mechanism for in\-place +\fBrealloc\fR\fB\fR; rather it is provided solely as a tool for introspection purposes\&. Any discrepancy between the requested allocation size and the size reported by +\fBmalloc_usable_size\fR\fB\fR +should not be depended on, since such behavior is entirely implementation\-dependent\&. .SS "Experimental API" .PP The experimental API is subject to change or removal without regard for backward compatibility\&. If @@ -302,7 +414,7 @@ is a power of 2\&. .PP \fBALLOCM_ZERO\fR .RS 4 -Initialize newly allocated memory to contain zero bytes\&. In the growing reallocation case, the real size prior to reallocation defines the boundary between untouched bytes and those that are initialized to contain zero bytes\&. If this option is absent, newly allocated memory is uninitialized\&. +Initialize newly allocated memory to contain zero bytes\&. In the growing reallocation case, the real size prior to reallocation defines the boundary between untouched bytes and those that are initialized to contain zero bytes\&. If this macro is absent, newly allocated memory is uninitialized\&. .RE .PP \fBALLOCM_NO_MOVE\fR @@ -313,9 +425,10 @@ For reallocation, fail rather than movin \fBALLOCM_ARENA(\fR\fB\fIa\fR\fR\fB) \fR .RS 4 Use the arena specified by the index -\fIa\fR\&. This macro does not validate that \fIa\fR -specifies an arena in the valid range\&. +(and by necessity bypass the thread cache)\&. This macro has no effect for huge regions, nor for regions that were allocated via an arena other than the one specified\&. This macro does not validate that +\fIa\fR +specifies an arena index in the valid range\&. .RE .PP The @@ -332,7 +445,7 @@ is not \fBNULL\fR\&. Behavior is undefined if \fIsize\fR is -\fB0\fR\&. +\fB0\fR, or if request size overflows due to size class and/or alignment constraints\&. .PP The \fBrallocm\fR\fB\fR @@ -350,11 +463,11 @@ is not \fBNULL\fR\&. If \fIextra\fR is non\-zero, an attempt is made to resize the allocation to be at least -\fIsize\fR + \fIextra\fR) +(\fIsize\fR + \fIextra\fR) bytes, though inability to allocate the extra byte(s) will not by itself result in failure\&. Behavior is undefined if \fIsize\fR is -\fB0\fR, or if +\fB0\fR, if request size overflows due to size class and/or alignment constraints, or if (\fIsize\fR + \fIextra\fR > \fBSIZE_T_MAX\fR)\&. .PP The @@ -384,7 +497,7 @@ to the real size of the allocation that function call\&. Behavior is undefined if \fIsize\fR is -\fB0\fR\&. +\fB0\fR, or if request size overflows due to size class and/or alignment constraints\&. .SH "TUNING" .PP Once, when the first call is made to one of the memory allocation routines, the allocator initializes its internals based in part on various options that can be specified at compile\- or run\-time\&. @@ -662,16 +775,18 @@ in these cases\&. This option is disable is specified during configuration, in which case it is enabled by default\&. .RE .PP -"opt\&.lg_chunk" (\fBsize_t\fR) r\- -.RS 4 -Virtual memory chunk size (log base 2)\&. If a chunk size outside the supported size range is specified, the size is silently clipped to the minimum/maximum supported size\&. The default chunk size is 4 MiB (2^22)\&. -.RE -.PP "opt\&.dss" (\fBconst char *\fR) r\- .RS 4 dss (\fBsbrk\fR(2)) allocation precedence as related to \fBmmap\fR(2) -allocation\&. The following settings are supported: \(lqdisabled\(rq, \(lqprimary\(rq, and \(lqsecondary\(rq (default)\&. +allocation\&. The following settings are supported: \(lqdisabled\(rq, \(lqprimary\(rq, and \(lqsecondary\(rq\&. The default is \(lqsecondary\(rq if +"config\&.dss" +is true, \(lqdisabled\(rq otherwise\&. +.RE +.PP +"opt\&.lg_chunk" (\fBsize_t\fR) r\- +.RS 4 +Virtual memory chunk size (log base 2)\&. If a chunk size outside the supported size range is specified, the size is silently clipped to the minimum/maximum supported size\&. The default chunk size is 4 MiB (2^22)\&. .RE .PP "opt\&.narenas" (\fBsize_t\fR) r\- @@ -726,7 +841,8 @@ option is enabled, the redzones are chec "opt\&.zero" (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR] .RS 4 Zero filling enabled/disabled\&. If enabled, each byte of uninitialized allocated memory will be initialized to 0\&. Note that this initialization only happens once for each byte, so -\fBrealloc\fR\fB\fR +\fBrealloc\fR\fB\fR, +\fBrallocx\fR\fB\fR and \fBrallocm\fR\fB\fR calls do not zero memory that was previously allocated\&. This is intended for debugging and will impact performance negatively\&. This option is disabled by default\&. @@ -804,7 +920,7 @@ Filename prefix for profile dumps\&. If jeprof\&. .RE .PP -"opt\&.prof_active" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR] +"opt\&.prof_active" (\fBbool\fR) rw [\fB\-\-enable\-prof\fR] .RS 4 Profiling activated/deactivated\&. This is a secondary control mechanism that makes it possible to start the application with profiling enabled (see the "opt\&.prof" @@ -1119,7 +1235,7 @@ Number of or similar calls made to purge dirty pages\&. .RE .PP -"stats\&.arenas\&.\&.npurged" (\fBuint64_t\fR) r\- [\fB\-\-enable\-stats\fR] +"stats\&.arenas\&.\&.purged" (\fBuint64_t\fR) r\- [\fB\-\-enable\-stats\fR] .RS 4 Number of pages purged\&. .RE @@ -1342,11 +1458,32 @@ function returns no value\&. .SS "Non\-standard API" .PP The -\fBmalloc_usable_size\fR\fB\fR -function returns the usable size of the allocation pointed to by +\fBmallocx\fR\fB\fR +and +\fBrallocx\fR\fB\fR +functions return a pointer to the allocated memory if successful; otherwise a +\fBNULL\fR +pointer is returned to indicate insufficient contiguous memory was available to service the allocation request\&. +.PP +The +\fBxallocx\fR\fB\fR +function returns the real size of the resulting resized allocation pointed to by +\fIptr\fR, which is a value less than +\fIsize\fR +if the allocation could not be adequately grown in place\&. +.PP +The +\fBsallocx\fR\fB\fR +function returns the real size of the allocation pointed to by \fIptr\fR\&. .PP The +\fBnallocx\fR\fB\fR +returns the real size that would result from a successful equivalent +\fBmallocx\fR\fB\fR +function call, or zero if insufficient memory is available to perform the size computation\&. +.PP +The \fBmallctl\fR\fB\fR, \fBmallctlnametomib\fR\fB\fR, and \fBmallctlbymib\fR\fB\fR @@ -1363,12 +1500,6 @@ is too large or too small\&. Alternative is too large or too small; in this case as much data as possible are read despite the error\&. .RE .PP -ENOMEM -.RS 4 -\fI*oldlenp\fR -is too short to hold the requested value\&. -.RE -.PP ENOENT .RS 4 \fIname\fR @@ -1393,6 +1524,11 @@ An interface with side effects failed in \fBmallctl*\fR\fB\fR read/write processing\&. .RE +.PP +The +\fBmalloc_usable_size\fR\fB\fR +function returns the usable size of the allocation pointed to by +\fIptr\fR\&. .SS "Experimental API" .PP The @@ -1501,6 +1637,10 @@ The \fBmallctl*\fR\fB\fR, and \fB*allocm\fR\fB\fR functions first appeared in FreeBSD 10\&.0\&. +.PP +The +\fB*allocx\fR\fB\fR +functions first appeared in FreeBSD 11\&.0\&. .SH "AUTHOR" .PP \fBJason Evans\fR Modified: head/contrib/jemalloc/include/jemalloc/internal/arena.h ============================================================================== --- head/contrib/jemalloc/include/jemalloc/internal/arena.h Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/include/jemalloc/internal/arena.h Thu Jan 23 02:47:36 2014 (r261071) @@ -158,6 +158,7 @@ struct arena_chunk_map_s { }; typedef rb_tree(arena_chunk_map_t) arena_avail_tree_t; typedef rb_tree(arena_chunk_map_t) arena_run_tree_t; +typedef ql_head(arena_chunk_map_t) arena_chunk_mapelms_t; /* Arena chunk header. */ struct arena_chunk_s { @@ -174,11 +175,12 @@ struct arena_chunk_s { size_t nruns_avail; /* - * Number of available run adjacencies. Clean and dirty available runs - * are not coalesced, which causes virtual memory fragmentation. The - * ratio of (nruns_avail-nruns_adjac):nruns_adjac is used for tracking - * this fragmentation. - * */ + * Number of available run adjacencies that purging could coalesce. + * Clean and dirty available runs are not coalesced, which causes + * virtual memory fragmentation. The ratio of + * (nruns_avail-nruns_adjac):nruns_adjac is used for tracking this + * fragmentation. + */ size_t nruns_adjac; /* @@ -404,7 +406,16 @@ void arena_tcache_fill_small(arena_t *ar size_t binind, uint64_t prof_accumbytes); void arena_alloc_junk_small(void *ptr, arena_bin_info_t *bin_info, bool zero); +#ifdef JEMALLOC_JET +typedef void (arena_redzone_corruption_t)(void *, size_t, bool, size_t, + uint8_t); +extern arena_redzone_corruption_t *arena_redzone_corruption; +typedef void (arena_dalloc_junk_small_t)(void *, arena_bin_info_t *); +extern arena_dalloc_junk_small_t *arena_dalloc_junk_small; +#else void arena_dalloc_junk_small(void *ptr, arena_bin_info_t *bin_info); +#endif +void arena_quarantine_junk_small(void *ptr, size_t usize); void *arena_malloc_small(arena_t *arena, size_t size, bool zero); void *arena_malloc_large(arena_t *arena, size_t size, bool zero); void *arena_palloc(arena_t *arena, size_t size, size_t alignment, bool zero); @@ -415,10 +426,18 @@ void arena_dalloc_bin(arena_t *arena, ar size_t pageind, arena_chunk_map_t *mapelm); void arena_dalloc_small(arena_t *arena, arena_chunk_t *chunk, void *ptr, size_t pageind); +#ifdef JEMALLOC_JET +typedef void (arena_dalloc_junk_large_t)(void *, size_t); +extern arena_dalloc_junk_large_t *arena_dalloc_junk_large; +#endif void arena_dalloc_large_locked(arena_t *arena, arena_chunk_t *chunk, void *ptr); void arena_dalloc_large(arena_t *arena, arena_chunk_t *chunk, void *ptr); -void *arena_ralloc_no_move(void *ptr, size_t oldsize, size_t size, +#ifdef JEMALLOC_JET +typedef void (arena_ralloc_junk_large_t)(void *, size_t, size_t); +extern arena_ralloc_junk_large_t *arena_ralloc_junk_large; +#endif +bool arena_ralloc_no_move(void *ptr, size_t oldsize, size_t size, size_t extra, bool zero); void *arena_ralloc(arena_t *arena, void *ptr, size_t oldsize, size_t size, size_t extra, size_t alignment, bool zero, bool try_tcache_alloc, @@ -473,7 +492,7 @@ size_t arena_bin_index(arena_t *arena, a unsigned arena_run_regind(arena_run_t *run, arena_bin_info_t *bin_info, const void *ptr); prof_ctx_t *arena_prof_ctx_get(const void *ptr); -void arena_prof_ctx_set(const void *ptr, prof_ctx_t *ctx); +void arena_prof_ctx_set(const void *ptr, size_t usize, prof_ctx_t *ctx); void *arena_malloc(arena_t *arena, size_t size, bool zero, bool try_tcache); size_t arena_salloc(const void *ptr, bool demote); void arena_dalloc(arena_t *arena, arena_chunk_t *chunk, void *ptr, @@ -885,10 +904,10 @@ arena_prof_ctx_get(const void *ptr) } JEMALLOC_INLINE void -arena_prof_ctx_set(const void *ptr, prof_ctx_t *ctx) +arena_prof_ctx_set(const void *ptr, size_t usize, prof_ctx_t *ctx) { arena_chunk_t *chunk; - size_t pageind, mapbits; + size_t pageind; cassert(config_prof); assert(ptr != NULL); @@ -896,10 +915,17 @@ arena_prof_ctx_set(const void *ptr, prof chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr); pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> LG_PAGE; - mapbits = arena_mapbits_get(chunk, pageind); - assert((mapbits & CHUNK_MAP_ALLOCATED) != 0); - if ((mapbits & CHUNK_MAP_LARGE) == 0) { + assert(arena_mapbits_allocated_get(chunk, pageind) != 0); + + if (usize > SMALL_MAXCLASS || (prof_promote && + ((uintptr_t)ctx != (uintptr_t)1U || arena_mapbits_large_get(chunk, + pageind) != 0))) { + assert(arena_mapbits_large_get(chunk, pageind) != 0); + arena_mapp_get(chunk, pageind)->prof_ctx = ctx; + } else { + assert(arena_mapbits_large_get(chunk, pageind) == 0); if (prof_promote == false) { + size_t mapbits = arena_mapbits_get(chunk, pageind); arena_run_t *run = (arena_run_t *)((uintptr_t)chunk + (uintptr_t)((pageind - (mapbits >> LG_PAGE)) << LG_PAGE)); @@ -911,12 +937,11 @@ arena_prof_ctx_set(const void *ptr, prof bin_info = &arena_bin_info[binind]; regind = arena_run_regind(run, bin_info, ptr); - *((prof_ctx_t **)((uintptr_t)run + bin_info->ctx0_offset - + (regind * sizeof(prof_ctx_t *)))) = ctx; - } else - assert((uintptr_t)ctx == (uintptr_t)1U); - } else - arena_mapp_get(chunk, pageind)->prof_ctx = ctx; + *((prof_ctx_t **)((uintptr_t)run + + bin_info->ctx0_offset + (regind * sizeof(prof_ctx_t + *)))) = ctx; + } + } } JEMALLOC_ALWAYS_INLINE void * Modified: head/contrib/jemalloc/include/jemalloc/internal/chunk_dss.h ============================================================================== --- head/contrib/jemalloc/include/jemalloc/internal/chunk_dss.h Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/include/jemalloc/internal/chunk_dss.h Thu Jan 23 02:47:36 2014 (r261071) @@ -7,7 +7,7 @@ typedef enum { dss_prec_secondary = 2, dss_prec_limit = 3 -} dss_prec_t ; +} dss_prec_t; #define DSS_PREC_DEFAULT dss_prec_secondary #define DSS_DEFAULT "secondary" Modified: head/contrib/jemalloc/include/jemalloc/internal/ckh.h ============================================================================== --- head/contrib/jemalloc/include/jemalloc/internal/ckh.h Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/include/jemalloc/internal/ckh.h Thu Jan 23 02:47:36 2014 (r261071) @@ -17,7 +17,7 @@ typedef bool ckh_keycomp_t (const void * * There are 2^LG_CKH_BUCKET_CELLS cells in each hash table bucket. Try to fit * one bucket per L1 cache line. */ -#define LG_CKH_BUCKET_CELLS (LG_CACHELINE - LG_SIZEOF_PTR - 1) +#define LG_CKH_BUCKET_CELLS (LG_CACHELINE - LG_SIZEOF_PTR - 1) #endif /* JEMALLOC_H_TYPES */ /******************************************************************************/ Modified: head/contrib/jemalloc/include/jemalloc/internal/hash.h ============================================================================== --- head/contrib/jemalloc/include/jemalloc/internal/hash.h Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/include/jemalloc/internal/hash.h Thu Jan 23 02:47:36 2014 (r261071) @@ -19,6 +19,11 @@ #ifdef JEMALLOC_H_INLINES #ifndef JEMALLOC_ENABLE_INLINE +uint32_t hash_x86_32(const void *key, int len, uint32_t seed); +void hash_x86_128(const void *key, const int len, uint32_t seed, + uint64_t r_out[2]); +void hash_x64_128(const void *key, const int len, const uint32_t seed, + uint64_t r_out[2]); void hash(const void *key, size_t len, const uint32_t seed, size_t r_hash[2]); #endif @@ -43,14 +48,14 @@ JEMALLOC_INLINE uint32_t hash_get_block_32(const uint32_t *p, int i) { - return p[i]; + return (p[i]); } JEMALLOC_INLINE uint64_t hash_get_block_64(const uint64_t *p, int i) { - return p[i]; + return (p[i]); } JEMALLOC_INLINE uint32_t @@ -63,7 +68,7 @@ hash_fmix_32(uint32_t h) h *= 0xc2b2ae35; h ^= h >> 16; - return h; + return (h); } JEMALLOC_INLINE uint64_t @@ -76,7 +81,7 @@ hash_fmix_64(uint64_t k) k *= QU(0xc4ceb9fe1a85ec53LLU); k ^= k >> 33; - return k; + return (k); } JEMALLOC_INLINE uint32_t @@ -127,12 +132,12 @@ hash_x86_32(const void *key, int len, ui h1 = hash_fmix_32(h1); - return h1; + return (h1); } UNUSED JEMALLOC_INLINE void hash_x86_128(const void *key, const int len, uint32_t seed, - uint64_t r_out[2]) + uint64_t r_out[2]) { const uint8_t * data = (const uint8_t *) key; const int nblocks = len / 16; @@ -234,7 +239,7 @@ hash_x86_128(const void *key, const int UNUSED JEMALLOC_INLINE void hash_x64_128(const void *key, const int len, const uint32_t seed, - uint64_t r_out[2]) + uint64_t r_out[2]) { const uint8_t *data = (const uint8_t *) key; const int nblocks = len / 16; @@ -310,7 +315,6 @@ hash_x64_128(const void *key, const int r_out[1] = h2; } - /******************************************************************************/ /* API. */ JEMALLOC_INLINE void Modified: head/contrib/jemalloc/include/jemalloc/internal/huge.h ============================================================================== --- head/contrib/jemalloc/include/jemalloc/internal/huge.h Thu Jan 23 02:10:30 2014 (r261070) +++ head/contrib/jemalloc/include/jemalloc/internal/huge.h Thu Jan 23 02:47:36 2014 (r261071) @@ -19,10 +19,14 @@ extern malloc_mutex_t huge_mtx; void *huge_malloc(size_t size, bool zero); void *huge_palloc(size_t size, size_t alignment, bool zero); -void *huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size, +bool huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size, size_t extra); void *huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra, *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***