From owner-freebsd-gecko@freebsd.org Thu Jul 16 21:38:07 2015 Return-Path: Delivered-To: freebsd-gecko@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 162689A3C94 for ; Thu, 16 Jul 2015 21:38:07 +0000 (UTC) (envelope-from jbeich@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EDD3B1DB8 for ; Thu, 16 Jul 2015 21:38:06 +0000 (UTC) (envelope-from jbeich@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id ECA6E9A3C93; Thu, 16 Jul 2015 21:38:06 +0000 (UTC) Delivered-To: gecko@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC2F99A3C92 for ; Thu, 16 Jul 2015 21:38:06 +0000 (UTC) (envelope-from jbeich@freebsd.org) Received: from vfemail.net (ninezero.vfemail.net [96.30.253.190]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA1E91DB6 for ; Thu, 16 Jul 2015 21:38:06 +0000 (UTC) (envelope-from jbeich@freebsd.org) Received: (qmail 84805 invoked by uid 89); 16 Jul 2015 21:31:24 -0000 Received: from localhost (HELO freequeue.vfemail.net) (127.0.0.1) by localhost with (DHE-RSA-AES256-SHA encrypted) SMTP; 16 Jul 2015 21:31:24 -0000 Received: (qmail 84788 invoked by uid 89); 16 Jul 2015 21:31:07 -0000 Received: by simscan 1.3.1 ppid: 84781, pid: 84784, t: 0.0039s scanners:none Received: from unknown (HELO smtp102-2.vfemail.net) (172.16.100.62) by FreeQueue with SMTP; 16 Jul 2015 21:31:07 -0000 Received: (qmail 14277 invoked by uid 89); 16 Jul 2015 21:31:07 -0000 Received: by simscan 1.4.0 ppid: 14247, pid: 14273, t: 0.8588s scanners:none Received: from unknown (HELO nil) (amJlaWNoQHZmZW1haWwubmV0@172.16.100.27) by 172.16.100.62 with ESMTPA; 16 Jul 2015 21:31:06 -0000 From: Jan Beich To: "Mikhail T." Cc: gecko@FreeBSD.org, flo@FreeBSD.org, beat@FreeBSD.org Subject: Re: Why use jemalloc bundled with Mozilla? References: <55918759.5060404@aldan.algebra.com> Date: Thu, 16 Jul 2015 23:30:47 +0200 Message-ID: <4ml3-94e0-wny@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: freebsd-gecko@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Gecko Rendering Engine issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jul 2015 21:38:07 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable "Mikhail T." writes: > The system malloc on FreeBSD-10.x is jemalloc-3.4.x. Why do Mozilla ports= build > with "--enable-jemalloc" turned on despite that? Firefox devs work with jemalloc upstream, so newer version is preferred. Some changes in 37.0 make it harder to support in-base version: # stats.bookkeeping, available as stats.metadata since jemalloc-4.0.0 https://bugzilla.mozilla.org/show_bug.cgi?id=3D899126 https://github.com/jemalloc/jemalloc/issues/163 # junk:free, available since jemalloc-4.0.0 https://bugzilla.mozilla.org/show_bug.cgi?id=3D1108045 https://github.com/jemalloc/jemalloc/pull/172 # *allocx() is N/A before jemalloc-3.4.x while *allocm() is N/A in 4.0.0 https://hg.mozilla.org/mozilla-central/diff/b5b1a8e8aee4/memory/build/mozje= malloc_compat.c https://hg.mozilla.org/mozilla-central/diff/96a98f1d5aa8/configure.in https://github.com/jemalloc/jemalloc/commit/9790b9667fd975b1f9a4f108f9d0a20= ab265c6b6 > Is it just a typo in bsd.gecko.mk and the fix as simple as: As nallocx() is not available on FreeBSD 10.x non-standard API usage would be silently disabled. As stats.bookkeeping isn't available in any released jemalloc version while CTL_GET macro lacks error-checking it'd return garbage in about:memory on FreeBSD 11.0C. However, using bundled jemalloc leads to unnecessary separation for memory accounting e.g., try looking at stderr after $ MALLOC_CONF=3Dstats_print:true firefox ... $ JE_MALLOC_CONF=3Dstats_print:true firefox ... > > --- /usr/ports/Mk/bsd.gecko.mk (revision 390575) > +++ /usr/ports/Mk/bsd.gecko.mk (working copy) > @@ -130,7 +130,7 @@ >=20=20=20=20=20=20 > # use jemalloc 3.0.0 API for stats/tuning > MOZ_EXPORT+=3D MOZ_JEMALLOC3=3D1 > -.if ${OSVERSION} < 1000012 || ${MOZILLA_VER:R:R} >=3D 37 > +.if ${OSVERSION} < 1000012 && ${MOZILLA_VER:R:R} >=3D 37 > MOZ_OPTIONS+=3D --enable-jemalloc > .endif >=20=20=20=20=20=20 > > or is there more to it? Thanks! Yours, If you're eager for preview try (before jasone@ tags and imports 4.0.0) # See VERSION or FREEBSD-upgrade under contrib/jemalloc/ to reproduce $ fetch https://people.freebsd.org/~jbeich/jemalloc_update.20150716.diff.= xz $ xzcat jemalloc_update.20150716.diff.xz | (cd /usr/src; patch -Efsp0) then apply the following ports' patch, rebuild firefox and check "Measure" in about:memory makes sense. heap-* reporters should be closer to what MALLOC_CONF=3Dstats_print:true shows. Index: Mk/bsd.gecko.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =2D-- Mk/bsd.gecko.mk (revision 392295) +++ Mk/bsd.gecko.mk (working copy) @@ -130,9 +130,6 @@ LDFLAGS+=3D -L${LOCALBASE}/lib -Wl,-rpath,${PREFIX} =20 # use jemalloc 3.0.0 API for stats/tuning MOZ_EXPORT+=3D MOZ_JEMALLOC3=3D1 =2D.if ${OSVERSION} < 1000012 || ${MOZILLA_VER:R:R} >=3D 37 =2DMOZ_OPTIONS+=3D --enable-jemalloc =2D.endif =20 # Standard depends _ALL_DEPENDS=3D cairo event ffi graphite harfbuzz hunspell icu jpeg nspr n= ss opus png pixman soundtouch sqlite vorbis vpx Index: www/firefox/files/patch-jemalloc4 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =2D-- www/firefox/files/patch-jemalloc4 (revision 0) +++ www/firefox/files/patch-jemalloc4 (working copy) @@ -0,0 +1,11 @@ +--- memory/build/mozjemalloc_compat.c~ ++++ memory/build/mozjemalloc_compat.c +@@ -150,7 +150,7 @@ jemalloc_stats_impl(jemalloc_stats_t *st + CTL_GET("stats.allocated", allocated); + CTL_GET("stats.mapped", mapped); + CTL_GET("opt.lg_chunk", lg_chunk); +- CTL_GET("stats.bookkeeping", stats->bookkeeping); ++ CTL_GET("stats.metadata", stats->bookkeeping); +=20 + /* get the summation for all arenas, i =3D=3D narenas */ + CTL_I_GET("stats.arenas.0.pdirty", pdirty, narenas); --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQF8BAEBCgBmBQJVqCKIXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXREQjQ0MzY3NEM3RDIzNTc4NkUxNDkyQ0VF NEM3Nzg4MzQ3OURCRERCAAoJEOTHeINHnb3bzqgH/ApZzOoJg8E0vSVQmWqHNRhb iFYxpo+YYXAzVu1XPSQzI8fJLYjU1ue1M4gC1uczAsHW0clJ8EDeQ8nzYRyhTXVL 3XhFtnc4CNIVQKnP5hRLpyMTmRT8N9pdYoJ/ub8CsB8KZC2XqS6aMJM5cpNK0XL1 zG4n2G7ByAyjQ40hBaiwjeDJbpBsEvbK51z7C5OLWAt8XC8YAjSmVMbHCG5VYE5x 2kTgVZNhmRzFMVnkHttiNVIZRpDEXoKn8NfWVo8yZ4CRe+kasnp6tI4qI5HK/JEn V0nBZUpPcnabvGUvLNxn1yYcon+PfzBkiW5lHvrLEHkTmNd9myFuQFMcvmgZn8U= =9BRh -----END PGP SIGNATURE----- --=-=-=--