From owner-freebsd-current@FreeBSD.ORG Fri Jun 1 18:50:30 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC658106564A for ; Fri, 1 Jun 2012 18:50:29 +0000 (UTC) (envelope-from uqs@FreeBSD.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 954A18FC17 for ; Fri, 1 Jun 2012 18:50:29 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.5/8.14.5) with ESMTP id q51IoOWn089510 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 1 Jun 2012 20:50:26 +0200 (CEST) (envelope-from uqs@FreeBSD.org) Date: Fri, 1 Jun 2012 20:50:24 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Peter Jeremy Message-ID: <20120601185024.GP10094@acme.spoerlein.net> Mail-Followup-To: Peter Jeremy , Dmitry Marakasov , freebsd-current@freebsd.org References: <20120527140541.GL2987@hades.panopticon> <20120528094427.GE2675@aspire.rulingia.com> <20120528195542.GB85856@hades.panopticon> <20120528200403.GB15356@server.rulingia.com> <20120528221825.GB38860@hades.panopticon> <20120530032703.GA99357@server.rulingia.com> <20120531212042.GA43155@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline In-Reply-To: <20120531212042.GA43155@server.rulingia.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@FreeBSD.org, Dmitry Marakasov Subject: Re: OptionalObsoleteFiles.inc completeness X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 18:50:30 -0000 --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, 2012-06-01 at 07:20:42 +1000, Peter Jeremy wrote: > On 2012-May-30 13:27:03 +1000, Peter Jeremy wrote: > >On 2012-May-29 02:18:25 +0400, Dmitry Marakasov wrote: > >>Then you should try to profile it - my script basically runs > >>delete-old delete-old-libs for every knob (131 of them), and it > >>hadn't taken more than 4 seconds even once. > > > >I've done some investigating and the problem is that "xargs -n1" > >fork()/exec()s /bin/echo on each file (and there are 5538 files for > >me). Changing this to "tr ' ' '\n'" reduces "make delete-old" runtime > >to 1.75s - which is much nicer. I've checked a variety of other > >systems running 8.x & 9.x and the 97s seems to be anomalously long so > >I'll do some more investigating. >=20 > I've tracked the problem down to excessive VM faults caused by > jemalloc. Whilst executing /bin/echo, jemalloc mmap()s two 4MiB > chunks of memory. Unless you build with MALLOC_PRODUCTION (which I > hadn't), it then proceeds to verify that both blocks are zero-filled. > This causes 2048 (unnecessary) page faults (out of a total of 2133). > When I rebuilt jemalloc with MALLOC_PRODUCTION, this dropped to 87 > page faults (cf 76 an 8.x and 62 on 9.x) and the elapsed time for > "make delete-old" dropped to slightly more than 8.x & 9.x. >=20 > "xargs -n1" is probably a worst case scenario for jemalloc but this > probably similarly affects other short-lived processes (and the shell > scripts that invoke them). It's a pity that this particular test is a > compile-time option. >=20 > I still think that saving 5500 fork()/exec() pairs is a good reason > to switch from "xargs -n1" to "tr ' ' '\n'". Why is xargs even calling /bin/echo when "utility" is not specified. Shouldn't it just print a certain number of arguments (one in this case)? Uli --Kj7319i9nmIyA2yE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iQEcBAEBAgAGBQJPyQ7wAAoJEKOmmGRKr4LOz5wIAIkAYiFbuk7UiOqbQnSGKX+J JVHFGDTSaZN5PLBA7LrlhKQ1ZjFKKNkp6UUQQaHb1juV21DG3oVFPidgt6C2r6x6 jPiOeSj5wE9Gxtvasgfz8Cp2O0L3d3pREitOBmtPpnhE+zPD1PbrZIqTkEKieRKJ a+cml7CV3bbyK6NrG6lITNnLljJebzH/g6lx57b4zIbxv7nLjObF2EZG9GC6fjGw IxxJJ3idHwl46tdhJ4UUncwUngsTBBttZ111fqbL0DizGF3nXhPP7wvEMA24p5lm X1kdUaI39gshImf7NWAinndprClWWIFDdzZdILcJN44IQk4osVwC0K3R5iOrdYs= =tP1N -----END PGP SIGNATURE----- --Kj7319i9nmIyA2yE--