From owner-freebsd-current@FreeBSD.ORG Thu May 31 21:20:59 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39760106566C for ; Thu, 31 May 2012 21:20:59 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id A12CD8FC0A for ; Thu, 31 May 2012 21:20:58 +0000 (UTC) Received: from server.rulingia.com (c220-239-254-65.belrs5.nsw.optusnet.com.au [220.239.254.65]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q4VLKom7073838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 1 Jun 2012 07:20:51 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q4VLKhaQ043491 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 1 Jun 2012 07:20:43 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q4VLKgvL043490; Fri, 1 Jun 2012 07:20:42 +1000 (EST) (envelope-from peter) Date: Fri, 1 Jun 2012 07:20:42 +1000 From: Peter Jeremy To: Dmitry Marakasov Message-ID: <20120531212042.GA43155@server.rulingia.com> References: <20120527140541.GL2987@hades.panopticon> <20120528094427.GE2675@aspire.rulingia.com> <20120528195542.GB85856@hades.panopticon> <20120528200403.GB15356@server.rulingia.com> <20120528221825.GB38860@hades.panopticon> <20120530032703.GA99357@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GvXjxJ+pjyke8COw" Content-Disposition: inline In-Reply-To: <20120530032703.GA99357@server.rulingia.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@freebsd.org Subject: Re: OptionalObsoleteFiles.inc completeness X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 21:20:59 -0000 --GvXjxJ+pjyke8COw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-May-30 13:27:03 +1000, Peter Jeremy wrote: >On 2012-May-29 02:18:25 +0400, Dmitry Marakasov wrote: >>Then you should try to profile it - my script basically runs >>delete-old delete-old-libs for every knob (131 of them), and it >>hadn't taken more than 4 seconds even once. > >I've done some investigating and the problem is that "xargs -n1" >fork()/exec()s /bin/echo on each file (and there are 5538 files for >me). Changing this to "tr ' ' '\n'" reduces "make delete-old" runtime >to 1.75s - which is much nicer. I've checked a variety of other >systems running 8.x & 9.x and the 97s seems to be anomalously long so >I'll do some more investigating. I've tracked the problem down to excessive VM faults caused by jemalloc. Whilst executing /bin/echo, jemalloc mmap()s two 4MiB chunks of memory. Unless you build with MALLOC_PRODUCTION (which I hadn't), it then proceeds to verify that both blocks are zero-filled. This causes 2048 (unnecessary) page faults (out of a total of 2133). When I rebuilt jemalloc with MALLOC_PRODUCTION, this dropped to 87 page faults (cf 76 an 8.x and 62 on 9.x) and the elapsed time for "make delete-old" dropped to slightly more than 8.x & 9.x. "xargs -n1" is probably a worst case scenario for jemalloc but this probably similarly affects other short-lived processes (and the shell scripts that invoke them). It's a pity that this particular test is a compile-time option. I still think that saving 5500 fork()/exec() pairs is a good reason to switch from "xargs -n1" to "tr ' ' '\n'". --=20 Peter Jeremy --GvXjxJ+pjyke8COw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/H4KoACgkQ/opHv/APuIeojACdEkai5QCwXoBhblK6jGfJuloD bVcAnRq1OLVpWWhU7OTgxACbyloYoY91 =yPPE -----END PGP SIGNATURE----- --GvXjxJ+pjyke8COw--