From owner-freebsd-current@FreeBSD.ORG Wed Jan 27 20:58:32 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B15F91065670 for ; Wed, 27 Jan 2010 20:58:32 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-ew0-f218.google.com (mail-ew0-f218.google.com [209.85.219.218]) by mx1.freebsd.org (Postfix) with ESMTP id 38FFA8FC13 for ; Wed, 27 Jan 2010 20:58:31 +0000 (UTC) Received: by ewy10 with SMTP id 10so1728607ewy.3 for ; Wed, 27 Jan 2010 12:58:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:organization:to:subject :date:user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:message-id; bh=YcVj5taHUAtaXhly+RBJYxgdExvogBg602pXoBYpYYc=; b=WElOBWBplTlxPCgFJr9DFYORp6sYZYOVPnZQ0lnmPQg3uD9fVcDz+BOeMSG6FhEvMn FoZ93vBbZ+YAj2ye+cmd4yo9+gaMIApUqhcUz8KwkMoQD2dobO+TeQTKSp8u99IpBrRI Y00NztPZfgalpbAclUhy/pr8V6PiRFBr+/ch8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:organization:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id; b=LBK6YIAtzgLCqderEi3C9yzdX866i3lh3NTsCzIiOsUmWf7rA38jdcMUnsamSVtnI9 QMFTJLWIKvDO7YzlU/HXaqYRSXx0vQjAQNekcGD0ijcTEWtttugH2mUtGuG0Ae5kvmTS d77WzYj3b4bLyeDMAmPeSWbimS4FU2Qn6IxXU= Received: by 10.213.44.17 with SMTP id y17mr4420486ebe.25.1264625910269; Wed, 27 Jan 2010 12:58:30 -0800 (PST) Received: from dragon.dg ([41.216.197.17]) by mx.google.com with ESMTPS id 10sm586948eyz.15.2010.01.27.12.58.27 (version=SSLv3 cipher=RC4-MD5); Wed, 27 Jan 2010 12:58:28 -0800 (PST) From: David Naylor Organization: Private To: gary.jennejohn@freenet.de Date: Wed, 27 Jan 2010 22:58:31 +0200 User-Agent: KMail/1.12.3 (FreeBSD/8.0-STABLE; KDE/4.3.3; amd64; ; ) References: <201001231233.18832.naylor.b.david@gmail.com> <20100126140459.61e3951d@ernst.jennejohn.org> <201001261841.23815.naylor.b.david@gmail.com> In-Reply-To: <201001261841.23815.naylor.b.david@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1335951.JWWOHOyWEC"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201001272258.35184.naylor.b.david@gmail.com> Cc: freebsd-current@freebsd.org Subject: Re: "tinderbox" using stacked unionfs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2010 20:58:32 -0000 --nextPart1335951.JWWOHOyWEC Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Tuesday 26 January 2010 18:41:20 David Naylor wrote: > On Tuesday 26 January 2010 15:04:59 Gary Jennejohn wrote: > > On Mon, 25 Jan 2010 20:39:11 +0200 > > > > David Naylor wrote: > > > On Saturday 23 January 2010 12:33:14 David Naylor wrote: > > > > Hi, > > > > > > > > I have been experimenting with using unionfs to build ports in a > > > > "tinderbox" environment. This avoids having to remove and extract > > > > files for the build of each port and it allows the discovery of > > > > installed/modified files by the port. > > > > > > > > Please see attached for a (updated) shell script that handles all t= he > > > > "heavy lifting" of building ports. Of importance is LOCALBASE and > > > > BUILDDIR. If you want to override LOCALBASE please use `env` as t= he > > > > script needs to know about it. BUILDDIR (/usr/build by default) is > > > > where the script stores everything (including PKG_DBDIR). > > > > > > Please see attached for an updated script. This no longer uses `sort > > > -u` but removed duplicates while maintaining dependency order. (See > > > below) > > > > ENOSCRIPT >=20 > See attached. Processes are still freezing periodically (requiring a > restart). The mtree problem appears when installing meta ports (x11/xorg- > apps, x11/xorg, etc). The script finally managed to finish without stalling. Here are some=20 comparative results: # time make -C /usr/ports/x11/xorg install clean 2185.180u 1066.927s 38:12.27 141.8% 4123+1745k 5185+1890io 51638pf+0w # time ./ports-union-builder.sh 2264.914u 6097.779s 2:15:07.39 103.1% 3777+1608k 171735+226215io 51424pf+= 0w This results in the unionfs method being ~2.5x longer in execution. This=20 difference should decrease a fair amount if the long execution times of mtr= ee=20 can be reduced. =20 The benefits of using memory backed UFS storage has not been explored. =20 NOTE: ports-union-builder.sh also produces packages. =20 --nextPart1335951.JWWOHOyWEC Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEABECAAYFAktgqPsACgkQUaaFgP9pFrInNQCfTbYbXHmYexfCExQfpWAbIAmG 9nsAn0eqWhjB+9dml9+AN6O5589UIinC =e8dc -----END PGP SIGNATURE----- --nextPart1335951.JWWOHOyWEC--