From owner-freebsd-current@FreeBSD.ORG Mon Jan 25 18:39:11 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 441CF1065692 for ; Mon, 25 Jan 2010 18:39:11 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id C321F8FC16 for ; Mon, 25 Jan 2010 18:39:10 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id e12so308096fga.13 for ; Mon, 25 Jan 2010 10:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:organization:to:subject :date:user-agent:references:in-reply-to:mime-version:content-type :content-transfer-encoding:message-id; bh=TXYA5ebh0IFWjCc83JRzO0gQxogcN9ENHVcLCvh4n64=; b=T7gL/Mw/L9BPJZOPQaopgEHKeSbKJRtQELa6MrKCKsN3+KDKXuTc3bu5Ps7toSdlWO k2STRUawrSf4ochA7Owwztw4sDXpcSLOLPNlvkvgB3tpLVUTC9db/euINlAz/K466+DK BZz8gWChSwbSe3keRPbNBxhvhtQq6f0SNuN7M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:organization:to:subject:date:user-agent:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; b=p1k/KlQWZuMsmIu31PLTNe5Rh80YqgyD+igt3gSLkvDWUJhaXaYbkzZaz9iTxLlbL6 8OO05S/kRwWdyUojEAfjRavFv38hO2Q+NrkvjynK+IZG98YrXJ3WnbjsBMEYlSH0RGF+ aKH+mkx571A7mvIf7WodswSHaYXg3nEFV5Ha8= Received: by 10.87.54.5 with SMTP id g5mr5623532fgk.78.1264444749754; Mon, 25 Jan 2010 10:39:09 -0800 (PST) Received: from dragon.dg ([41.216.197.17]) by mx.google.com with ESMTPS id 16sm2888423fxm.8.2010.01.25.10.39.06 (version=SSLv3 cipher=RC4-MD5); Mon, 25 Jan 2010 10:39:08 -0800 (PST) From: David Naylor Organization: Private To: freebsd-current@freebsd.org Date: Mon, 25 Jan 2010 20:39:11 +0200 User-Agent: KMail/1.12.3 (FreeBSD/9.0-CURRENT; KDE/4.3.3; amd64; ; ) References: <201001231233.18832.naylor.b.david@gmail.com> In-Reply-To: <201001231233.18832.naylor.b.david@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1671660.s51KELAnJF"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201001252039.16220.naylor.b.david@gmail.com> Subject: Re: "tinderbox" using stacked unionfs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2010 18:39:11 -0000 --nextPart1671660.s51KELAnJF Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Saturday 23 January 2010 12:33:14 David Naylor wrote: > Hi, >=20 > I have been experimenting with using unionfs to build ports in a > "tinderbox" environment. This avoids having to remove and extract files > for the build of each port and it allows the discovery of > installed/modified files by the port. >=20 > Please see attached for a (updated) shell script that handles all the > "heavy lifting" of building ports. Of importance is LOCALBASE and > BUILDDIR. If you want to override LOCALBASE please use `env` as the > script needs to know about it. BUILDDIR (/usr/build by default) is where > the script stores everything (including PKG_DBDIR). Please see attached for an updated script. This no longer uses `sort -u` b= ut=20 removed duplicates while maintaining dependency order. (See below) > # env LOCALBASE=3D/tmp/local BUILDDIR=3D/tmp/build ./ports-union-builder.= sh >=20 > Will install x11/xorg without affecting already installed systems. >=20 > CURRENT STATUS: > - *** Currently kernel stack size is too small and the above will trigger > a stack overflow. Recompiling a kernel with ``options KSTACK_PAGES=3D32= '' > will alleviate that problem. In building xorg there were at least 207 stacked unionfs (206 ro, 1 rw, all= =20 noatime). =20 > - Currently there is a build problem that affects eggdbus/polkit (possib= ly > others) thus preventing x11/xorg from being built. I will investigate the > cause (help welcome). This is due to not mounting the dependencies in the correct order. If=20 dependency 'a' modified file from dependency 'b' then mounting in order 'a'= ,=20 'b' will result in those changes being lost as the original files from 'b'= =20 will supersede 'a'. The dependencies need to be mounted 'b', 'a'. =20 This has been fixed although may cause a problem if multiple "independent"= =20 ports modify the same file. This is a detectable problem. =20 > - I highly recommend running this in a chroot > - NO WARRANTY, SLIPPERY WHEN WET, EATS CHILDREN. - I am experiencing process freeze (anything involved in the directories t= hat=20 are unionfs). Any way that I can figure out the problem? I can run a kern= el=20 will full debug capability. =20 - mtree does not behave well with unionfs and consumed a fair amount of=20 resources: # /usr/sbin/mtree -U -f /usr/ports/Templates/BSD.local.dist -d -e -p=20 /usr/local/ took a long time (many minutes) to complete. =20 > Since the script doesn't complete a full build I am unable to compare the > build speeds (thus the overhead of unionfs) but here are some partial > results (with FORCE_MAKE_JOBS and quad core): The script is now able to complete building but not at once due to process= =20 freezing. Please help with debugging the process freezing. (There is a LOR I have=20 reported for unionfs: kern/141950) Regards David --nextPart1671660.s51KELAnJF Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEABECAAYFAktd5VQACgkQUaaFgP9pFrL67wCfdjKnUsfNxGYIgHXQBu7LeEka SvkAnjc6BOd0Nl9W2agsKjox52t+7Sgr =Hh/M -----END PGP SIGNATURE----- --nextPart1671660.s51KELAnJF--