From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 25 15:29:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 870B2106566C for ; Thu, 25 Feb 2010 15:29:31 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id 1CC1A8FC12 for ; Thu, 25 Feb 2010 15:29:30 +0000 (UTC) Received: by ewy26 with SMTP id 26so309106ewy.3 for ; Thu, 25 Feb 2010 07:29:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=oCfGaV7eurfHZTLdm+JhCmjvCNm9k7KTlXwfaGTgXns=; b=GAqmCJ+JPUYFjwG+7YNamPFwEMlDAs1qfmLKZcQjBfhe7azxYs8NztQuGmfuSGH2tR 3HJPNmRAMlJYkn97ZJbMSFUN4v6a97FPHKmlesMUVSkqrNybN53ycL4goi4g1zuMNL+B JJpSczRVgNcdrsdgih7t+xQufMsC1fUv8KEVo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mn8zC4lLOMKzJC03eJd64/pEFGDPbgZqw7FzJWCj5VAPFFDQsf445m5pnrRaf6L8ZR qsnXa2ReeUF2Az+FNh9f6iIIHDgpL8zzbw73g+TjdGsxv8kpWY16xVVKqma+jXkN/yeM Vxq0+R7wFjPIp+DgL2UreZ4jsmvPMUm4FffhQ= MIME-Version: 1.0 Received: by 10.216.182.10 with SMTP id n10mr662149wem.142.1267111760766; Thu, 25 Feb 2010 07:29:20 -0800 (PST) In-Reply-To: <20100225135303.GM57731@acme.spoerlein.net> References: <20100225135303.GM57731@acme.spoerlein.net> Date: Thu, 25 Feb 2010 17:29:19 +0200 Message-ID: From: David Naylor To: David Naylor , freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: [Proof of Concept] Stacked unionfs based 'tinderbox' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 15:29:31 -0000 On 25 February 2010 15:53, Ulrich Sp=C3=B6rlein wrote: > On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote: >> Hi, >> >> As some may have noticed on -current I have been working on using >> stacked unionfs to implement a 'tinderbox' type build system. =C2=A0I ha= ve >> successfully used the scripts to build x11/xorg (and have compared the >> results to using the traditional approach using pkg_add). =C2=A0The buil= d >> system is stable except for one nasty corner case: deadlocks. > > When I did this a couple of years ago, the major problems were failing > chdir(2) calls during ports build, etc. > >> To setup a compatible test environment requires: >> =C2=A0- recompile the kernel with `options KSTACK_PAGES=3D32`, otherwise= the >> =C2=A0kernel will panic with a double fault. =C2=A0WITNESS options resul= ts in >> =C2=A0substantial performance degradation. >> =C2=A0- patch mtree (see below) [optional] >> =C2=A0- create the appropriate chroot environment (and reboot) [see belo= w >> =C2=A0for corner case] >> >> A performance bottleneck in mtree was identified. =C2=A0This resulted in >> mtree (as run by port install) consuming ~20% of the build time. =C2=A0S= ee >> bin/143732 for a patch and further details. > > Good work! > >> The normal tinderbox approach takes ~80% more time to install compared t= o the >> quick and dirty approach. =C2=A0The stacked unionfs approach takes ~170%= more time >> (an increase of ~50% over the tinderbox approach). =C2=A0Some performanc= e gains can >> be had if one uses memory backed storage (vs HDD in this case). > > Please explain: what is the quick and dirty approach and which one is > faster now? The quick and dirty is `make -C /usr/ports/x11/xorg install clean`. The stacked unionfs is still the slowest (even with a 20% improvement from patching mtree). > As your scripts did not make it through, perhaps you can upload them to > the wiki? What I did back then was using a clean base system as the > underlying unionfs store to avoid re-generating the clean base over and > over again. Nowadays, a ZFS clone would probably be the way to go. > > I'm not sure if a recursive approach is feasible here, as you can have > only one underlying unionfs mount. But special casing, e.g., perl may > still give a massive speedup. So for each port that has perl as > dependancy, you would not pull in the clean base + pkg_add perl, but > instead grab the clean-base+perl directory as an underlying unionfs. > > Cheers > Uli >