From owner-freebsd-ports@FreeBSD.ORG Tue Dec 18 02:54:54 2007 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D62716A419 for ; Tue, 18 Dec 2007 02:54:54 +0000 (UTC) (envelope-from alepulver@FreeBSD.org) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id 3B5D013C447 for ; Tue, 18 Dec 2007 02:54:54 +0000 (UTC) (envelope-from alepulver@FreeBSD.org) Received: (qmail 59639 invoked by uid 0); 18 Dec 2007 02:54:52 -0000 Received: from unknown (HELO deimos.mars.bsd) (unknown) by unknown with SMTP; 18 Dec 2007 02:54:52 -0000 X-pair-Authenticated: 200.127.63.61 Date: Mon, 17 Dec 2007 23:54:45 -0300 From: Alejandro Pulver To: linimon@lonesome.com (Mark Linimon) Message-ID: <20071217235445.5977bbe2@deimos.mars.bsd> In-Reply-To: <20071217180735.GA19258@soaustin.net> References: <4766650C.4020305@gmail.com> <47668B49.7050804@brianwhalen.net> <20071217125954.22b3bff4@deimos.mars.bsd> <20071217180735.GA19258@soaustin.net> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.1; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/U17piTQKaLmJxY11EUXwX2q"; protocol="application/pgp-signature"; micalg=PGP-SHA1 Cc: Brian , "Aryeh M. Friedman" , freebsd-ports@freebsd.org Subject: Re: Request for Features: Ports Re-engineering X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 02:54:54 -0000 --Sig_/U17piTQKaLmJxY11EUXwX2q Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Mon, 17 Dec 2007 12:07:35 -0600 linimon@lonesome.com (Mark Linimon) wrote: > On Mon, Dec 17, 2007 at 12:59:54PM -0300, Alejandro Pulver wrote: > > if not already done, AFAIK tinderbox has an option to run parallel > > builds, but don't know about pointyhat) using multiple processors. >=20 > pointyhat is actually the dispatch machine. It does not actually > build packages. >=20 Of course, I should've said it another way, but I was referring to the build scripts that run in the build clusters. > The current machine total: >=20 > amd64: 3 (2.2GHz) > i386: 40 (400MHz - 1.8GHz, depending on machine) > sparc64: 7 (440MHz) >=20 Interesting, I only knew about the 3 new amd64s (for which you posted a mail) but didn't know the details of the rest. I remember some of my ports failed on ia64 (some time ago), what happened to it/them? > Several of these systems (in particular the amd64s) have multiple CPUs in= them. >=20 > In any case, each machine is given several packages to work on simultaneo= usly, > depending on its capacity, each in its own fully clean jail. Prerequisit= es > are loaded via the previously built packages; if they are not available, > they are first built, then cached. >=20 > The way the current code works is documented in the following article: >=20 > http://www.freebsd.org/doc/en_US.ISO8859-1/articles/portbuild/index.html >=20 > Although the documentation is terse, it is believed to be generally > correct. >=20 Will read it, didn't know about it. > In general, we use -incremental for the builds, unless there has been > a large change in the src tree, or it's release. -incremental will only > build packages for ports whose metadata (as determined by INDEX) has > changed since the last run. This holds the build time for each package > set down to a few days (amd64/i386) or a couple of weeks (sparc64), > instead of around a week, or a couple of months, respectively. >=20 > Remember that we have builds for 5.x, 6.x, 7.x, and 8.x all running > at various times (some things can be overlapped without adding too much > delay). The machines are almost never idle. >=20 > I strongly suggest that before designing a new system, some careful > attention be paid to how extensive and optimized the current system alrea= dy > is. I don't think the people following this thread are going to get the > right perspective on the scope of this work, otherwise. >=20 To clarify: this rewrite of the current ports system does not include (at least for the moment) a replacement scripts/program for the build machines. I didn't say the build scripts could be optimized, I just said that if the system is improved in general less time would be lost in INDEX generation, dependency gathering (well, AFAIK portbuild takes it from INDEX so it won't be different), package registration/removal, etc. a little speed would be gained. Best Regards, Ale --Sig_/U17piTQKaLmJxY11EUXwX2q Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHZzZ1iV05EpRcP2ERAiDjAJ9vLkDTPjaV7dIB6A10YMQ6u5OBZwCfVXF4 OqdFYUUE6CAldyapxD3U/a8= =hmkH -----END PGP SIGNATURE----- --Sig_/U17piTQKaLmJxY11EUXwX2q--