From owner-freebsd-ports@FreeBSD.ORG Fri Sep 29 22:38:48 2006 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E13C16A40F for ; Fri, 29 Sep 2006 22:38:48 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1141943D7B for ; Fri, 29 Sep 2006 22:38:45 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id CA0C61A4DA1 for ; Fri, 29 Sep 2006 15:38:45 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 4E81C514F7; Fri, 29 Sep 2006 18:38:45 -0400 (EDT) Date: Fri, 29 Sep 2006 18:38:45 -0400 From: Kris Kennaway To: ports@freebsd.org Message-ID: <20060929223845.GA38759@xor.obsecurity.org> References: <20060929172122.GA80952@lpthe.jussieu.fr> <20060929222519.GB63185@graf.pompo.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DocE+STaALJfprDB" Content-Disposition: inline In-Reply-To: <20060929222519.GB63185@graf.pompo.net> User-Agent: Mutt/1.4.2.2i Cc: Subject: Re: Yet another INDEX builder. X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 22:38:48 -0000 --DocE+STaALJfprDB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 30, 2006 at 12:25:19AM +0200, Thierry Thomas wrote: > Le Ven 29 sep 06 ? 19:21:22 +0200, Michel Talon > ?crivait?: > > Hello, >=20 > Hi, >=20 > > i have written yet another index builder in python, which can be found = here: > > http://www.lpthe.jussieu.fr/~talon/build_index.py > > I think it works relatively well and fast, and can provide building blo= cks for > > doing a package updater alternative to portupgrade. Of course, like the > > documentation, it is completely free. I would be happy if some people c= ould > > test it on a real biproc machine, or a machine with fast disks, to see = the > > timings (*) one gets. On my machine, a P4 3Ghz, oldish, it takes 23 min= utes to > > complete building the INDEX, but the python script doesn't run more tha= n 1 > > minute. Everything else is IO, i think. So one cannot expect any perfor= mance >=20 > It took longer on my P4 3.6GHz: >=20 > The tree (15689ports) takes 2789.84287214 seconds to explore. > Now computing the recursively extended dependencies. > Took 3.02815794945 seconds. > Now converting to packages and sorting. > Last phase takes 4.7659368515 seconds. > Total time spent: 2800.94901609 seconds. > python build_index.py 3024,42s user 2050,37s system 181% cpu 46:41,64 to= tal >=20 > Remarks: > - this is not a benchmark, and I was using the machine in the same time; > - I have several local ports or local tweaks which have been reported as > "The port xxx/yyy is obsolete". As Michel mentions, INDEX build is almost completely I/O bound, so it almost entirely depends on the speed of your disk and how congested it is from other I/O. Thus it's not really meaningful to compare numbers from one machine to another, but it still may be useful to compare to 'make index' times. You should not expect a significant difference unless something is going wrong though. FYI, I'm not sure if the python version is parallelizable, but you do get a small benefit from using parallelized 'make index' builds (via INDEX_JOBS) on a typical SMP machine. Kris --DocE+STaALJfprDB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFHaB0Wry0BWjoQKURAjWKAJ9vl/LtfBi88+nHat6l/yra0/WWiwCgwN6E PBfhpqZAPiyvqdgWpsfCQdc= =wdwt -----END PGP SIGNATURE----- --DocE+STaALJfprDB--