From owner-freebsd-ports@FreeBSD.ORG Sat Sep 30 18:46:37 2006 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A12AF16A412 for ; Sat, 30 Sep 2006 18:46:37 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 627E843D49 for ; Sat, 30 Sep 2006 18:46:37 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 491BA1A3C19; Sat, 30 Sep 2006 11:46:37 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id C324551567; Sat, 30 Sep 2006 14:46:36 -0400 (EDT) Date: Sat, 30 Sep 2006 14:46:36 -0400 From: Kris Kennaway To: Michel Talon Message-ID: <20060930184636.GA76783@xor.obsecurity.org> References: <20060930134951.GA89865@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cWoXeonUoKmBZSoM" Content-Disposition: inline In-Reply-To: <20060930134951.GA89865@lpthe.jussieu.fr> User-Agent: Mutt/1.4.2.2i Cc: ports@freebsd.org Subject: Re: Yet another INDEX builder. X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2006 18:46:37 -0000 --cWoXeonUoKmBZSoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 30, 2006 at 03:49:51PM +0200, Michel Talon wrote: > Kris Kennaway said: >=20 > > FYI, I'm not sure if the python version is parallelizable, but you do > > get a small benefit from using parallelized 'make index' builds (via > > INDEX_JOBS) on a typical SMP machine. >=20 > My script runs parallel makes, of the order of ten simultaneous makes. OK, FYI I see a performance drop if you use too much parallelism (10 is "far too much" on a typical system; ~3 is probably optimal for a HTT machine). > You can see them, running top at the same time, but of course they are ve= ry > transient, since usually one make lasts of the order of 0.1 s on my machi= ne. > Anyways, still on my machine i have gained of the order of 20% wall clock= time > by doing that. And it is only an hyperthreaded P4 (running in SMP mode). = This > is why i was curious to know the effect of this idea on a machine with re= al > SMP and perhaps disks supporting tagged queuing, which should allow better > throughput for these simultaneous makes. I had, perhaps naively, assumed = that > with a good machine one can divide time by 2. The only situation where you might reasonably expect to see something approaching a factor of 2 speed improvement is if your ports tree is on a striped or mirrored device on separate controllers, so 2 reads can be completely decoupled from each other in hardware. I've not tried this though. Kris --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFHruMWry0BWjoQKURAuv/AKD0L+qcyOV1VTINQXolKXsvau56PQCglije WrNHv5UEtgCOQapobtg0imY= =gFHc -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM--