From owner-freebsd-ports@FreeBSD.ORG Sat Sep 30 13:39:02 2006 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8623416A417 for ; Sat, 30 Sep 2006 13:39:02 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id D243043D53 for ; Sat, 30 Sep 2006 13:39:01 +0000 (GMT) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.13.7/jtpda-5.4) with ESMTP id k8UDd0lj083446 for ; Sat, 30 Sep 2006 15:39:00 +0200 (CEST) X-Ids: 168 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 386F2A008A for ; Sat, 30 Sep 2006 15:38:59 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id CE53335; Sat, 30 Sep 2006 15:39:13 +0200 (CEST) Date: Sat, 30 Sep 2006 15:39:13 +0200 From: Michel Talon To: ports@freebsd.org Message-ID: <20060930133913.GA89767@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (shiva.jussieu.fr [134.157.0.168]); Sat, 30 Sep 2006 15:39:00 +0200 (CEST) X-Virus-Scanned: ClamAV 0.88.2/1951/Sat Sep 30 12:22:17 2006 on shiva.jussieu.fr X-Virus-Status: Clean X-Miltered: at shiva.jussieu.fr with ID 451E7374.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! Cc: Subject: Re: Yet another INDEX builder. X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2006 13:39:02 -0000 > It took longer on my P4 3.6GHz: > > The tree (15689ports) takes 2789.84287214 seconds to explore. > Now computing the recursively extended dependencies. > Took 3.02815794945 seconds. > Now converting to packages and sorting. > Last phase takes 4.7659368515 seconds. > Total time spent: 2800.94901609 seconds. > python build_index.py 3024,42s user 2050,37s system 181% cpu 46:41,64 total This time 46 minutes is compatible with the timing 2800 seconds. My ports tree has less ports that you own niobe% wc -l /usr/ports/INDEX-6 14186 /usr/ports/INDEX-6 As you can see i am not a frequent updater :-(, which says something about the ease of use of the present system, and more positively about the growth rate of the ports tree. Also perhaps my disk is faster niobe% diskinfo -t /dev/ad0 ... Transfer rates: outside: 102400 kbytes in 2.092096 sec = 48946 kbytes/sec middle: 102400 kbytes in 2.429019 sec = 42157 kbytes/sec inside: 102400 kbytes in 3.887401 sec = 26342 kbytes/sec - this is not a benchmark, and I was using the machine in the same time; I was doing that under KDE myself. - I have several local ports or local tweaks which have been reported as "The port xxx/yyy is obsolete". This is intentional. Since ports are frequently deleted but this deletion is not always effective on you own port tree (for example because there is a remaining README.html here) i compare subdirectories of a top directory to what is in its Makefile, listed as SUBDIR. If a directory exists but is not listed in SUBDIR i mark it obsolete, because i used to think it was a directory which should have been deleted. However i have seen in my port tree such situations, which are obviously strange and need human intervention. I have no idea what is the correct way to resolve these problems, at least thay are flagged. I have also noticed that a number of dependencies are incorrectly presented with trailing / and \n when you run make -V, and i was outputting them, but when i ran the script on the full port tree, there were so many of them, that i censored all that stuff by stripping white space and / at the end of origins. There is however certainly a problem in the way such things are computed. Let the function "clean_path" return "return path.replace(portdir, '', 1)" and you will see them. If you have kept the safe.pkl you can read it and just run everything after "safe = open('safe.pkl','r')", it will run in something like 5 seconds. -- Michel TALON