From owner-freebsd-ports@FreeBSD.ORG Fri Sep 29 21:04:56 2006 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AED516A403 for ; Fri, 29 Sep 2006 21:04:56 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id A6EF643D6A for ; Fri, 29 Sep 2006 21:04:55 +0000 (GMT) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.13.7/jtpda-5.4) with ESMTP id k8TL4r7X026953 ; Fri, 29 Sep 2006 23:04:53 +0200 (CEST) X-Ids: 166 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 783D69FC31; Fri, 29 Sep 2006 23:04:52 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id 4373835; Fri, 29 Sep 2006 23:05:07 +0200 (CEST) Date: Fri, 29 Sep 2006 23:05:07 +0200 From: Michel Talon To: Peter Jeremy Message-ID: <20060929210507.GA81520@lpthe.jussieu.fr> References: <20060929172122.GA80952@lpthe.jussieu.fr> <20060929201426.GG983@turion.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060929201426.GG983@turion.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.1i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (shiva.jussieu.fr [134.157.0.166]); Fri, 29 Sep 2006 23:04:53 +0200 (CEST) X-Virus-Scanned: ClamAV 0.88.2/1950/Thu Sep 28 16:11:54 2006 on shiva.jussieu.fr X-Virus-Status: Clean X-Miltered: at shiva.jussieu.fr with ID 451D8A75.003 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! Cc: ports@freebsd.org Subject: Re: Yet another INDEX builder. X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 21:04:56 -0000 On Sat, Sep 30, 2006 at 06:14:26AM +1000, Peter Jeremy wrote: > On Fri, 2006-Sep-29 19:21:22 +0200, Michel Talon wrote: > >timings (*) one gets. On my machine, a P4 3Ghz, oldish, it takes 23 minutes to > >complete building the INDEX, but the python script doesn't run more than 1 > >minute. Everything else is IO, i think. So one cannot expect any performance > >improvement by writing such a program in C. > > IMHO, the advantage of using C is that it would remove the need for > yet another interpreter to be installed to manage my ports. Or, to > put is another way, by writing it in python, you have made your script > less attractive to me than it would be if it was in C/sh/awk. > I completely agree with you on that. I see python here as a prototyping tool to be able to write quickly something that works, so that people may easily understand it, and hack it if they want. Assuming, and we are very far from that, this tool seems useful, then it may be time to think about rewriting it in C. Writing directly in C, without knowing if it has any real interest would be a loss of time. Writing in python has many advantages: - it is very easy to read - it can be readily translated in C assuming availability of libraries helping to simplify the things that python does very simply. I am here thinking of the boost libraries, but you will object this is still another port. - there are no high brain constructions such as in ruby and frequently in perl, so the way to translate the logic to C is completely direct. - python and perl are performant, ruby is dog slow. - if you have some big package installed such as Gnome or KDE you have python installed as a dependency. You have perl installed as soon as you install the most innocent ports, even xorg does that if i remember well. - finally python comes with a complete set of tools, so i don't have used any supplementary port besides python. No database connector, nothing. This is knowing the problems which afflicted portupgrade when you upgrade ruby or the database connector. Python is a stable language, scripts that work will not break after an upgrade. Coredumps you don't see. > Several years ago, the Project went to a lot of effort to re-writes > bits of the base system in C/sh/awk to allow perl to be removed. > Ports-based maintenance tools are (of course) free to use any language > but the proliferation of different tools using different languages > means that I wind up with a large number of languages and interpreters > installed just to easily manage my system: > - CVSup needs Modula-2 Not really, only to build cvsup, not to run it. With a precompiled package you dont need Modula-2. But i am precisely advocating the idea that one should have a reliable and easy way to upgrade using precompiled packages, and ports only as a fallback, or in case one wants to tweak something. As soon as you are using ports, speed or efficiency doesn't matter any more, since you will obviously spend an enormous of time compiling, making the time spent in the tool negligible. Hence a tool like portmaster will be amply sufficient to fill this need. It is when you try to upgrade with precompiled packages that you discover that portupgrade is intolerably slow. > - portupgrade needs Ruby > - your new index builder needs Python > - various other ports management tools use perl Almost any use of the ports system uses perl. There are several perl invocations in bsd.port.mk. For example in make describe. The idea of my script is to remove all tools forked by make index, innumerable copies of perl, sed, and whatever else forked at any instant, by just one python program doing the job, and just the minimal number of forks, one fork of make -V in each port directory. Obviously this alone fork consumes almost all the run time of the program. > > Please don't take the above as a personal attack - I'm merely pointing > out that there are other reasons apart from performance for using C. Once more i agree with you. > > -- > Peter Jeremy -- Michel TALON