From owner-freebsd-hackers@FreeBSD.ORG Sun May 13 12:26:02 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0AB7616A403 for ; Sun, 13 May 2007 12:26:02 +0000 (UTC) (envelope-from Thomas.Sparrevohn@btinternet.com) Received: from smtp805.mail.ird.yahoo.com (smtp805.mail.ird.yahoo.com [217.146.188.65]) by mx1.freebsd.org (Postfix) with SMTP id 8C2DA13C459 for ; Sun, 13 May 2007 12:26:01 +0000 (UTC) (envelope-from Thomas.Sparrevohn@btinternet.com) Received: (qmail 56519 invoked from network); 13 May 2007 11:59:18 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=btinternet.com; h=Received:X-YMail-OSG:From:To:Subject:Date:User-Agent:Cc:References:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-Disposition:Message-Id; b=Kv5Q30c7iS9IfLoaCs7mMmsFf8R2MiesFwOee0KozKa6ArG4n5593cp/w/kPA/oGCuxhmOMlxiEYZtpZZo5mQpj3baOY9LjpnkJerCnUAKTR4jQT2+fmbk20XLsP5NPdYsPhtnAwEpNBiQUU5wUY+Qfd/GH+9ILFTj/HrhyzKX8= ; Received: from unknown (HELO ?192.168.0.22?) (thomas.sparrevohn@btinternet.com@86.134.25.217 with login) by smtp805.mail.ird.yahoo.com with SMTP; 13 May 2007 11:59:18 -0000 X-YMail-OSG: R0eWN3cVM1lNrsXtPONWUe7v90Vw8SoRJK3G9lS9BBC1qGLNF524OT9EMO.cXXNV.A6n_NSsuw-- From: Thomas Sparrevohn To: freebsd-hackers@freebsd.org Date: Sun, 13 May 2007 12:58:49 +0100 User-Agent: KMail/1.9.6 References: <20070512004209.GA12218@lpthe.jussieu.fr> <20070512214422.GA88480@lpthe.jussieu.fr> <20070513103757.GA33322@turion.vk2pj.dyndns.org> In-Reply-To: <20070513103757.GA33322@turion.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200705131258.50309.Thomas.Sparrevohn@btinternet.com> Cc: Michel Talon Subject: Re: DPS Initial Ideas X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2007 12:26:02 -0000 On Sunday 13 May 2007 11:37:57 Peter Jeremy wrote: > > The options I can see are: > - Ignore the existence of INDEX - which makes computing dependencies > very time consuming > - Fully rebuild INDEX via "make describe" whenever you update any ports > - this takes of the order of an hour > - Find and rebuild the changed bits of INDEX - p5-FreeBSD-Portindex > uses this approach. > - Build a tool that functionally does "make describe" but does it in > bulk much faster (eg by pre-parsing the include files once instead > of 17000 times). > Having played around with using Postgres as a database for ports - I must stress that its not a database vs. flatfile issue - It is quite easy to build a reasonable "Ports database" - however it does not help on the issue - namely that dependencies and options means that it is needed to run make in order to gurantee that the INDEX file are correct It seems to be a non-debate what format the database is in if there not a good answer to how ensure that only ports that has changed are updated. At the end of the day - "make based ports" are the only real safe way to manage ports - However the focus on the indexing side seems misplaced - example - make INDEX on this host take 8-12 Minutes - compiling all ports installed takes 24 Hours - now if I "hand build" the dependencies structure and run the builds in parallel it takes down to 4-5 Hours - so lets say we half the time it takes to maintain the index - well - it cuts minimum time off the entire build process and the effort and energy proberly better spend on trying to define a build sequence that allows ports to build with "make -j x" and with parallel builds where "-j n" does not work Using XML for INDEX are a very good idea mainly because it allows "ports" to interface in an easy way to external tools - e.g. java frontends - web browsers etc, etc. However there are drawbacks - Yet I feel that the discussion about what tool to use as indexing are completely misplaced if the only point is that somebody likes SQL better than a directory tree. > >Yes, and i don't buy the idea that using *existing* tools is better than > >using the best tool for the job (assuming one can prove what is the best tool, > >considering power, familiarity, etc.). > Remind me - we are told that SQL are the answer but what was the question again? > Demonstrate a better tool. > Always the best way ;-)