From owner-freebsd-ports@FreeBSD.ORG Sun Dec 2 08:26:11 2012 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7433725D for ; Sun, 2 Dec 2012 08:26:11 +0000 (UTC) (envelope-from itetcu@FreeBSD.org) Received: from mx.tetcu.info (mx.tetcu.info [217.19.15.179]) by mx1.freebsd.org (Postfix) with ESMTP id 2237B8FC08 for ; Sun, 2 Dec 2012 08:26:10 +0000 (UTC) Received: from it.tim.tetcu.info (unknown [188.27.98.5]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx.tetcu.info (Postfix) with ESMTPSA id A527528E6FD; Sun, 2 Dec 2012 10:26:09 +0200 (EET) Date: Sun, 2 Dec 2012 10:26:08 +0200 From: Ion-Mihai Tetcu To: Matthew Seaman Subject: Re: web FreeBSD Ports Search is out of sync Message-ID: <20121202102608.0b091887@it.tim.tetcu.info> In-Reply-To: <50BB0D7D.6030406@infracaninophile.co.uk> References: <50BA347D.9080301@quip.cz> <50BB0D7D.6030406@infracaninophile.co.uk> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Kimmo Paasiala , Kevin Oberman , Miroslav Lachman <000.fbsd@quip.cz>, freebsd-ports@freebsd.org, Eitan Adler X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2012 08:26:11 -0000 On Sun, 02 Dec 2012 08:12:45 +0000 Matthew Seaman wrote: > On 02/12/2012 06:50, Kimmo Paasiala wrote: > > It takes about half an hour on my dog slow intel atom D510. Is there > > anything in the works to speed up 'make index'? I understand that > > the slowest part is the evaluation of dependencies, is that right? > > The slow part is parsing around 25,000 Makefiles ( + reading about > 25000 pkg-descr files, but that's relatively quick) in order to > extract the values of certain variables. make(1) typically takes > about 200ms per Makefile, which is ages. Speedups are possible: > parallelization (which is what make index does) or cacheing the > results and only rerunning make when changes could affect the output > (which is what FreeBSD::Portindex does) > > Once you've done all that Makefile processing, chasing all the > dependencies is fairly quick. > > A great deal of time and effort has been spent on making 'make index' > run as fast as possible. All the easy optimizations and many of the > not-so-easy ones have already been applied. Further improvement would > probably require a radical 'throw it away and write something better > from scratch' approach, which I'm not at all sure is worth it since > for the most part, the ports works just fine without any sort of > index at all, and it is the various websites that essentially show > nicely formatted data from the INDEX that would really benefit. BTW, I'm seeing this on my machines: Checking timestamps on options makefiles: cache-update: WARNING unknown options file "/var/db/ports/rsync/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/subversion/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/apr/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/python27/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/m4/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/help2man/options" -- ignored cache-update:1: Checking for category changes at www .... # cat /var/db/ports/rsync/options # This file is auto-generated by 'make config'. # Options for rsync-3.0.9_2 _OPTIONS_READ=rsync-3.0.9_2 _FILE_COMPLETE_OPTIONS_LIST=ACL ATIMES FLAGS ICONV POPT_PORT RENAMED SSH TIMELIMIT OPTIONS_FILE_UNSET+=ACL OPTIONS_FILE_UNSET+=ATIMES OPTIONS_FILE_UNSET+=FLAGS OPTIONS_FILE_UNSET+=ICONV OPTIONS_FILE_UNSET+=POPT_PORT OPTIONS_FILE_SET+=RENAMED OPTIONS_FILE_SET+=SSH OPTIONS_FILE_UNSET+=TIMELIMIT Any hints? Is it the fact that it's in the OPTIONSng format? Thanks, -- IOnut - Un^d^dregistered ;) FreeBSD "user" "Intellectual Property" is nowhere near as valuable as "Intellect" FreeBSD committer -> itetcu@FreeBSD.org, PGP Key ID F0808380