Date: Sun, 2 Dec 2012 10:26:08 +0200 From: Ion-Mihai Tetcu <itetcu@FreeBSD.org> To: Matthew Seaman <m.seaman@infracaninophile.co.uk> Cc: Kimmo Paasiala <kpaasial@gmail.com>, Kevin Oberman <kob6558@gmail.com>, Miroslav Lachman <000.fbsd@quip.cz>, freebsd-ports@freebsd.org, Eitan Adler <lists@eitanadler.com> Subject: Re: web FreeBSD Ports Search is out of sync Message-ID: <20121202102608.0b091887@it.tim.tetcu.info> In-Reply-To: <50BB0D7D.6030406@infracaninophile.co.uk> References: <50BA347D.9080301@quip.cz> <CAF6rxgkhbOLN3ZZhxRZgYtQWF4yeHjiPS68YryKE-vAqitAvFg@mail.gmail.com> <CAN6yY1tqH857Ju2C8ADVPYG9VzhJYmY3Rya9fAvP=pcbVvvuRA@mail.gmail.com> <CA%2B7WWSeT-FTD=YBvbBcjhWWSWUbcMjfUiwtwaXYVPW=yZruo8w@mail.gmail.com> <50BB0D7D.6030406@infracaninophile.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 02 Dec 2012 08:12:45 +0000 Matthew Seaman <m.seaman@infracaninophile.co.uk> wrote: > On 02/12/2012 06:50, Kimmo Paasiala wrote: > > It takes about half an hour on my dog slow intel atom D510. Is there > > anything in the works to speed up 'make index'? I understand that > > the slowest part is the evaluation of dependencies, is that right? > > The slow part is parsing around 25,000 Makefiles ( + reading about > 25000 pkg-descr files, but that's relatively quick) in order to > extract the values of certain variables. make(1) typically takes > about 200ms per Makefile, which is ages. Speedups are possible: > parallelization (which is what make index does) or cacheing the > results and only rerunning make when changes could affect the output > (which is what FreeBSD::Portindex does) > > Once you've done all that Makefile processing, chasing all the > dependencies is fairly quick. > > A great deal of time and effort has been spent on making 'make index' > run as fast as possible. All the easy optimizations and many of the > not-so-easy ones have already been applied. Further improvement would > probably require a radical 'throw it away and write something better > from scratch' approach, which I'm not at all sure is worth it since > for the most part, the ports works just fine without any sort of > index at all, and it is the various websites that essentially show > nicely formatted data from the INDEX that would really benefit. BTW, I'm seeing this on my machines: Checking timestamps on options makefiles: cache-update: WARNING unknown options file "/var/db/ports/rsync/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/subversion/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/apr/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/python27/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/m4/options" -- ignored cache-update: WARNING unknown options file "/var/db/ports/help2man/options" -- ignored cache-update:1: Checking for category changes at www .... # cat /var/db/ports/rsync/options # This file is auto-generated by 'make config'. # Options for rsync-3.0.9_2 _OPTIONS_READ=rsync-3.0.9_2 _FILE_COMPLETE_OPTIONS_LIST=ACL ATIMES FLAGS ICONV POPT_PORT RENAMED SSH TIMELIMIT OPTIONS_FILE_UNSET+=ACL OPTIONS_FILE_UNSET+=ATIMES OPTIONS_FILE_UNSET+=FLAGS OPTIONS_FILE_UNSET+=ICONV OPTIONS_FILE_UNSET+=POPT_PORT OPTIONS_FILE_SET+=RENAMED OPTIONS_FILE_SET+=SSH OPTIONS_FILE_UNSET+=TIMELIMIT Any hints? Is it the fact that it's in the OPTIONSng format? Thanks, -- IOnut - Un^d^dregistered ;) FreeBSD "user" "Intellectual Property" is nowhere near as valuable as "Intellect" FreeBSD committer -> itetcu@FreeBSD.org, PGP Key ID F0808380
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121202102608.0b091887>