Date: Fri, 23 Jul 2010 18:29:32 -0400 From: jhell <jhell@dataix.net> To: Doug Barton <dougb@freebsd.org> Cc: freebsd-ports@freebsd.org Subject: Re: CFT: New version of portmaster in svn Message-ID: <4C4A17CC.8020707@dataix.net> In-Reply-To: <alpine.BSF.2.00.1007231145060.1713@qbhto.arg> References: <201007231421.o6NELsGa022354@monk.cnd.dundas.on.ca> <alpine.BSF.2.00.1007231145060.1713@qbhto.arg>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/23/2010 15:00, Doug Barton wrote:
> The problem is that in order to accomplish that portmaster would have to
> check every port in the tree. Assuming 22,000 ports, and that any given
> port is equally likely to fall anywhere in the tree, on average you'd
> have to search 11,000 ports for every distfile that is not related to a
> port installed on that system.
>
> Now what I _could_ do is use the same technique I use in
> --clean-distfiles (create a text file with the distfile information) but
> instead of limiting it to installed ports, do all of them. I have never
> even tested that to see how long it would take, but I suppose I could
> take a look.
Hi Doug Barton, Ports@,
Here is a suggestion that I have just played around with involving
locate(1) with environment LOCATE_PATH set in portmaster & ports-side
locate.updatedb(8) with LOCATE_CONFIG before the port snapshots are made.
Server side: ( at port snapshot creation )
LOCATE_CONFIG=/usr/ports/locate.rc /usr/libexec/locate.updatedb
locate.rc:
FCODES="/usr/ports/locate.database"
SEARCHPATHS="/usr/ports"
real 149.79
user 3.51
sys 27.53
2076 maximum resident set size
37 average shared memory size
3946 average unshared data size
134 average unshared stack size
2893 page reclaims
335 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
35875 voluntary context switches
38020 involuntary context switches
In portmaster(8):
LOCATE_PATH=/usr/ports/locate.database
Generate a list of distinfo files and create a distfile index using
``SIZE''.
for file in $(locate "/usr/ports/*/*/distinfo"); do
grep "SIZE " $file >>DISTINDEX.8
done
Use the output from that to judge if the file in /usr/ports/distfiles
needs to exist or of the file does not meet the SIZE requirements & you
can kill two birds with one stone.
You could always use the MD5 or SHA256 sum that's recorded but that may
be IMO a slight bit of overhead.
This may not be an exact proccess right now but it sure would be nice to
have a locate database in ports that could be updated by (make
locatedb) or something similar.
Just for creating the list of distfiles from the SIZE contents:
$ du -sh DISTINDEX.8.1.gz
512B DISTINDEX.8.1.gz
time -alhp ./createindex.sh
real 490.99
user 28.03
sys 117.14
1820 maximum resident set size
228 average shared memory size
2297 average unshared data size
133 average unshared stack size
2353331 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
77884 voluntary context switches
165392 involuntary context switches
Regards,
--
jhell,v
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C4A17CC.8020707>