Date: Mon, 1 Oct 2001 03:00:44 -0300 From: "Mario Sergio Fujikawa Ferreira" <lioux@uol.com.br> To: freebsd-ports@FreeBSD.org Cc: portmgr@FreeBSD.org Subject: review final MASTER_SITES with grouping (ala OpenBSD) patch for bsd.port.mk Message-ID: <20011001030044.A1185@exxodus.fedaykin.here>
next in thread | raw e-mail | index | archive | help
Hi,
With no further ado, this is the final proposal.
First, a little background for first timers.
------
OpenBSD has a neat feature, inside the DISTFILES area, files
can be postfixed with :n where n can be [0-9] and denotes a group.
For example:
DISTFILES= file1:0 file2:1
Distfile file1 will be associated with variable MASTER_SITES0
instead of our common MASTER_SITES and files2 with MASTER_SITES1
This is a very neat feature which can decrease that endless
search for the correct download site.
Just picture 2 DISTFILES and 20 MASTER_SITES, the sites are
slow as hell and (of course); while file2 is carried by all
MASTER_SITES, file1 is only carried by MASTER_SITE number 20. Seems
a waste to check all of them if the maintainer knew beforehand,
does not it? Not a good start for that lovely weekend. :-)
Now that you got the picture, just imagine more distfiles
and more master_sites. I am sure that our "distfiles survey meister"
would appreciate the network strain relieve this would bring. :)
------
After debating this for some time and collecting suggestions,
here goes our version of this idea:
1) Elements can be postfixed with :n where n is [^:,]+ n
can be any alphanumerical string but we will limit it to
[0-9a-zA-Z_]+ for now. We are not limited to a single
numerical [0-9] as OpenBSD is. Also, n should be
neither DEFAULT nor all words (since it is used internally in (3)
and (C2***) respectively below), they are reserved words
2) Elements postfixed with :n belong to the group n, with
:m belong to group m and so forth
3) Elements without a postfix are groupless; that is, they
all do belong to the special group DEFAULT
4) Groups are not exclusive, an element can belong to
several different groups at the same time and a group can
have several different elements. Repeated elements within
the same group will be simply that, repeated elements
5) The comma ',' operator; This one is tempting, what if
we want an element belonging to several groups at the same
time? Instead of repeating it several times, each time with
a different postfix, we could list several groups at once
in a single postfix. For instance, :m,n,o meaning an element
belongs to group m, n and o
6) Group semantics can be used in any of the following
variables {MASTER,PATCH}{FILES,_SITE_SUBDIR,_SITES} with the
following syntax:
6.1) all {MASTER,PATCH}_{SITE_SUBDIR,SITES} elements
must be terminated with the character slash '/'.
If any elements belong to any groups, the group
postfix :n will come right after the terminator
'/'. The patch will rely on the existence of this
terminator '/' to avoid confusing elements where
a :n is a valid part of the element with occurences
where :n denotes group n. For compatibility purposes,
since the '/' terminator was not required before
in {MASTER,PATCH}_SITE_SUBDIR elements, even if an
element is postfixed with :n, if the postfix
immediate preceeding character isn't a '/' then :n
will be considered a valid part of the element
instead of a group postfix
MASTER_SITE_SUBDIR= old:n new/:NEW
directories within group DEFAULT -> old:n
directories within group NEW -> new
MASTER_SITES= http://site1/%SUBDIR%/ http://site2/:DEFAULT \
http://site3/:group3 http://site4/:group4 \
http://site5/:group5 http://site6/:group6 \
http://site7/:DEFAULT,group6 \
http://site8/%SUBDIR%/:group6,group7 \
http://site9/:group8
DISTFILES= file1 file2:DEFAULT file3:group3 \
file4:group4,group5,group6 file5:grouping \
file6:group7
MASTER_SITE_SUBDIR= directory-trial:1 directory-n/:groupn \
directory-one/:group6,DEFAULT \
directory
- results:
o file1 will be fetched from
${_MASTER_SITE_OVERRIDE}
http://site1/directory/
http://site1/directory-one/
http://site1/directory-trial:1/
http://site2/
http://site7/
${_MASTER_SITE_BACKUP}
o file2 will be fetched from same as file1 since they
belong to the same group
${_MASTER_SITE_OVERRIDE}
http://site1/directory/
http://site1/directory-one/
http://site1/directory-trial:1/
http://site2/
http://site7/
${_MASTER_SITE_BACKUP}
o file3 will be fetched from
${_MASTER_SITE_OVERRIDE}
http://site3/
${_MASTER_SITE_BACKUP}
o file4 will be fetched from
${_MASTER_SITE_OVERRIDE}
http://site4/
http://site5/
http://site6/
http://site7/
http://site8/directory-one/
${_MASTER_SITE_BACKUP}
o file5 will be fetched from
${_MASTER_SITE_OVERRIDE}
${_MASTER_SITE_BACKUP}
o file6 will be fetched from
${_MASTER_SITE_OVERRIDE}
http://site8/directory-one/
${_MASTER_SITE_BACKUP}
7) How do I group one of the special variables from
bsd.sites.mk? Example
MASTER_SITES= http://site1/ ${MASTER_SITE_SOURCEFORGE:S/$/:sourceforge,TEST/}
DISTFILES= something.tar.gz:sourceforge
- something.tar.gz will be fetched from all sites within
${MASTER_SITE_SOURCEFORGE}
8) How do I use this with PATCH* variables? All examples
were done with MASTER* but they are the same for PATCH*
PATCH_SITES= http://site1/ http://site2/:test
PATCHFILES= patch1:test
9) All sites within a given group are sorted according to
${MASTER_SORT_AWK}. All groups within {MASTER,PATCH}SITES
are sorted
------
What changes for current working ports? What doesn't?
C1) All current ports remain the same. The feature code is
only activated if there are elements postfixed with :n given
the aforementioned syntax rules, specially (6)
C2) The port targets remain the same: checksum, makesum,
patch, configure, build, ... All of them, with the
obvious exceptions of do-fetch, fetch-list, master-sites
and patch-sites
- do-fetch: deploys the new grouping postfixed {DIST,PATCH}FILES
with their matching group elements within {MASTER,PATCH}_SITES
which use matching group elements within {MASTER,PATCH}_SITE_SUBDIR
Check (6.1) for an example
- fetch-list: works like old fetch-list with the
exception that it groups just like do-fetch
- {master,patch}-sites: (incompatible with older versions)
only returns the elements of group DEFAULT, in fact, it
executes targets {master,patch}-sites-default respectively
* using target {master,patch}-sites is preferred to directly
checking {MASTER,PATCH}_SITES. Also, directly checking is
not guaranteed to work in any future versions
** (NEW) there are {master,patch}-sites-n targets which
will list the elements of the respective group n within
{MASTER,PATCH}_SITES. {master,patch}-sites-DEFAULT will
return the elements of group DEFAULT
*** (NEW) there are new targets {master,patch}-sites-all
which do the work of the old {master,patch}-sites. Return
the elements of all groups as if they all belonged to the
same group with the caveat that it lists as many both
MASTER_SITE_BACKUP and MASTER_SITE_OVERRIDE as there are
groups defined within {DIST,PATCH}FILES
------
Please let me know if you agree with this and I'll start
writing code for it right away.
If I missed something, please correct me. This is important
since I hope this email will be the base of The Porter's Handbook
entry on this.
Regards,
--
Mario S F Ferreira - UnB - Brazil - "I guess this is a signature."
lioux at ( freebsd dot org | linf dot unb dot br )
flames to beloved devnull@someotherworldbeloworabove.org
feature, n: a documented bug | bug, n: an undocumented feature
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20011001030044.A1185>