Date: Thu, 21 Feb 2002 19:10:17 -0300 From: Mario Sergio Fujikawa Ferreira <lioux@FreeBSD.org> To: portmgr@FreeBSD.org Cc: tkato@prontomail.com, Pete Fritchman <petef@databits.net>, Patrick Li <pat@databits.net>, Anton Berezin <tobez@tobez.org>, FreeBSD-ports@FreeBSD.org Subject: bsd.port.mk MASTER_SITES_NN misbehavior (was ghostscript-afpt and MASTER_SITES_NN patch) Message-ID: <20020221221039.19324.qmail@exxodus.fedaykin.here>
next in thread | raw e-mail | index | archive | help
--1LKvkjL3sHcu1TtY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Hi,
I was writing MASTER_SITES_NN patches for ports/mail/qmail;
thus, allowing it to save download time by only trying the correct
master sites for each distfile.
However, I stumbled in a weird behavior. Sites were
being replicated, I was getting several occurences of the same site
instead of expected one.
One example is better. Apply attached patch-mail:qmail:Makefile
to ports/mail/qmail port. Then try the following test. Suppose the
only file you are missing is qmail-103.patch For the sake of testing,
do a make fetch then move qmail-103.patch somewhere. Then, try the
following
$ make fetch-list | tr "||" "\012"
$ make fetch-list | tr "||" "\012" |sort|uniq
$ make fetch-list | tr "||" "\012" |wc
389 974 17593
$ make fetch-list | tr "||" "\012" |sort|uniq|wc
35 169 2962
Hummm, they should all return the same number of elements.
There are 2 issues here:
1) The MASTER_FILTER_PERL idea needs work. This idea
can be found in attached patch patch-bsd.port.mk;
2) For some reason, my MASTER_SITES_NN patch is being
recursive and adding multiple instances of the same
variables.
At first, I want to focus in (1) but I would appreciate help on
(2).
The MASTER_FILTER_PERL idea was discussed with Anton Berezin
(project tobez :). After constructing any variables, we use it to
remove ANY repeated instances since repetition means bogus instance.
It is a neat thing, not only it "hides" the MASTER_SITES_NN problem
but also add a level of protection against poorly written MASTER_SITES.
Let me know what do you think.
Regards,
--
Mario S F Ferreira - DF - Brazil - "I guess this is a signature."
Computer Science Undergraduate | FreeBSD Committer | CS Developer
flames to beloved devnull@someotherworldbeloworabove.org
feature, n: a documented bug | bug, n: an undocumented feature
--1LKvkjL3sHcu1TtY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=patch-mail:qmail:Makefile
--- Makefile.orig Thu Jan 24 09:53:59 2002
+++ Makefile Thu Jan 24 18:12:37 2002
@@ -24,8 +24,8 @@
# Patch necessary to cope with non-RFC >512 dns entries
# Since AOL has been using those, the problem has skyrocketed from minor to
# groundzero. qmail being RFC compliant need to be "fixed" to work with those
-PATCH_SITES+= http://www.ckdhr.com/ckd/
-PATCHFILES+= qmail-103.patch
+PATCH_SITES+= http://www.ckdhr.com/ckd/:dns
+PATCHFILES+= qmail-103.patch:dns,local
PATCH_DIST_STRIP+= -p1
.endif # MAIN_QMAIL_PORT_DNS_PATCH_BARRIER
@@ -61,8 +61,8 @@
||(defined(WITH_BIG_TODO_PATCH)&&!defined(MAIN_QMAIL_PORT_WITH_BIG_TODO_PATCH_BARRIER)) \
||(defined(WITH_BIG_CONCURRENCY_PATCH)&&!defined(MAIN_QMAIL_PORT_WITH_BIG_CONCURRENCY_PATCH_BARRIER))
PATCH_SITES+= ${MASTER_SITE_QMAIL} \
- ${MASTER_SITE_LOCAL}
-PATCH_SITE_SUBDIR+= lioux
+ ${MASTER_SITE_LOCAL:S/$/:DEFAULT,local/}
+PATCH_SITE_SUBDIR+= lioux/:DEFAULT,local
.endif
# qmail-* skeleton ports including this port may not want
--1LKvkjL3sHcu1TtY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="patch-bsd.port.mk"
--- bsd.port.mk.orig Thu Jan 24 10:44:29 2002
+++ bsd.port.mk Thu Jan 24 18:18:44 2002
@@ -1449,6 +1449,12 @@
MASTER_SORT?=
MASTER_SORT_REGEX?=
MASTER_SORT_REGEX+= ${MASTER_SORT:S|.|\\.|g:S|^|//[^/]*|:S|$|/|}
+#MASTER_SORT_PERL= ${PERL} -lpe 'chop; foreach (split/ /) { if (!$$elements{$$_}) { $$elements{$$_} = "." ; print "$$_ "} }'
+#MASTER_SORT_PERL= ${PERL} -lpe '1 while s|(\b\S+\b)(.*?)\1|$$1$$2|'
+#MASTER_SORT_PERL= ${PERL} -pe '1 while s| (\S+) (.*?)\1| $$1 $$2|'
+#MASTER_SORT_PERL= ${PERL} -pe '1 while s!(?:^|\s+)(\S+)\s+(.*?)\1! $$1 $$2!'
+MASTER_FILTER_PERL= -lne 'for (split) { print unless $$_{$$_}++ }'
+#MASTER_FILTER_PERL= -lne 'for (split) { print }'
MASTER_SORT_AWK= BEGIN { RS = " "; ORS = " "; IGNORECASE = 1 ; gl = "${MASTER_SORT_REGEX}"; }
.for srt in ${MASTER_SORT_REGEX}
@@ -1472,7 +1478,7 @@
. if !target(master-sites-${_group})
SORTED_MASTER_SITES_${_group}_CMD= cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} master-sites-${_group}
master-sites-${_group}:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_${_group}}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_${_group}}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` | ${_MASTER_SITE_BACKUP}
. endif
. endfor
. endif
@@ -1484,7 +1490,7 @@
. if !target(patch-sites-${_group})
SORTED_PATCH_SITES_${_group}_CMD= cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} patch-sites-${_group}
patch-sites-${_group}:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_${_group}}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_${_group}}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
. endif
. endfor
. endif
@@ -1517,14 +1523,14 @@
.endfor
master-sites-ALL:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_ALL}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_ALL}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
patch-sites-ALL:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_ALL}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_ALL}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
# has similar effect to old targets, i.e., access only {MASTER,PATCH}_SITES, not working with the new _n variables
master-sites-DEFAULT:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_DEFAULT}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_MASTER_SITES_DEFAULT}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
patch-sites-DEFAULT:
- @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_DEFAULT}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
+ @echo ${_MASTER_SITE_OVERRIDE} `echo '${_PATCH_SITES_DEFAULT}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
# synonyms, mnemonics
master-sites-all: master-sites-ALL
@@ -2030,7 +2036,7 @@
fi \
done; \
___MASTER_SITES_TMP= ; \
- SORTED_MASTER_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__MASTER_SITES_TMP}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
+ SORTED_MASTER_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__MASTER_SITES_TMP}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
else \
SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \
fi ; \
@@ -2075,7 +2081,7 @@
fi \
done; \
___PATCH_SITES_TMP= ; \
- SORTED_PATCH_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__PATCH_SITES_TMP}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
+ SORTED_PATCH_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__PATCH_SITES_TMP}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
else \
SORTED_PATCH_SITES_CMD_TMP="${SORTED_PATCH_SITES_DEFAULT_CMD}" ; \
fi ; \
@@ -2728,7 +2734,7 @@
fi \
done; \
___MASTER_SITES_TMP= ; \
- SORTED_MASTER_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__MASTER_SITES_TMP}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
+ SORTED_MASTER_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__MASTER_SITES_TMP}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
else \
SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \
fi ; \
@@ -2760,7 +2766,7 @@
fi \
done; \
___PATCH_SITES_TMP= ; \
- SORTED_PATCH_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__PATCH_SITES_TMP}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
+ SORTED_PATCH_SITES_CMD_TMP="echo ${_MASTER_SITE_OVERRIDE} `echo '$${__PATCH_SITES_TMP}' | ${PERL} ${MASTER_FILTER_PERL} | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}" ; \
else \
SORTED_PATCH_SITES_CMD_TMP="${SORTED_PATCH_SITES_DEFAULT_CMD}" ; \
fi ; \
--1LKvkjL3sHcu1TtY--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020221221039.19324.qmail>