Date: Sat, 31 Aug 2013 13:22:02 +0000 (UTC) From: Bryan Drewery <bdrewery@FreeBSD.org> To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r325805 - head/Mk Message-ID: <201308311322.r7VDM2wa076581@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: bdrewery Date: Sat Aug 31 13:22:01 2013 New Revision: 325805 URL: http://svnweb.freebsd.org/changeset/ports/325805 Log: - make fetch/checksum: If a fetched file does not match the expected size, delete it and try the next site, if there is one to try. Normally fetch(1) will detect that the size is wrong, fail, and the 'fetch' target will move on to the next site: => Attempting to fetch http://www.chiark.greenend.org.uk/~sgtatham/agedu/agedu-r9723.tar.gz fetch: http://www.chiark.greenend.org.uk/~sgtatham/agedu/agedu-r9723.tar.gz: size mismatch: expected 155321, actual 163319 => Attempting to fetch http://ftp.FreeBSD.org/pub/FreeBSD/ports/local-distfiles/sunpoet/agedu-r9723.tar.gz agedu-r9723.tar.gz 100% of 151 kB 259 kBps However, if the remote server does not properly advertise the size of the file, fetch(1) will still download the file, and succeed. The 'fetch' target then stops, assuming it has a proper file. This is returned to 'checksum', which fails, and then tries a refetch, but on the same broken site. In this example, the size is expected to be 214984, but a 4830 size is fetched and checksummed. => Attempting to fetch http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz fetch: http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz: size unknown fetch: http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz: size of remote file is not known picpuz-2.1.1.tar.gz 4830 B 48 kBps ===> Fetching all distfiles required by picpuz-2.1.1_5 for building => SHA256 Checksum mismatch for picpuz-2.1.1.tar.gz. Now the 'fetch' target will verify the size is proper before returning to 'checksum': => Attempting to fetch http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz fetch: http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz: size unknown fetch: http://kornelix.squarespace.com/storage/downloads/picpuz-2.1.1.tar.gz: size of remote file is not known picpuz-2.1.1.tar.gz 4830 B 130 kBps => Fetched file size mismatch (expected 214984 actual 4830) => Trying next site => Attempting to fetch http://www.stasyan.com/devel/distfiles/picpuz-2.1.1.tar.gz picpuz-2.1.1.tar.gz 100% of 209 kB 20 kBps 00m00s ===> Fetching all distfiles required by picpuz-2.1.1_5 for building => SHA256 Checksum OK for picpuz-2.1.1.tar.gz. Reviewed by: bapt With hat: portmgr Modified: head/Mk/bsd.port.mk Modified: head/Mk/bsd.port.mk ============================================================================== --- head/Mk/bsd.port.mk Sat Aug 31 13:14:24 2013 (r325804) +++ head/Mk/bsd.port.mk Sat Aug 31 13:22:01 2013 (r325805) @@ -3413,7 +3413,13 @@ do-fetch: else \ SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \ fi; \ - for site in `eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \ + sites_remaining=0; \ + sites="`eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`"; \ + for site in $${sites}; do \ + sites_remaining=$$(($${sites_remaining} + 1)); \ + done; \ + for site in $${sites}; do \ + sites_remaining=$$(($${sites_remaining} - 1)); \ ${ECHO_MSG} "=> Attempting to fetch $${site}$${file}"; \ CKSIZE=`alg=SIZE; ${DISTINFO_DATA}`; \ case $${file} in \ @@ -3422,7 +3428,16 @@ do-fetch: *) args=$${site}$${file};; \ esac; \ if ${SETENV} ${FETCH_ENV} ${FETCH_CMD} ${FETCH_BEFORE_ARGS} $${args} ${FETCH_AFTER_ARGS}; then \ - continue 2; \ + actual_size=`stat -f %z "$${file}"`; \ + if [ $${actual_size} -eq $${CKSIZE} ]; then \ + continue 2; \ + else \ + ${ECHO_MSG} "=> Fetched file size mismatch (expected $${CKSIZE}, actual $${actual_size})"; \ + if [ $${sites_remaining} -gt 1 ]; then \ + ${ECHO_MSG} "=> Trying next site"; \ + ${RM} -f $${file}; \ + fi; \ + fi; \ fi; \ done; \ ${ECHO_MSG} "=> Couldn't fetch it - please try to retrieve this";\
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201308311322.r7VDM2wa076581>