From owner-freebsd-ports-bugs@FreeBSD.ORG Fri Mar 12 05:20:17 2004 Return-Path: Delivered-To: freebsd-ports-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8AEAC16A4D0 for ; Fri, 12 Mar 2004 05:20:17 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6A62543D53 for ; Fri, 12 Mar 2004 05:20:17 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) i2CDKHbv062395 for ; Fri, 12 Mar 2004 05:20:17 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i2CDKHPv062394; Fri, 12 Mar 2004 05:20:17 -0800 (PST) (envelope-from gnats) Resent-Date: Fri, 12 Mar 2004 05:20:17 -0800 (PST) Resent-Message-Id: <200403121320.i2CDKHPv062394@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-ports-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Oliver Eikemeier Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5997716A4CE for ; Fri, 12 Mar 2004 05:17:42 -0800 (PST) Received: from postman.arcor.de (newsread1.arcor-online.net [151.189.0.146]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6985E43D2F for ; Fri, 12 Mar 2004 05:17:41 -0800 (PST) (envelope-from eikemeier@fillmore-labs.com) Received: from fillmore.dyndns.org (port-212-202-51-138.reverse.qsc.de [212.202.51.138]) (authenticated bits=0)i2CDHdck023006 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Fri, 12 Mar 2004 14:17:39 +0100 (MET) Received: from [172.16.0.2] (helo=fillmore-labs.com) by fillmore.dyndns.org with esmtp (Exim 4.30; FreeBSD) id 1B1mXg-000OJg-42 for FreeBSD-gnats-submit@FreeBSD.org; Fri, 12 Mar 2004 14:17:36 +0100 Message-Id: <4051B86A.7050109@fillmore-labs.com> Date: Fri, 12 Mar 2004 14:17:30 +0100 From: Oliver Eikemeier To: FreeBSD-gnats-submit@FreeBSD.org Subject: ports/64157: [PATCH] bsd.port.mk: new terget check-distfiles X-BeenThere: freebsd-ports-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Ports bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2004 13:20:17 -0000 >Number: 64157 >Category: ports >Synopsis: [PATCH] bsd.port.mk: new variable DISTVERSION, raw distribution version >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-ports-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Fri Mar 12 05:20:17 PST 2004 >Closed-Date: >Last-Modified: >Originator: Oliver Eikemeier >Release: FreeBSD 4.9-STABLE i386 >Organization: Fillmore Labs - http://www.fillmore-labs.com >Environment: System: FreeBSD nuuk.fillmore-labs.com 4.9-STABLE >Description: Kind of a mini-portsurvey for home use. For committers and port maintainers that have bandwidth to waste. type `make check-distfiles' and all distfiles will be fetched and checksummed. >How-To-Repeat: >Fix: This is a proff-of-concept patch. If this functionality is deemed useful, I might do a rework of the fetching / checksuming code to avoid code duplication. Index: Mk/bsd.port.mk =================================================================== RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v retrieving revision 1.484 diff -u -r1.484 bsd.port.mk --- Mk/bsd.port.mk 4 Feb 2004 04:27:04 -0000 1.484 +++ Mk/bsd.port.mk 12 Mar 2004 13:13:44 -0000 @@ -3219,6 +3219,158 @@ .endif .endif +# Fetch and checksum all distfiles + +.if !target(check-distfiles) +check-distfiles: + @(FAILED_DISTFILES=""; \ + COUNT_FAILED=0; \ + COUNT_SUCCESSFUL=0; \ + ${_MASTER_SITES_ENV} ; \ + for _file in ${DISTFILES}; do \ + file=`${ECHO_CMD} $$_file | ${SED} -E -e 's/:[^:]+$$//'`; \ + select=`${ECHO_CMD} $${_file#$${file}} | ${SED} -e 's/^://' -e 's/,/ /g'`; \ + DIR=${DIST_SUBDIR}; \ + pattern="$${DIR:+$$DIR/}`${ECHO_CMD} $$file | ${SED} -e 's/\./\\\\./g'`"; \ + if [ -f ${MD5_FILE} -a "x${NO_CHECKSUM}" = "x" ]; then \ + CKSUM2=`${GREP} "^MD5 ($$pattern)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + if [ -z "$${CKSUM2}" ]; then \ + ${ECHO_MSG} ">> $${DIR:+$$DIR/}$$file is not in ${MD5_FILE}."; \ + ${ECHO_MSG} ">> Either ${MD5_FILE} is out of date, or"; \ + ${ECHO_MSG} ">> $${DIR:+$$DIR/}$$file is spelled incorrectly."; \ + exit 1; \ + fi; \ + fi; \ + if [ ! -z "$$select" ]; then \ + __MASTER_SITES_TMP=""; \ + for group in $$select; do \ + if [ ! -z \$${_MASTER_SITES_$${group}} ]; then \ + eval ___MASTER_SITES_TMP="\$${_MASTER_SITES_$${group}}"; \ + __MASTER_SITES_TMP="$${__MASTER_SITES_TMP} $${___MASTER_SITES_TMP}"; \ + fi \ + done; \ + ___MASTER_SITES_TMP=; \ + SORTED_MASTER_SITES_CMD_TMP="${ECHO_CMD} ${_MASTER_SITE_OVERRIDE} `${ECHO_CMD} $${__MASTER_SITES_TMP} | ${AWK} '${MASTER_SORT_AWK:S|\\|\\\\|g}'` ${_MASTER_SITE_BACKUP}"; \ + else \ + SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}"; \ + fi; \ + for site in `eval $$SORTED_MASTER_SITES_CMD_TMP`; do \ + ${ECHO_MSG} ">> Attempting to fetch $${site}$${file}."; \ + CKSIZE=`${GREP} "^SIZE ($$pattern)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + args=" -o - $${site}$${file}"; \ + _FETCHRESULT=`{ if ${SETENV} ${FETCH_ENV} ${FETCH_CMD} \ + ${FETCH_BEFORE_ARGS} $${args} ${FETCH_AFTER_ARGS}; then :; \ + else ${ECHO_CMD} "FAILURE: $$?" >&3; fi | ${MD5}; } 3>&1`; \ + CKSUM=`${ECHO_CMD} "$$_FETCHRESULT" | ${SED} -ne '/^[[:xdigit:]]\{32\}$$/p'`; \ + _FETCHFAILED=`${ECHO_CMD} "$$_FETCHRESULT" | ${SED} -ne '/^FAILURE: /s///p'`; \ + ckmatch=${FALSE}; \ + if [ -n "$$_FETCHFAILED" ]; then \ + ${ECHO_MSG} ">> Couldn't fetch from $${site}."; \ + else \ + for cksum2 in $$CKSUM2; do \ + if [ "$$cksum2" = "$$CKSUM" ]; then \ + ckmatch=${TRUE}; \ + break; \ + fi; \ + done; \ + if $$ckmatch; then \ + ${ECHO_MSG} ">> Checksum OK for $$site."; \ + else \ + ${ECHO_MSG} ">> Checksum mismatch for $$site."; \ + fi; \ + fi; \ + if $$ckmatch; then \ + COUNT_SUCCESSFUL=$$(($$COUNT_SUCCESSFUL+1)); \ + else \ + COUNT_FAILED=$$(($$COUNT_FAILED+1)); \ + FAILED_DISTFILES="$$FAILED_DISTFILES $${site}$${file}"; \ + fi; \ + done; \ + done; \ + ${ECHO_MSG} ">> fetched $$COUNT_SUCCESSFUL distfiles."; \ + if [ $$COUNT_FAILED -gt 0 ]; then \ + ${ECHO} ">> $$COUNT_FAILED distfiles failed:"; \ + ${ECHO} $$FAILED_DISTFILES | ${TR} ' ' '\n' | ${SED} -e 's/^/ - /'; \ + else \ + ${ECHO} ">> All distfiles are fetchable."; \ + fi \ + ) +.if defined(PATCHFILES) + @(FAILED_DISTFILES=""; \ + COUNT_FAILED=0; \ + COUNT_SUCCESSFUL=0; \ + ${_PATCH_SITES_ENV}; \ + for _file in ${PATCHFILES}; do \ + file=`${ECHO_CMD} $$_file | ${SED} -E -e 's/:[^:]+$$//'`; \ + select=`${ECHO_CMD} $${_file#$${file}} | ${SED} -e 's/^://' -e 's/,/ /g'`; \ + DIR=${DIST_SUBDIR}; \ + pattern="$${DIR:+$$DIR/}`${ECHO_CMD} $$file | ${SED} -e 's/\./\\\\./g'`"; \ + if [ -f ${MD5_FILE} -a "x${NO_CHECKSUM}" = "x" ]; then \ + CKSUM2=`${GREP} "^MD5 ($$pattern)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + if [ -z "$${CKSUM2}" ]; then \ + ${ECHO_MSG} ">> $${DIR:+$$DIR/}$$file is not in ${MD5_FILE}."; \ + ${ECHO_MSG} ">> Either ${MD5_FILE} is out of date, or"; \ + ${ECHO_MSG} ">> $${DIR:+$$DIR/}$$file is spelled incorrectly."; \ + exit 1; \ + fi; \ + fi; \ + if [ ! -z "$$select" ]; then \ + __PATCH_SITES_TMP=; \ + for group in $$select; do \ + if [ ! -z \$${_PATCH_SITES_$${group}} ]; then \ + eval ___PATCH_SITES_TMP="\$${_PATCH_SITES_$${group}}"; \ + __PATCH_SITES_TMP="$${__PATCH_SITES_TMP} $${___PATCH_SITES_TMP}"; \ + fi \ + done; \ + ___PATCH_SITES_TMP=; \ + SORTED_PATCH_SITES_CMD_TMP="${ECHO_CMD} ${_MASTER_SITE_OVERRIDE} `${ECHO_CMD} $${__PATCH_SITES_TMP} ${_RANK_SITES} | ${AWK} '${MASTER_SORT_AWK:S|\\|\\\\|g}'` ${_MASTER_SITE_BACKUP}"; \ + else \ + SORTED_PATCH_SITES_CMD_TMP="${SORTED_PATCH_SITES_DEFAULT_CMD}"; \ + fi; \ + for site in `eval $$SORTED_PATCH_SITES_CMD_TMP`; do \ + ${ECHO_MSG} ">> Attempting to fetch $${site}$${file}."; \ + CKSIZE=`${GREP} "^SIZE ($$pattern)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + args="-o - $${site}$${file}"; \ + _FETCHRESULT=`{ if ${SETENV} ${FETCH_ENV} ${FETCH_CMD} \ + ${FETCH_BEFORE_ARGS} $${args} ${FETCH_AFTER_ARGS}; then :; \ + else ${ECHO_CMD} "FAILURE: $$?" >&3; fi | ${MD5}; } 3>&1`; \ + CKSUM=`${ECHO_CMD} "$$_FETCHRESULT" | ${SED} -ne '/^[[:xdigit:]]\{32\}$$/p'`; \ + _FETCHFAILED=`${ECHO_CMD} "$$_FETCHRESULT" | ${SED} -ne '/^FAILURE: /s///p'`; \ + ckmatch=${FALSE}; \ + if [ -n "$$_FETCHFAILED" ]; then \ + ${ECHO_MSG} ">> Couldn't fetch from $${site}."; \ + else \ + for cksum2 in $$CKSUM2; do \ + if [ "$$cksum2" = "$$CKSUM" ]; then \ + ckmatch=${TRUE}; \ + break; \ + fi; \ + done; \ + if $$ckmatch; then \ + ${ECHO_MSG} ">> Checksum OK for $$site."; \ + else \ + ${ECHO_MSG} ">> Checksum mismatch for $$site."; \ + fi; \ + fi; \ + if $$ckmatch; then \ + COUNT_SUCCESSFUL=$$(($$COUNT_SUCCESSFUL+1)); \ + else \ + COUNT_FAILED=$$(($$COUNT_FAILED+1)); \ + FAILED_DISTFILES="$$FAILED_DISTFILES $${site}$${file}"; \ + fi; \ + done; \ + done; \ + ${ECHO_MSG} ">> fetched $$COUNT_SUCCESSFUL patchfiles."; \ + if [ $$COUNT_FAILED -gt 0 ]; then \ + ${ECHO} ">> $$COUNT_FAILED patchfiles failed:"; \ + ${ECHO} $$FAILED_DISTFILES | ${TR} ' ' '\n' | ${SED} -e 's/^/ - /'; \ + else \ + ${ECHO} ">> All patchfiles are fetchable."; \ + fi \ + ) +.endif +.endif + # Extract .if !target(do-extract) >Release-Note: >Audit-Trail: >Unformatted: