From owner-svn-ports-all@freebsd.org Wed Sep 12 14:53:22 2018 Return-Path: Delivered-To: svn-ports-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4062910959B7; Wed, 12 Sep 2018 14:53:22 +0000 (UTC) (envelope-from dbn@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E736185CB3; Wed, 12 Sep 2018 14:53:21 +0000 (UTC) (envelope-from dbn@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E12AA27FEF; Wed, 12 Sep 2018 14:53:21 +0000 (UTC) (envelope-from dbn@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w8CErLVo023002; Wed, 12 Sep 2018 14:53:21 GMT (envelope-from dbn@FreeBSD.org) Received: (from dbn@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w8CErLng023001; Wed, 12 Sep 2018 14:53:21 GMT (envelope-from dbn@FreeBSD.org) Message-Id: <201809121453.w8CErLng023001@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dbn set sender to dbn@FreeBSD.org using -f From: David Naylor Date: Wed, 12 Sep 2018 14:53:21 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r479607 - head/Mk/Uses X-SVN-Group: ports-head X-SVN-Commit-Author: dbn X-SVN-Commit-Paths: head/Mk/Uses X-SVN-Commit-Revision: 479607 X-SVN-Commit-Repository: ports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2018 14:53:22 -0000 Author: dbn Date: Wed Sep 12 14:53:21 2018 New Revision: 479607 URL: https://svnweb.freebsd.org/changeset/ports/479607 Log: USES=mono: improve makenupkg - add support for v3 of the nuget API - switch to using v3 for NUGET [1] - rewrite makenupkg to avoid full feed list [2] - change fetch group to allow "+" in package name/version [1] NUGET redirects to the source under the v2 API, however this breaks `makenupkg` as curl fails to follow the redirect when getting the header. [2] Previously, makenupkg got the complete list of packages (and versions) a feed. However, for some feeds this takes a very really long time (slow server response rate). To avoid this, rather check if the package is available (without downloading it). Modified: head/Mk/Uses/mono.mk Modified: head/Mk/Uses/mono.mk ============================================================================== --- head/Mk/Uses/mono.mk Wed Sep 12 14:13:14 2018 (r479606) +++ head/Mk/Uses/mono.mk Wed Sep 12 14:53:21 2018 (r479607) @@ -102,7 +102,8 @@ _NUGET_PACKAGEDIR= ${WRKDIR}/.nuget/packages NUGET_PACKAGEDIR?= ${WRKSRC}/packages NUGET_LAYOUT?= legacy NUGET_FEEDS?= NUGET -NUGET_URL?= https://www.nuget.org/api/v2/ +NUGET_URL?= https://api.nuget.org/v3-flatcontainer/ +NUGET_VERSION?= v3 PAKET_PACKAGEDIR?= PAKET_DEPENDS?= @@ -112,19 +113,25 @@ NUGET_DEPENDS?= ${PAKET_DEPENDS} ${feed}_DEPENDS?= ${feed}_FILE?= ${PKGDIR}/nupkg-${feed:tl} ${feed}_URL?= https://dotnet.myget.org/F/${feed:tl:S/_/-/g}/api/v2/ +${feed}_VERSION?= v2 . if exists(${${feed}_FILE}) ${feed}_EXTRA!= ${CAT} ${${feed}_FILE} . else ${feed}_EXTRA= . endif +MAKENUPKG_ENV+= ${feed:tl}_URL="${${feed}_URL}" ${feed:tl}_VERSION="${${feed}_VERSION}" . for depend in ${${feed}_DEPENDS} ${${feed}_EXTRA} . if empty(_NUGET_DEPENDS:M${depend}) id= ${depend:C/=.*$//} version= ${depend:C/^.*=//} -group= nuget_${depend:S/.//g:S/-//g:S/=//g} +group= nuget_${depend:C/[.+=-]//g} nupkg= ${id:tl}.${version}.nupkg DISTFILES_${group}:= ${nupkg}:${group} +. if ${${feed}_VERSION} == v2 MASTER_SITES_${group}:= ${${feed}_URL}package/${id}/${version}?dummy=/:${group} +. else +MASTER_SITES_${group}:= ${${feed}_URL}${id:tl}/${version}/:${group} +. endif NUGET_NUPKGS_${group}:= ${nupkg}:${depend} NUPKGS_${id}:= ${NUPKGS_${id}} ${version} @@ -202,29 +209,26 @@ makenuget: patch -e '$$!s|$$| \\|g' makenupkg: - @[ -f ${NUGET_EXE} ] || fetch -o ${NUGET_EXE} ${NUGET_LATEST_URL} -.for feed in ${NUGET_FEEDS} - @[ -f ${WRKDIR}/.nupkg-${feed:tl} -o ${feed} = NUGET ] || mono ${NUGET_EXE} list -AllVersions -IncludeDelisted -PreRelease -Source ${${feed}_URL} | ${SED} 's/ /=/g' > ${WRKDIR}/.nupkg-${feed:tl} - @${RM} ${WRKDIR}/nupkg-${feed:tl} -.endfor + @${RM} ${WRKDIR}/nupkg-* @for nuspec in `${FIND} ${_NUGET_PACKAGEDIR} -name '*.nuspec'`; do \ name="`${SED} -nE 's|.*(.*).*|\1|p' $$nuspec`"; \ version="`${SED} -nE 's|.*(.*).*|\1|p' $$nuspec`"; \ - ${ECHO} $$name=$$version; \ - done | ${SORT} -u > ${WRKDIR}/.nupkgs - @${CAT} ${WRKDIR}/.nupkgs | while read nupkg; do \ - default=no; \ - for feed in ${NUGET_FEEDS:tl}; do \ - if [ $$feed = nuget ]; then \ - default=yes; \ - elif ${GREP} -q "^$$nupkg\$$" ${WRKDIR}/.nupkg-$$feed; then \ - ${ECHO} $$nupkg >> ${WRKDIR}/nupkg-$$feed; \ - default=na; \ + echo $$name=$$version ; \ + done | ${SORT} -u | ${SETENV} ${MAKENUPKG_ENV} ${XARGS} -n1 sh -c ' \ + for feed in ${NUGET_FEEDS:MNUGET:tl} ${NUGET_FEEDS:NNUGET:tl}; do \ + if eval [ "\$${$${feed}_VERSION}" = v2 ]; then \ + eval url="\$${$${feed}_URL}package/$${0%%=*}/$${0##*=}"; \ + else \ + eval url="\$${$${feed}_URL}$${0%%=*}/$${0##*=}/$${0%%=*}.$${0##*=}.nupkg"; \ + fi; \ + if curl --output /dev/null --silent --head --fail $$url; then\ + ${ECHO} $$0 >> ${WRKDIR}/nupkg-$$feed; \ + found=yes; \ break; \ fi; \ done; \ - if [ $$default = yes ]; then \ - ${ECHO} $$nupkg >> ${WRKDIR}/nupkg-nuget; \ - fi; \ - done + if [ -z "$$found" ]; then \ + echo "$$0: no feed found"; \ + exit 1; \ + fi' .endif