Date: Thu, 4 Feb 2016 08:48:40 +0000 (UTC) From: Kubilay Kocak <koobs@FreeBSD.org> To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r408017 - in head/ports-mgmt/portscout: . files Message-ID: <201602040848.u148meUa000812@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: koobs Date: Thu Feb 4 08:48:40 2016 New Revision: 408017 URL: https://svnweb.freebsd.org/changeset/ports/408017 Log: ports-mgmt/portscout: Loop through all PyPI files While processing Issue 206746 [1] for a security update to security/py-rsa (For versions < 3.3), it was noticed that Portscout had not identified the the newer version, released on 2016-01-13. Investigation revealed that the PyPI SiteHandler in Portscout only processed the first url/filename returned by PyPI, which in many cases is not a tar.gz, the default EXTRACT_SUFFIX for source distribution (sdist) files: [py-rsa] VersionCheck() [py-rsa] Checking site: https://pypi.python.org/packages/source/r/rsa/ Does site handler exist ... Yes (Portscout::SiteHandler::PyPI) GET https://pypi.python.org/pypi/rsa/json (Portscout::SiteHandler::PyPI) GET success: 200 Filename: rsa-3.3-py2.py3-none-any.whl FindNewest: Checking rsa-3.3-py2.py3-none-any.whl ... against port DISTFILES. FindNewest: Checking DISTFILE ... rsa-3.1.4.tar.gz (ver: 3.1.4, sufx: .tar.gz) [py-rsa] Done This change backports a commit [1] made to Portroach which adds a loop to enumerate all URLs/filenames in the PyPI JSON response, not just the first. [1] https://github.com/jasperla/portroach/commit/e93b8331f6e5f850bbb5faca866efcbf73de756c PR: 206746 [1] Obtained from: https://github.com/jasperla/portroach Modified: head/ports-mgmt/portscout/Makefile head/ports-mgmt/portscout/files/files-Portscout-SiteHandler-PyPI.pm head/ports-mgmt/portscout/files/patch-Portscout_SiteHandler_PyPI.pm Modified: head/ports-mgmt/portscout/Makefile ============================================================================== --- head/ports-mgmt/portscout/Makefile Thu Feb 4 08:35:33 2016 (r408016) +++ head/ports-mgmt/portscout/Makefile Thu Feb 4 08:48:40 2016 (r408017) @@ -3,7 +3,7 @@ PORTNAME= portscout PORTVERSION= 0.8.1 -PORTREVISION= 6 +PORTREVISION= 7 CATEGORIES= ports-mgmt MASTER_SITES= http://mirror.inerd.com/FreeBSD/distfiles/${PORTNAME}/ \ http://www.atarininja.org/~wxs/distfiles/ \ @@ -39,7 +39,7 @@ PORTDOCS= UPDATING portscout-portconfig. HTTPS_RUN_DEPENDS= p5-LWP-Protocol-https>=0:${PORTSDIR}/www/p5-LWP-Protocol-https -SQLITE_USE= SQLITE=3 +SQLITE_USES= sqlite SQLITE_RUN_DEPENDS= p5-DBD-SQLite>=0:${PORTSDIR}/databases/p5-DBD-SQLite PGSQL_USES= pgsql @@ -60,7 +60,7 @@ post-extract: @${CP} ${FILESDIR}/files-Portscout-SiteHandler-GitHub.pm ${WRKSRC}/Portscout/SiteHandler/GitHub.pm @${CP} ${FILESDIR}/files-Portscout-SiteHandler-PyPI.pm ${WRKSRC}/Portscout/SiteHandler/PyPI.pm -post-patch: +pre-install: .if ${PORT_OPTIONS:MSQLITE} @${REINPLACE_CMD} 's/^\([^#]*DBI:Pg.*\)$$/#\1/g' ${WRKSRC}/portscout.conf @${REINPLACE_CMD} 's/^#\(.*DBI:SQLite.*\)$$/\1/g' ${WRKSRC}/portscout.conf Modified: head/ports-mgmt/portscout/files/files-Portscout-SiteHandler-PyPI.pm ============================================================================== --- head/ports-mgmt/portscout/files/files-Portscout-SiteHandler-PyPI.pm Thu Feb 4 08:35:33 2016 (r408016) +++ head/ports-mgmt/portscout/files/files-Portscout-SiteHandler-PyPI.pm Thu Feb 4 08:48:40 2016 (r408017) @@ -109,14 +109,13 @@ sub GetFiles $ua->agent(USER_AGENT); $resp = $ua->request(HTTP::Request->new(GET => $query)); if ($resp->is_success) { - my ($json, $info, $version); + my ($json, $urls); - $json = decode_json($resp->decoded_content); - $info = $json->{info}; - $version = $info->{version}; - next unless $version; - - push(@$files, $json->{releases}{$version}[0]{filename}); + $json = decode_json($resp->decoded_content); + $urls = $json->{urls}; + foreach my $url (@$urls) { + push(@$files, $url->{filename}); + } } else { _debug("GET failed: " . $resp->code); return 0; Modified: head/ports-mgmt/portscout/files/patch-Portscout_SiteHandler_PyPI.pm ============================================================================== --- head/ports-mgmt/portscout/files/patch-Portscout_SiteHandler_PyPI.pm Thu Feb 4 08:35:33 2016 (r408016) +++ head/ports-mgmt/portscout/files/patch-Portscout_SiteHandler_PyPI.pm Thu Feb 4 08:48:40 2016 (r408017) @@ -1,11 +1,16 @@ ---- Portscout/SiteHandler/PyPI.pm.orig 2015-10-25 05:00:48 UTC +--- Portscout/SiteHandler/PyPI.pm.orig 2016-02-04 08:23:53 UTC +++ Portscout/SiteHandler/PyPI.pm -@@ -115,7 +115,7 @@ sub GetFiles - $info = $json->{info}; - $version = $info->{version}; - next unless $version; -- -+ _debug("GET success: " . $resp->code . " Filename: " . $json->{releases}{$version}[0]{filename}); - push(@$files, $json->{releases}{$version}[0]{filename}); +@@ -109,11 +109,13 @@ sub GetFiles + $ua->agent(USER_AGENT); + $resp = $ua->request(HTTP::Request->new(GET => $query)); + if ($resp->is_success) { ++ _debug("GET success: " . $resp->code); + my ($json, $urls); + + $json = decode_json($resp->decoded_content); + $urls = $json->{urls}; + foreach my $url (@$urls) { ++ _debug("PyPi File: " . $url->{filename}); + push(@$files, $url->{filename}); + } } else { - _debug("GET failed: " . $resp->code);
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201602040848.u148meUa000812>