From owner-svn-ports-head@FreeBSD.ORG Wed Jul 23 12:14:32 2014 Return-Path: Delivered-To: svn-ports-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1FEC363; Wed, 23 Jul 2014 12:14:32 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B278F2435; Wed, 23 Jul 2014 12:14:32 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6NCEWnB026066; Wed, 23 Jul 2014 12:14:32 GMT (envelope-from sbz@svn.freebsd.org) Received: (from sbz@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6NCEW1o026065; Wed, 23 Jul 2014 12:14:32 GMT (envelope-from sbz@svn.freebsd.org) Message-Id: <201407231214.s6NCEW1o026065@svn.freebsd.org> From: Sofian Brabez Date: Wed, 23 Jul 2014 12:14:32 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r362667 - head/Tools/scripts X-SVN-Group: ports-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-head@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the ports tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 12:14:33 -0000 Author: sbz Date: Wed Jul 23 12:14:32 2014 New Revision: 362667 URL: http://svnweb.freebsd.org/changeset/ports/362667 QAT: https://qat.redports.org/buildarchive/r362667/ Log: - Better handling when custom encoding is specified in details - Use only one request to retreive patchs CR: D400 CR: D401 Submitted by: rakuco Modified: head/Tools/scripts/getpatch Modified: head/Tools/scripts/getpatch ============================================================================== --- head/Tools/scripts/getpatch Wed Jul 23 12:13:02 2014 (r362666) +++ head/Tools/scripts/getpatch Wed Jul 23 12:14:32 2014 (r362667) @@ -120,24 +120,29 @@ class BzGetPatch(GetPatch): URL_BASE= 'https://bugs.freebsd.org/bugzilla/' URL_SHOW = '%s/show_bug.cgi?id=' % URL_BASE REGEX_URL = r'Details' - REGEX = r'
([^ ]+) \(text/plain\)' + REGEX = r'
([^ ]+) \(text/plain(?:; charset=[-\w]+)?\)' def __init__(self, pr, category): GetPatch.__init__(self, pr, category) - def _extract_patchs_url(self, data): - pattern = re.compile(self.REGEX_URL) - return re.findall(pattern, data) - - def _extract_patchs_name(self, urls): - names = [] - pattern = re.compile(self.REGEX) - for url in urls: - u = urllib2.urlopen('%s/%s' % (self.URL_BASE, url)) - data = u.read() - names.append(re.findall(pattern, data)[0]) - - return names + def _get_patch_name(self, url): + match = re.search(self.REGEX, urllib2.urlopen(url).read()) + if match is None: + return None + return match.group(1) + + def _get_patch_urls(self, data): + patch_urls = {} + for url in re.findall(self.REGEX_URL, data): + url = '%s/%s' % (self.URL_BASE, url) + file_name = self._get_patch_name(url) + if file_name is None: + self.out("[-] Could not determine the patch file name in %s. " + "Skipping." % url) + continue + download_url = url[:url.find('&')] + patch_urls[download_url] = file_name + return patch_urls def fetch(self, *largs, **kwargs): category = kwargs['category'] @@ -149,19 +154,13 @@ class BzGetPatch(GetPatch): self.out("[-] No patch found") sys.exit(1) - urls = self._extract_patchs_url(data) - nb_urls = len(urls) - names = self._extract_patchs_name(urls) - nb_names = len(names) - - urls = ['%s/%s' % (self.URL_BASE, u[:u.find('&')]) for u in urls] - - if nb_names == 0 or nb_urls == 0 or nb_names != nb_urls: + patch_urls = self._get_patch_urls(data) + if not patch_urls: self.out("[-] No patch found") sys.exit(1) - for i in range(nb_urls): - self.add_patch(urls[i], names[i]) + for url, file_name in patch_urls.iteritems(): + self.add_patch(url, file_name) def main():