From owner-svn-ports-all@freebsd.org Tue Oct 1 03:47:17 2019 Return-Path: Delivered-To: svn-ports-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1051D13F3CC; Tue, 1 Oct 2019 03:47:17 +0000 (UTC) (envelope-from koobs@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46j4w46jlrz3NPp; Tue, 1 Oct 2019 03:47:16 +0000 (UTC) (envelope-from koobs@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C9B2AD775; Tue, 1 Oct 2019 03:47:16 +0000 (UTC) (envelope-from koobs@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x913lGt3041746; Tue, 1 Oct 2019 03:47:16 GMT (envelope-from koobs@FreeBSD.org) Received: (from koobs@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x913lGkb041744; Tue, 1 Oct 2019 03:47:16 GMT (envelope-from koobs@FreeBSD.org) Message-Id: <201910010347.x913lGkb041744@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: koobs set sender to koobs@FreeBSD.org using -f From: Kubilay Kocak Date: Tue, 1 Oct 2019 03:47:16 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r513446 - in head/irc/py-limnoria: . files X-SVN-Group: ports-head X-SVN-Commit-Author: koobs X-SVN-Commit-Paths: in head/irc/py-limnoria: . files X-SVN-Commit-Revision: 513446 X-SVN-Commit-Repository: ports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Oct 2019 03:47:17 -0000 Author: koobs Date: Tue Oct 1 03:47:16 2019 New Revision: 513446 URL: https://svnweb.freebsd.org/changeset/ports/513446 Log: irc/py-limnoria: Backport Web plugin bugfixes Backport fixes for the Web plugin [1][2][3]. [1] https://github.com/ProgVal/Limnoria/pull/1371 [2] https://github.com/ProgVal/Limnoria/issues/1362 [3] https://github.com/ProgVal/Limnoria/issues/1359 Submitted by: DanDare (GitHub: Rodrigo-NH, via IRC) Added: head/irc/py-limnoria/files/ head/irc/py-limnoria/files/patch-plugins_Web_plugin.py (contents, props changed) Modified: head/irc/py-limnoria/Makefile Modified: head/irc/py-limnoria/Makefile ============================================================================== --- head/irc/py-limnoria/Makefile Tue Oct 1 03:23:17 2019 (r513445) +++ head/irc/py-limnoria/Makefile Tue Oct 1 03:47:16 2019 (r513446) @@ -3,7 +3,7 @@ PORTNAME= limnoria PORTVERSION= 2018.09.09 -PORTREVISION= 1 +PORTREVISION= 2 CATEGORIES= irc python PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} Added: head/irc/py-limnoria/files/patch-plugins_Web_plugin.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/irc/py-limnoria/files/patch-plugins_Web_plugin.py Tue Oct 1 03:47:16 2019 (r513446) @@ -0,0 +1,62 @@ +# https://github.com/ProgVal/Limnoria/pull/1371 +# https://github.com/ProgVal/Limnoria/issues/1362 +# https://github.com/ProgVal/Limnoria/issues/1359 + +--- plugins/Web/plugin.py.orig 2019-09-29 03:00:58 UTC ++++ plugins/Web/plugin.py +@@ -149,32 +149,31 @@ class Web(callbacks.PluginRegexp): + def getTitle(self, irc, url, raiseErrors): + size = conf.supybot.protocols.http.peekSize() + timeout = self.registryValue('timeout') +- (target, text) = utils.web.getUrlTargetAndContent(url, size=size, +- timeout=timeout) + try: +- text = text.decode(utils.web.getEncoding(text) or 'utf8', +- 'replace') +- except UnicodeDecodeError: +- pass +- parser = Title() +- if minisix.PY3 and isinstance(text, bytes): +- if raiseErrors: +- irc.error(_('Could not guess the page\'s encoding. (Try ' +- 'installing python-charade.)'), Raise=True) ++ (target, text) = utils.web.getUrlTargetAndContent(url, size=size,timeout=timeout) ++ encoding = utils.web.getEncoding(text) ++ if encoding is None: # Condition if charade not installed ++ self.log.info('Web plugin TitleSnarfer: Could not guess the page\'s' ++ ' encoding. (Try installing python-charade.)') ++ encoding = 'utf-8' # Assume UTF-8 and replace unknown chars to the UTF-8 codec for U+FFFD in the next hop ++ text = text.decode(utils.web.getEncoding(text) or 'utf-8','replace') ++ parser = Title() ++ try: ++ parser.feed(text) ++ except: ++ parser = Title() ++ parser.feed(bytes(text)) # Explicitly pack to bytes in encoding errors for (more) python2 compatibility ++ parser.close() ++ title = utils.str.normalizeWhitespace(''.join(parser.data).strip()) ++ if title: ++ return (target, title) + else: +- return None +- parser.feed(text) +- parser.close() +- title = utils.str.normalizeWhitespace(''.join(parser.data).strip()) +- if title: +- return (target, title) +- elif raiseErrors: +- if len(text) < size: +- irc.error(_('That URL appears to have no HTML title.'), +- Raise=True) +- else: +- irc.error(format(_('That URL appears to have no HTML title ' +- 'within the first %S.'), size), Raise=True) ++ if len(text) < size: ++ self.log.info('Web plugin TitleSnarfer: <' + url + '> appears to have no HTML title.') ++ else: ++ self.log.info('Web plugin TitleSnarfer: Could not retrieve title of <' + url + '>') ++ except Exception as e: ++ self.log.info('Web plugin TitleSnarfer: <' + str(e) + '> while trying to process <' + url +'>') + + @fetch_sandbox + def titleSnarfer(self, irc, msg, match):