From owner-svn-ports-all@freebsd.org Sat Dec 7 18:26:37 2019 Return-Path: Delivered-To: svn-ports-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A71141C93F0; Sat, 7 Dec 2019 18:26:37 +0000 (UTC) (envelope-from tcberner@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47VdFn3wV8z44rJ; Sat, 7 Dec 2019 18:26:37 +0000 (UTC) (envelope-from tcberner@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6A5451DD56; Sat, 7 Dec 2019 18:26:37 +0000 (UTC) (envelope-from tcberner@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xB7IQb6p059730; Sat, 7 Dec 2019 18:26:37 GMT (envelope-from tcberner@FreeBSD.org) Received: (from tcberner@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xB7IQaJL059724; Sat, 7 Dec 2019 18:26:36 GMT (envelope-from tcberner@FreeBSD.org) Message-Id: <201912071826.xB7IQaJL059724@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tcberner set sender to tcberner@FreeBSD.org using -f From: "Tobias C. Berner" Date: Sat, 7 Dec 2019 18:26:36 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r519228 - in head/net: . py-cloudscraper X-SVN-Group: ports-head X-SVN-Commit-Author: tcberner X-SVN-Commit-Paths: in head/net: . py-cloudscraper X-SVN-Commit-Revision: 519228 X-SVN-Commit-Repository: ports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Dec 2019 18:26:37 -0000 Author: tcberner Date: Sat Dec 7 18:26:36 2019 New Revision: 519228 URL: https://svnweb.freebsd.org/changeset/ports/519228 Log: net/py-cloudscraper: Python module to bypass Cloudflare's anti-bot page Added: head/net/py-cloudscraper/ head/net/py-cloudscraper/Makefile (contents, props changed) head/net/py-cloudscraper/distinfo (contents, props changed) head/net/py-cloudscraper/pkg-descr (contents, props changed) Modified: head/net/Makefile Modified: head/net/Makefile ============================================================================== --- head/net/Makefile Sat Dec 7 18:20:19 2019 (r519227) +++ head/net/Makefile Sat Dec 7 18:26:36 2019 (r519228) @@ -1022,6 +1022,7 @@ SUBDIR += py-cjdns SUBDIR += py-cloudflare-scrape SUBDIR += py-cloudflare-scrape-js2py + SUBDIR += py-cloudscraper SUBDIR += py-coherence SUBDIR += py-confluent-kafka SUBDIR += py-cymruwhois Added: head/net/py-cloudscraper/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/net/py-cloudscraper/Makefile Sat Dec 7 18:26:36 2019 (r519228) @@ -0,0 +1,28 @@ +# $FreeBSD$ + +PORTNAME= cloudscraper +DISTVERSION= 1.2.9 +CATEGORIES= net python +PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} + +MAINTAINER= tcberner@FreeBSD.org +COMMENT= Python module to bypass Cloudflare's anti-bot page #' + +LICENSE= MIT +LICENSE_FILE= ${WRKSRC}/LICENSE + +RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}openssl>17.0:security/py-openssl@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}requests>0:www/py-requests@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}Js2Py>0:devel/py-Js2Py@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}requests-toolbelt>0:www/py-requests-toolbelt@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}brotli>0:archivers/py-brotli@${PY_FLAVOR} + +USES= python +USE_PYTHON= distutils autoplist + +USE_GITHUB= yes +GH_ACCOUNT= VeNoMouS + +CONFLICTS_INSTALL= ${PYTHON_PKGNAMEPREFIX}cloudflare-scrape-1.* + +.include Added: head/net/py-cloudscraper/distinfo ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/net/py-cloudscraper/distinfo Sat Dec 7 18:26:36 2019 (r519228) @@ -0,0 +1,3 @@ +TIMESTAMP = 1575742950 +SHA256 (VeNoMouS-cloudscraper-1.2.9_GH0.tar.gz) = a16b4b457660b5f8a4eb6336be636eb4f3d05409c9389a06e965d0651dea65e5 +SIZE (VeNoMouS-cloudscraper-1.2.9_GH0.tar.gz) = 396670 Added: head/net/py-cloudscraper/pkg-descr ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/net/py-cloudscraper/pkg-descr Sat Dec 7 18:26:36 2019 (r519228) @@ -0,0 +1,14 @@ +A simple Python module to bypass Cloudflare's anti-bot page (also known as "I'm +Under Attack Mode", or IUAM), implemented with Requests. Cloudflare changes +their techniques periodically, so I will update this repo frequently. + +This can be useful if you wish to scrape or crawl a website protected with +Cloudflare. Cloudflare's anti-bot page currently just checks if the client +supports Javascript, though they may add additional techniques in the future. + +Due to Cloudflare continually changing and hardening their protection page, +cloudscraper requires a JavaScript Engine/interpreter to solve Javascript +challenges. This allows the script to easily impersonate a regular web browser +without explicitly deobfuscating and parsing Cloudflare's Javascript. + +WWW: https://github.com/VeNoMouS/cloudscraper