From owner-dev-commits-ports-all@freebsd.org Sat Jun 26 16:47:35 2021 Return-Path: Delivered-To: dev-commits-ports-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B5E7C656D8C; Sat, 26 Jun 2021 16:47:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GC0Cq4mn6z3Dtj; Sat, 26 Jun 2021 16:47:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8C1BD221CD; Sat, 26 Jun 2021 16:47:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 15QGlZRZ061562; Sat, 26 Jun 2021 16:47:35 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 15QGlZid061561; Sat, 26 Jun 2021 16:47:35 GMT (envelope-from git) Date: Sat, 26 Jun 2021 16:47:35 GMT Message-Id: <202106261647.15QGlZid061561@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: "Danilo G. Baio" Subject: git: a9ed9dd7a32f - main - www/py-scrapy: Update to 2.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dbaio X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a9ed9dd7a32fd90be5a71024854bb5a6197ac837 Auto-Submitted: auto-generated X-BeenThere: dev-commits-ports-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the ports repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jun 2021 16:47:35 -0000 The branch main has been updated by dbaio: URL: https://cgit.FreeBSD.org/ports/commit/?id=a9ed9dd7a32fd90be5a71024854bb5a6197ac837 commit a9ed9dd7a32fd90be5a71024854bb5a6197ac837 Author: Danilo G. Baio AuthorDate: 2021-06-26 16:43:12 +0000 Commit: Danilo G. Baio CommitDate: 2021-06-26 16:46:42 +0000 www/py-scrapy: Update to 2.5.0 Changelog: https://docs.scrapy.org/en/latest/news.html?highlight=news#scrapy-2-5-0-2021-04-06 This also removes/disables the experimental HTTP/2 support, there are issues with Twisted and h2 at this moment which breaks scrapy. PR: 256259 Approved by: maintainer timeout (skreuzer) --- www/py-scrapy/Makefile | 32 ++++++++++++++++++-------------- www/py-scrapy/distinfo | 6 +++--- www/py-scrapy/files/patch-setup.py | 19 +++++++++++++++++++ www/py-scrapy/pkg-descr | 6 +++--- 4 files changed, 43 insertions(+), 20 deletions(-) diff --git a/www/py-scrapy/Makefile b/www/py-scrapy/Makefile index 8abedc76b161..6527fdb02154 100644 --- a/www/py-scrapy/Makefile +++ b/www/py-scrapy/Makefile @@ -1,8 +1,7 @@ # Created by: Qing Feng PORTNAME= Scrapy -DISTVERSION= 1.6.0 -PORTREVISION= 2 +DISTVERSION= 2.5.0 CATEGORIES= www python MASTER_SITES= CHEESESHOP PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} @@ -13,23 +12,28 @@ COMMENT= High level scraping and web crawling framework LICENSE= BSD3CLAUSE LICENSE_FILE= ${WRKSRC}/LICENSE -RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}twisted>=13.1.0:devel/py-twisted@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}lxml>0:devel/py-lxml@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}sqlite3>0:databases/py-sqlite3@${PY_FLAVOR} \ +RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}cryptography>=2.0:security/py-cryptography@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}cssselect>=0.9.1:www/py-cssselect@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}itemloaders>=1.0.1:devel/py-itemloaders@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}parsel>=1.5:textproc/py-parsel@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}openssl>=16.2.0:security/py-openssl@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}queuelib>=1.4.2:sysutils/py-queuelib@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}service_identity>=16.0.0:security/py-service_identity@${PY_FLAVOR} \ ${PYTHON_PKGNAMEPREFIX}w3lib>=1.17.0:www/py-w3lib@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}cssselect>=0.9:www/py-cssselect@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}queuelib>0:sysutils/py-queuelib@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}pydispatcher>=2.0.5:devel/py-pydispatcher@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}service_identity>0:security/py-service_identity@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}six>=1.5.2:devel/py-six@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}parsel>=1.5:textproc/py-parsel@${PY_FLAVOR} + ${PYTHON_PKGNAMEPREFIX}zope.interface>=4.1.3:devel/py-zope.interface@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}protego>=0.1.15:www/py-protego@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}itemadapter>=0.1.0:devel/py-itemadapter@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}lxml>=3.5.0:devel/py-lxml@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}pydispatcher>=2.0.5:devel/py-pydispatcher@${PY_FLAVOR} USES= python:3.6+ USE_PYTHON= distutils concurrent autoplist -OPTIONS_DEFINE= SSL -OPTIONS_DEFAULT= SSL +NO_ARCH= yes -SSL_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}openssl>0:security/py-openssl@${PY_FLAVOR} +# Remove experimental HTTP/2 support, issues with Twisted and h2 +post-extract: + @${RM} -r ${WRKSRC}/scrapy/core/http2 + @${RM} ${WRKSRC}/scrapy/core/downloader/handlers/http2.py .include diff --git a/www/py-scrapy/distinfo b/www/py-scrapy/distinfo index d83923a10cae..a5f52400e87d 100644 --- a/www/py-scrapy/distinfo +++ b/www/py-scrapy/distinfo @@ -1,3 +1,3 @@ -TIMESTAMP = 1549229032 -SHA256 (Scrapy-1.6.0.tar.gz) = 558dfd10ac53cb324ecd7eefd3eac412161c7507c082b01b0bcd2c6e2e9f0766 -SIZE (Scrapy-1.6.0.tar.gz) = 926576 +TIMESTAMP = 1622321274 +SHA256 (Scrapy-2.5.0.tar.gz) = 0a68ed41f7173679f160c4cef2db05288548c21e7164170552adae8b13cefaab +SIZE (Scrapy-2.5.0.tar.gz) = 1071824 diff --git a/www/py-scrapy/files/patch-setup.py b/www/py-scrapy/files/patch-setup.py new file mode 100644 index 000000000000..8571812fecc3 --- /dev/null +++ b/www/py-scrapy/files/patch-setup.py @@ -0,0 +1,19 @@ +# Remove experimental HTTP/2 support, issues with Twisted and h2 +--- setup.py.orig 2021-04-06 14:48:02 UTC ++++ setup.py +@@ -19,7 +19,6 @@ def has_environment_marker_platform_impl_support(): + + + install_requires = [ +- 'Twisted[http2]>=17.9.0', + 'cryptography>=2.0', + 'cssselect>=0.9.1', + 'itemloaders>=1.0.1', +@@ -31,7 +30,6 @@ install_requires = [ + 'zope.interface>=4.1.3', + 'protego>=0.1.15', + 'itemadapter>=0.1.0', +- 'h2>=3.0,<4.0', + ] + extras_require = {} + cpython_dependencies = [ diff --git a/www/py-scrapy/pkg-descr b/www/py-scrapy/pkg-descr index 01e275464787..3da8287b29bf 100644 --- a/www/py-scrapy/pkg-descr +++ b/www/py-scrapy/pkg-descr @@ -1,5 +1,5 @@ -Scrapy is a high level scraping and web crawling framework for writing -spiders to crawl and parse web pages for all kinds of purposes, from -information retrieval to monitoring or testing web sites. +Scrapy is a fast high-level web crawling and web scraping framework, used to +crawl websites and extract structured data from their pages. It can be used for +a wide range of purposes, from data mining to monitoring and automated testing. WWW: https://scrapy.org/