From owner-freebsd-python@freebsd.org Fri Nov 29 10:56:01 2019
Return-Path: <owner-freebsd-python@freebsd.org>
Delivered-To: freebsd-python@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.nyi.freebsd.org (Postfix) with ESMTP id 52BE31AC06A
for <freebsd-python@mailman.nyi.freebsd.org>;
Fri, 29 Nov 2019 10:56:01 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::50:13])
by mx1.freebsd.org (Postfix) with ESMTP id 47PWdY1bC9z3HGM
for <freebsd-python@freebsd.org>; Fri, 29 Nov 2019 10:56:01 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: by mailman.nyi.freebsd.org (Postfix)
id 347371AC069; Fri, 29 Nov 2019 10:56:01 +0000 (UTC)
Delivered-To: python@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.nyi.freebsd.org (Postfix) with ESMTP id 343A31AC068
for <python@mailman.nyi.freebsd.org>; Fri, 29 Nov 2019 10:56:01 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits)
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 47PWdY0gzgz3HGK
for <python@FreeBSD.org>; Fri, 29 Nov 2019 10:56:01 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org (kenobi.freebsd.org
[IPv6:2610:1c1:1:606c::50:1d])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EF19BD18F
for <python@FreeBSD.org>; Fri, 29 Nov 2019 10:56:00 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org ([127.0.1.5])
by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xATAu0Ij014019
for <python@FreeBSD.org>; Fri, 29 Nov 2019 10:56:00 GMT
(envelope-from bugzilla-noreply@freebsd.org)
Received: (from bugzilla@localhost)
by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xATAu02m014018
for python@FreeBSD.org; Fri, 29 Nov 2019 10:56:00 GMT
(envelope-from bugzilla-noreply@freebsd.org)
X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to
bugzilla-noreply@freebsd.org using -f
From: bugzilla-noreply@freebsd.org
To: python@FreeBSD.org
Subject: [Bug 242274] lang/python??: adopt local closefrom(2) patches
Date: Fri, 29 Nov 2019 10:56:00 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: Ports & Packages
X-Bugzilla-Component: Individual Port(s)
X-Bugzilla-Version: Latest
X-Bugzilla-Keywords: needs-qa
X-Bugzilla-Severity: Affects Only Me
X-Bugzilla-Who: commit-hook@freebsd.org
X-Bugzilla-Status: In Progress
X-Bugzilla-Resolution:
X-Bugzilla-Priority: ---
X-Bugzilla-Assigned-To: koobs@FreeBSD.org
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-242274-21822-y16xkTYG8Z@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-242274-21822@https.bugs.freebsd.org/bugzilla/>
References: <bug-242274-21822@https.bugs.freebsd.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-python>,
<mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python/>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-python>,
<mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Nov 2019 10:56:01 -0000
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242274
--- Comment #4 from commit-hook@freebsd.org ---
A commit references this bug:
Author: koobs
Date: Fri Nov 29 10:55:03 UTC 2019
New revision: 518640
URL: https://svnweb.freebsd.org/changeset/ports/518640
Log:
lang/python{27,35,36,37,38}: Add closefrom(2) support
A single close(fd) syscall is cheap, but when MAXFDS (maximum file
descriptor number) is high, the loop calling close(fd) on each file
descriptor can take several milliseconds.
The default value of subprocess.Popen "close_fds" parameter changed to Tr=
ue
in Python 3. Compared to Python 2, close_fds=3DTrue can make Popen 10x
slower: see bpo-37790 [1]
The present workaround on FreeBSD to improve performance is to load and
mount the fdescfs kernel module, but this is not enabled by default.
This change adds minimum viable (and upstreamable) closefrom(2) syscall
support to Python's subprocess and posix modules, improving performance
significantly for loads that involve working with many processes, such as
diffoscope, ansible, and many others.
For additional optimizations, upstream recently (3.8) landed posix_spawn(=
2)
support [3] and has stated that they will adopt close_range(2) after Linux
merges it [4]. Linux/FreeBSD developers are already collaborating on
ensuring compatible implementations, with FreeBSD's implementation pending
in D21627. [5]
Thank you emaste, cem, kevans for providing analysis, input,
clarifications, comms/upstream support and patches.
[1] https://bugs.python.org/issue37790
[2] https://bugs.python.org/issue38061
[3] https://bugs.python.org/issue35537
[4] https://lwn.net/Articles/789023/
[5] https://reviews.freebsd.org/D21627
Additional References:
https://bugs.python.org/issue8052
https://bugs.python.org/issue11284
https://bugs.python.org/issue13788
https://bugs.python.org/issue1663329
https://www.python.org/dev/peps/pep-0446/
PR: 242274, 221700
Submitted by: kevans (emaste, cem)
Approved by: koobs (python (maintainer), santa)
Changes:
head/lang/python27/Makefile
head/lang/python27/files/patch-Modules_posixmodule.c
head/lang/python35/Makefile
head/lang/python35/files/patch-Modules___posixsubprocess.c
head/lang/python35/files/patch-Modules_posixmodule.c
head/lang/python36/Makefile
head/lang/python36/files/patch-Modules___posixsubprocess.c
head/lang/python36/files/patch-Modules_posixmodule.c
head/lang/python37/Makefile
head/lang/python37/files/patch-Modules___posixsubprocess.c
head/lang/python37/files/patch-Modules_posixmodule.c
head/lang/python38/Makefile
head/lang/python38/files/patch-Modules___posixsubprocess.c
head/lang/python38/files/patch-Modules_posixmodule.c
--=20
You are receiving this mail because:
You are on the CC list for the bug.=