Date: Tue, 9 May 2017 22:13:14 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: pmhausen <pmhausen@me.com> Cc: python@freebsd.org Subject: Re: Proposal for pkg-message for python 3.x ports Message-ID: <20170509191314.GK1622@kib.kiev.ua> In-Reply-To: <36BF7EEF-EE06-4A70-B0F2-69D1454CD1FF@me.com> References: <36BF7EEF-EE06-4A70-B0F2-69D1454CD1FF@me.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, May 09, 2017 at 07:23:38PM +0200, pmhausen wrote: > Dear python port maintainers, > > I discovered an odd performance problem with python 3.6 on servers with > large amounts of memory. > > You can find all the details in a thread of the help@python.org > mailinglist with subject > > "Python bug? Closing *all* file descriptors on fork() instead of only open ones?" > > Unfortunately I cannot get at the list archives. Otherwise I would have sent > you a direct link. > > Short version: since python 3.3 (if my google-fu worked) python tries to > close all open file descriptors in the child after fork(). Since there is > no standard POSIX compliant way to do this for *open* descriptors only, > it tries to use /dev/fd on FreeBSD and if that is not available it closes > *all* descriptors (resulting in ERR#9 for the majority). On a 64 bit FreeBSD > 11 system with 128 GB of RAM the auto-tuned kern.maxfilesperproc > is between 3 and 4 million. That's a heck of a lot of system calls for each > fork() ;-) > > I suggest adding a hint to mount fdescfs and add it to /etc/fstab in the > pkg-message for the affected python ports. We have closefrom(2), which is semi-portable, in the sense that it is available not only on FreeBSD.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170509191314.GK1622>