From owner-freebsd-virtualization@freebsd.org Fri Nov 13 08:02:03 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D2C1A2DAF9; Fri, 13 Nov 2015 08:02:03 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5F1C1AD5; Fri, 13 Nov 2015 08:02:02 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by wmec201 with SMTP id c201so68781851wme.0; Fri, 13 Nov 2015 00:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=dAiKPp+eGvhdA6t2xfoWW+f5N0pM1azLbjVpfeVzb2U=; b=vEZ1fUOAu0MqB7LhBH+GJkuW1d/aBMqbJaQRSXMbHYt48CVtIy5AMqXLvNSDi4HoUt 1jTdsBHwoGZZzibvLtveH+dlrMhtgD9HUjlmfdwnI6GlTuMu4TcEgc4PAQS8agLfkh7o XGwqI0mgzqErtwC71aUcCzYd35qsdB593E6KRSlQfHBshc9vMMS6AJHXgDnVm55ulRjn D60pJJUu5ZHMGyYHjboq7is6FSNz0E+DjHTuK7MC7yaK/3SnLsGYem8PrlnggdeuBFvh jSiRZXgayVoUfDq/H+Ade09ZoAhA0GCjki5b9S9gGI0k0XXUkB2t6zHnkrobCWZLryJP eAVw== X-Received: by 10.28.63.22 with SMTP id m22mr2196960wma.58.1447401720514; Fri, 13 Nov 2015 00:02:00 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id q1sm18573643wjy.31.2015.11.13.00.01.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Nov 2015 00:01:59 -0800 (PST) Sender: Baptiste Daroussin Date: Fri, 13 Nov 2015 09:01:57 +0100 From: Baptiste Daroussin To: Kubilay Kocak Cc: Vladimir Bogrecov , python@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Python on FreeBSD is slower than on Linux Message-ID: <20151113080157.GA93991@ivaldir.etoilebsd.net> References: <56453E9D.3040809@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <56453E9D.3040809@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 08:02:03 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 13, 2015 at 12:36:29PM +1100, Kubilay Kocak wrote: > On 13/11/2015 6:26 AM, Vladimir Bogrecov wrote: > > Hello, > >=20 > > I'm developing a little project on Python 3.5. The server's operating > > system is FreeBSD 10.2. Today I decided to do a little test "just for f= un" > > and the result has confused me. I ran the following code > >=20 > > import random > > import time > >=20 > >=20 > > def test_sort(size): > > sequence =3D [i for i in range(0, size)] > > random.shuffle(sequence) > > start =3D time.time() > > ordered_sequence =3D sorted(sequence) > > print(time.time() - start) > >=20 > >=20 > > if __name__ =3D=3D '__main__': > > test_sort(1000000) > >=20 > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > > (5$ per month) virtual machines on the Digital Ocean ( > > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > > sec, on the Debian 1.0 sec. Both machines was created specially for test > > and had not any customization. Could you help me to understand why pyth= on > > is so slower on FreeBSD and may be there are some steps I can perform to > > speed up the python to work not slower than on Debian. > >=20 > > I have found in Google the similar question: > > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.htm= l so > > it has an interest not only for me. > >=20 > > P.S. I really like FreeBSD and I would be happy to solve this issue. If= you > > will have an interest to this issue I can provide SSH access for both > > machines :) > >=20 > > Thank You! >=20 > From FreeBSD Python's (team) point of view, I can't think of anything > obvious off the top of my head that might cause a ~30% performance issue > for that workload. >=20 > Let's get a trace (truss, strace, dtrace) of what's going during the run > so we can figure out exactly what's happening and in what context. >=20 > With respect to the testing environment, certain VPS providers throttle > bursts of CPU pretty heavily, so you'll want to account for/isolate that > as a potential contributor. Yes both OS's are being run on the same > provider, but as Alfred said, one OS may be mitigating/working around > certain virtualisation 'issues'. >=20 > A full trace of what the test case is doing is definitely the next best > step I can think of, even before profiling in python, which is probably > going to provide insight as well. >=20 > Personally, I'd love to hear about anything that might result in FreeBSD > always topping the charts for Python performance. >=20 Well the python devs are aware by themselves of potential performances issu= es on FreeBSD (and non linux in general) for example subprocess will try to close= fds, on linux by getting the list of fd from /proc/fd and only close the one the= y do not want among the existing ones. on freebsd they do the same if /dev/fd is mounted meaning without /dev/fd, perfs will suck. They do not use closefrom= (2) here because on linux it is not async-signal-safe. one could make them use closefrom(2) on non linux for example or even more efficiently but freebsd = only modify the code to use kinfo_getfile(3). https://bugs.python.org/issue11284 Another area is the AIO iirc (needs to be double checked) the python uses l= inux only things for aio which makes this way slower on FreeBSD. I'm kind of surprised given the number of pythonic people we have that no o= ne has had a look at how python perform on FreeBSD and how things are implemen= ted in the python VM to help them. Bapt --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlZFmPUACgkQ8kTtMUmk6EzWBQCfZhE1Y9Dk68y2EZgyXViTe+ac cp8An3S8uaHRkqOfP5cCVK5TuYcD1Zug =66lI -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM--