From owner-freebsd-virtualization@freebsd.org Sun Nov 8 10:41:47 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60D26A271E0 for ; Sun, 8 Nov 2015 10:41:47 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D60A4119A for ; Sun, 8 Nov 2015 10:41:46 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: by lbbwb3 with SMTP id wb3so82161582lbb.1 for ; Sun, 08 Nov 2015 02:41:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=KNbiGoFGd89KRQOGhuLeIS6PHDewvOtOKyJQBLmIbNQ=; b=UvFKIlb5A36B0qpELWfWfkdpIosNkpQHAKh4dvOJVFE9vmHM7k6PNNYB0qkX32gl1N HvpEcKuyCB390p8obd3nnN9U8ZnCAQ39iMbhytIb/6aQ2g2cfdTdJjuIiHbnb7Njj+hZ +ZmKy21czTpW4+zqcIS4RuP6YTXGP2WpujDq2uoIBEI+qw64airlBbq850pH0lWcsyE3 h9az3YoTRAK+pzQ/Dk+DLpxnFzl88V5kMfjX08WehyeCm99d4U10TNAu0nauVqXFMTOo WJ3A1wUFSo1VAPy0pbRhGbAHAYaU9aWbQuQdvM3nb5Lc7tKaNObhsRkieyPGEVEM96Mo NRrQ== X-Received: by 10.112.166.102 with SMTP id zf6mr11628419lbb.124.1446979304775; Sun, 08 Nov 2015 02:41:44 -0800 (PST) Received: from kloomba.lvv.mirantis.net ([217.65.211.30]) by smtp.gmail.com with ESMTPSA id z62sm1506595lfd.20.2015.11.08.02.41.42 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Nov 2015 02:41:43 -0800 (PST) Sender: Roman Bogorodskiy Date: Sun, 8 Nov 2015 13:41:37 +0300 From: Roman Bogorodskiy To: Nicolas Gilles Cc: freebsd-virtualization@freebsd.org Subject: Re: Illumos support in bhyve Message-ID: <20151108104136.GA2188@kloomba.lvv.mirantis.net> References: <56118B2B.2040101@freebsd.org> <20151105173717.GB2025@kloomba.lvv.mirantis.net> <20151106153332.GB1881@kloomba.lvv.mirantis.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9amGYk9869ThD9tj" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2015 10:41:47 -0000 --9amGYk9869ThD9tj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Nicolas Gilles wrote: > SmartOS is built to work that way, booting from an "static" ISO or USB > or PXE, and just using the hard-drives in the system as store for the > zones + vms, the os "drives" is loaded into memory. >=20 > In a VM, it's not going to change you much to boot form a virtual DVD > drive vs virtual HD install, what troubles you booting from the ISO? Sorry, I should have made myself more clear: my question about self-contained image that does not need an ISO to boot was not SmartOS specific, but general, mainly regarding FreeBSD images or, say, Linux images. > On Fri, Nov 6, 2015 at 4:33 PM, Roman Bogorodskiy wro= te: > > grehan@freebsd.org wrote: > > > >> Hi Roman, > >> > >> > I tried this instruction and I was able to get into smartos > >> > installation. I choose all the default options there, it reported > >> > successful installation. Then I dropped 'ahci-cd' part, but I cannot > >> > get > >> > smartos boot. > >> > >> For smartos, the CD should always be left there since that's the boot > >> media. The first boot just detects that the hard drive is empty which > >> triggers the install. > > > > Ah, I see. It works now, thanks! > > > >> > And, by the way, is it possible to run FreeBSD using UEFI? > >> > > >> > I encountered problems with that as well: > >> > > >> > https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-Nove= mber/003904.html > >> > > >> > Any hints how to debug this are appreciated. > >> > >> The VM images don't include the UEFI loader. You'll have to boot from > >> the ISO for that, and force serial console output at the loader prompt. > > > > Is it possible to prepare an image that would not require booting from > > the ISO? > > > > Roman Bogorodskiy > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" Roman Bogorodskiy --9amGYk9869ThD9tj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJWPybfAAoJEMltX/4IwiJqrSAH/05xcl4daiO6fj5iRscwBbqS 6Erzls3tCihkKNNHA3BcDus6LDkN0e5HXzRUMcW/txRlkZ9I0cc5KQoGC2CsqDtf aKKXZOrM2XLMsMSLV20v+vPWcOeuF0u6vIMciMxc5H13lZ416czEmur8NT+NBRPu mWWSpCEj/25ZzjjIlAMg9skc2PNFeOdmkXpQE7WQMb4I9c2wSBsVawIqYWd0Nx7/ Sb3/VD05Vd3UIMjj7ctBUQEf5UJZQcX1ZCyyL7QyFA1I8AUx1VwuzRXO26k6vNyT SboLC3Obiao8Czjpwa6DvkmO6E+XAeYkAX9QmLiU70uTsUjb9tKAVZgrIdn13bM= =dXud -----END PGP SIGNATURE----- --9amGYk9869ThD9tj-- From owner-freebsd-virtualization@freebsd.org Mon Nov 9 03:25:15 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF45CA29F9D for ; Mon, 9 Nov 2015 03:25:15 +0000 (UTC) (envelope-from vk@it-club.kz) Received: from mail-01.nurhost.kz (mail-01.nurhost.kz [91.201.215.187]) by mx1.freebsd.org (Postfix) with ESMTP id 2B62D1AA2 for ; Mon, 9 Nov 2015 03:25:14 +0000 (UTC) (envelope-from vk@it-club.kz) Received: from it-club.kz (account vk@it-club.kz [91.201.215.202] verified) by mail-01.nurhost.kz (CommuniGate Pro SMTP 6.1.5) with ESMTPA id 31492237 for freebsd-virtualization@freebsd.org; Mon, 09 Nov 2015 09:25:05 +0600 MIME-Version: 1.0 Date: Mon, 09 Nov 2015 09:25:05 +0600 From: vk@it-club.kz To: freebsd-virtualization@freebsd.org Subject: bhyve uefi error please help me! Message-ID: X-Sender: vk@it-club.kz User-Agent: Roundcube Webmail/1.0.2 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2015 03:25:15 -0000 Hello! not workink bhyve and windows please help me: bhyve -c 2 -s 0,hostbridge -s 3,ahci-hd,/images/win.img -s 4,ahci-cd,/images/win_repack.iso -s 10,virtio-net,tap0 -s 31,lpc -l com1,/dev/nmdm0A -l com2,/dev/nmdm1A -l bootrom,/path/to/BHYVE_UEFI.fd -m 2G -H -w windows bhyve: invalid lpc device configuration 'bootrom,BHYVE_UEFI_20151002.fd' https://people.freebsd.org/~grehan/bhyve_uefi/windows_install.txt From owner-freebsd-virtualization@freebsd.org Mon Nov 9 09:01:13 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E26D8A26C69 for ; Mon, 9 Nov 2015 09:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCF901A50 for ; Mon, 9 Nov 2015 09:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tA991DnG004493 for ; Mon, 9 Nov 2015 09:01:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Mon, 09 Nov 2015 09:01:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: mfc-stable8- mfc-stable9- mfc-stable10+ X-Bugzilla-Changed-Fields: bug_status flagtypes.name resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2015 09:01:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 NGie Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Progress |Closed Flags| |mfc-stable8-, mfc-stable9-, | |mfc-stable10+ Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Mon Nov 9 10:47:17 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EBAFA297EA for ; Mon, 9 Nov 2015 10:47:17 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ECCE3111B for ; Mon, 9 Nov 2015 10:47:16 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id tA9Al1pA032424 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 9 Nov 2015 10:47:05 GMT (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1447066025; bh=jUw2+vjAa+Wrm0GknSqrR0TRb8sGuyL8P4upP4BYvH8=; h=From:To:Subject:Date:References; b=oULoup7jXfyhXm9JqFuuxe+hUJQQ87X/F52XIkXw1R2oiqS+U4b+eCtKre1dJbJvS DBC0pl4HKmUwuX2CDLCW8nmrNF47vI6RbRSnz9jDtoNOtpu9+8+qc/eAcRfyAdmcw7 jPKJPUPSJRDodcYy5VIYV9LhNvhkqvttGG2vr6RM= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 9 Nov 2015 10:46:56 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 9 Nov 2015 10:46:56 +0000 From: Matt Churchyard To: FreeBSD virtualization Subject: bhyve uefi error please help me! Thread-Topic: bhyve uefi error please help me! Thread-Index: AQHRGp5KOWHJ84dJykyKxof0f/XfCZ6TgbsAgAAA3VA= Date: Mon, 9 Nov 2015 10:46:55 +0000 Message-ID: References: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2015 10:47:17 -0000 > Hello! > not workink bhyve and windows please help me: > bhyve > -c 2 > -s 0,hostbridge > -s 3,ahci-hd,/images/win.img > -s 4,ahci-cd,/images/win_repack.iso > -s 10,virtio-net,tap0 > -s 31,lpc > -l com1,/dev/nmdm0A > -l com2,/dev/nmdm1A > -l bootrom,/path/to/BHYVE_UEFI.fd > -m 2G -H -w > windows > bhyve: invalid lpc device configuration 'bootrom,BHYVE_UEFI_20151002.fd' > https://people.freebsd.org/~grehan/bhyve_uefi/windows_install.txt Is this on FreeBSD CURRENT (FreeBSD-11)? Other people on this list will know for certain but that error suggests to = me that your version of bhyve might not have bootrom support. Matt From owner-freebsd-virtualization@freebsd.org Wed Nov 11 08:25:42 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37FCEA1F854 for ; Wed, 11 Nov 2015 08:25:42 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7E4DF1C97 for ; Wed, 11 Nov 2015 08:25:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA26529 for ; Wed, 11 Nov 2015 10:25:39 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZwQio-000G7j-VF for freebsd-virtualization@freebsd.org; Wed, 11 Nov 2015 10:25:39 +0200 To: "freebsd-virtualization@freebsd.org" From: Andriy Gapon Subject: bhyve vtblk error X-Enigmail-Draft-Status: N1110 Message-ID: <5642FB32.1040006@FreeBSD.org> Date: Wed, 11 Nov 2015 10:24:18 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2015 08:25:42 -0000 The following happened with an illumos VM in bhyve. Usually I reboot that VM by first powering it off using illumos 'poweroff' command. But yesterday I tried to do that using 'reboot' command. Apparently in that case illumos performs a warm reboot where a new kernel instance is executed without the full hardware reset and firmware boot. So, I got the following error in the guest during the reboot: Block device: blkdev@0,0, blkdev0 blkdev0 is /pci@0,0/pci1af4,2@4/blkdev@0,0 /pci@0,0/pci1af4,2@4/blkdev@0,0 (blkdev0) online WARNING: blkdev0: Invalid media block size (0) WARNING: blkdev0: Invalid media block size (0) NOTICE: Cannot read the pool label from '/pci@0,0/pci1af4,2@4/blkdev@0,0:a' NOTICE: spa_import_rootpool: error 5 Cannot mount root on /pci@0,0/pci1af4,2@4/blkdev@0,0:a fstype zfs At the same time bhyve printed the following message: vtblk: read from CFGVEC: bad size 4 P.S. There is another problem that perhaps is worth some attention as well. The guest paniced after it failed to import the root pool. bhyve got stuck in a weird way after that: vm exit[2] vm exit[1] vm exit[3] reason SVM rip 0xfffffffffb8440aa inst_length 0 exitcode 0x7f exitinfo1 0 exitinfo2 0 reason SVM rip 0xfffffffffb8440aa inst_length 0 exitcode 0x7f exitinfo1 0 exitinfo2 0 reason SVM rip 0xfffffffffb8440aa inst_length 0 exitcode 0x7f exitinfo1 0 exitinfo2 0 ^T load: 0.98 cmd: bhyve 87140 [suspended] 20861.13r 643.47u 41127.33s 0% 3727592k As you can see the process somehow became suspended and I was not able to recover it (SIGKILL didn't help) and had to reboot the host. -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Wed Nov 11 09:16:35 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59565A2B9E2 for ; Wed, 11 Nov 2015 09:16:35 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7EADB1948 for ; Wed, 11 Nov 2015 09:16:34 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA27405 for ; Wed, 11 Nov 2015 11:16:26 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZwRVy-000GBP-5h for freebsd-virtualization@freebsd.org; Wed, 11 Nov 2015 11:16:26 +0200 To: "freebsd-virtualization@freebsd.org" From: Andriy Gapon Subject: bhyve: possible apic timer problem? Message-ID: <5643072A.1070404@FreeBSD.org> Date: Wed, 11 Nov 2015 11:15:22 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2015 09:16:35 -0000 I've just booted an illumos VM and it started spewing messages like the following: pcplusmp: apic_timer_reprogram, request at 99710003906250 too far in future, current time 49867371230160 This has never happened before. Also, another reboot fixed it. -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Wed Nov 11 12:01:30 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0AF5A2CD91 for ; Wed, 11 Nov 2015 12:01:30 +0000 (UTC) (envelope-from luchengji_8@hotmail.com) Received: from BLU004-OMC4S34.hotmail.com (blu004-omc4s34.hotmail.com [65.55.111.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AFE971CEF for ; Wed, 11 Nov 2015 12:01:30 +0000 (UTC) (envelope-from luchengji_8@hotmail.com) Received: from BLU436-SMTP196 ([65.55.111.135]) by BLU004-OMC4S34.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 11 Nov 2015 04:00:23 -0800 X-TMN: [FscblxSPpirRc/7p53hYj3TKTwoGA6MW] X-Originating-Email: [luchengji_8@hotmail.com] Message-ID: From: Ray*Ban Sunglasses To: freebsd-virtualization@freebsd.org Date: Wed, 11 Nov 2015 20:00:45 +0800 Subject: Breaking:Don't miss out on our new selection of products!152 X-OriginalArrivalTime: 11 Nov 2015 12:00:22.0421 (UTC) FILETIME=[8B010C50:01D11C78] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2015 12:01:31 -0000 From owner-freebsd-virtualization@freebsd.org Thu Nov 12 10:38:17 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6262EA2D74D for ; Thu, 12 Nov 2015 10:38:17 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17BD517A5 for ; Thu, 12 Nov 2015 10:38:16 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id tACAc2eI008584 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 12 Nov 2015 10:38:05 GMT (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1447324685; bh=udGX4J5HuaIsUUwKbIV64ACh5g047SOlOR9Kj35FIPM=; h=From:To:Subject:Date; b=Rdq8cfOkFuNxyJMZ5qeBA+VzG+1iSs56NBZvz1wRnFFbJ000jl6IKGhgpNFun5U0W Z/fPVxhm7i9/2z5mHwGBmopdBAxU6YNYUhV8xtXD2X6fquKY314An/hB/uzIjhzdDo hCvsseMpvuBlUG3Bd+3kCDUdjyEiPyTWrM3zMJ0o= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 12 Nov 2015 10:37:56 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Thu, 12 Nov 2015 10:37:56 +0000 From: Matt Churchyard To: FreeBSD virtualization Subject: vm-bhyve port upgrade Thread-Topic: vm-bhyve port upgrade Thread-Index: AdEdM8JNehpOBZk9QBqvYRW9jVOGfA== Date: Thu, 12 Nov 2015 10:37:56 +0000 Message-ID: <7ac3a4411ec04711bec75cab3a68ae81@SERVER.ad.usd-group.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2015 10:38:17 -0000 Hello, For anyone interested I have submitted a PR to update the version of vm-bhy= ve in the ports tree. Primarily this fixes the off-putting, but completely benign error printed w= hen users run 'vm init' (the very first thing to run...) I've no idea if I've got the diff format right though. Also adds various small fixes, and the following changes since the last por= ts version: Command to rename a guest Configuration options for utctime, hostbridge, disk options, debug mode, cu= stom grub commands, virtual random device Snapshot and rollback commands when using ZFS Allows use of custom bridges and/or tap devices in addition to the normal a= utomated networking Ability to specify a custom path for disk devices Guests can now automatically attach correctly to virtual switches if the re= al interface(s) (and thus the bridge) are using jumbo frames Template options to specify zfs dataset/zvol properties to apply when creat= ing a guest (most useful for zvol volblocksize) New 'info' commands showing detailed guest/switch details including disk & = network usage No longer replaces dnsmasq.conf, just provides a sample config for the user= to apply if they want dhcp on a nat-enabled virtual switch. Matt (churchers) From owner-freebsd-virtualization@freebsd.org Thu Nov 12 19:35:08 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C39EA2DED0; Thu, 12 Nov 2015 19:35:08 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 7CC421075; Thu, 12 Nov 2015 19:35:08 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from AlfredMacbookAir.local (unknown [IPv6:2601:645:8004:7515:240d:167c:ca3f:1d20]) by elvis.mu.org (Postfix) with ESMTPSA id 9CCA4345A942; Thu, 12 Nov 2015 11:35:07 -0800 (PST) Subject: Re: Python on FreeBSD is slower than on Linux To: Vladimir Bogrecov , python@freebsd.org, "freebsd-virtualization@freebsd.org" References: From: Alfred Perlstein Organization: FreeBSD Message-ID: <5644E9F0.40805@freebsd.org> Date: Thu, 12 Nov 2015 11:35:12 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2015 19:35:08 -0000 I'm adding Freebsd-virtualization to this thread as both problems point to some possible issue with FreeBSD as a guest. (although a bare metal comparison should likely be done as well). -Alfred On 11/12/15 11:26 AM, Vladimir Bogrecov wrote: > Hello, > > I'm developing a little project on Python 3.5. The server's operating > system is FreeBSD 10.2. Today I decided to do a little test "just for fun" > and the result has confused me. I ran the following code > > import random > import time > > > def test_sort(size): > sequence = [i for i in range(0, size)] > random.shuffle(sequence) > start = time.time() > ordered_sequence = sorted(sequence) > print(time.time() - start) > > > if __name__ == '__main__': > test_sort(1000000) > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > (5$ per month) virtual machines on the Digital Ocean ( > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > sec, on the Debian 1.0 sec. Both machines was created specially for test > and had not any customization. Could you help me to understand why python > is so slower on FreeBSD and may be there are some steps I can perform to > speed up the python to work not slower than on Debian. > > I have found in Google the similar question: > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html so > it has an interest not only for me. > > P.S. I really like FreeBSD and I would be happy to solve this issue. If you > will have an interest to this issue I can provide SSH access for both > machines :) > > Thank You! > _______________________________________________ > freebsd-python@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-python > To unsubscribe, send any mail to "freebsd-python-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Fri Nov 13 01:36:41 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31113A2D421; Fri, 13 Nov 2015 01:36:41 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0009D1A79; Fri, 13 Nov 2015 01:36:40 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: by pasz6 with SMTP id z6so85591248pas.2; Thu, 12 Nov 2015 17:36:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:reply-to:subject:references:to:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=75bI2I2t+H5PRsNsF5L75NmMGXI1o1UXh4YKXoSNMqw=; b=Dd6sj7B7sAQYzahTNNL4uBXDR2jQuQBs8VL/jMAJ6xcZizyNx02SvY/DUNHJq3qv7j 2ppVnXYXl2BbgXiBdumOEPhK4gmnylGGr4pLxddXqb0ZjVAbEYzjjyL8JByr8EKCmefP 5qWKWvrnMiDC4GIF59ov4yNj7ecC1DjFhO+VoedQWndGwPde/T8A0GrFEY0kkNem7ka0 BQBH69nNEYKkzAuszAG6ISfm4aEHFfTiby3ZXzm1WRTeQ6VG3pEQ8SvYuDa7Fx9d/BNb DUTG1T5AHFYT3BOitOmd/npQUGzf/7Nd4tz3Dwb/3P8NMQPshHuWdyC0lBsOBvGhRVsy 0hGA== X-Received: by 10.66.220.33 with SMTP id pt1mr28115930pac.71.1447378600095; Thu, 12 Nov 2015 17:36:40 -0800 (PST) Received: from ?IPv6:2001:44b8:31ae:7b01:d06c:e63f:c1f0:dfb5? (2001-44b8-31ae-7b01-d06c-e63f-c1f0-dfb5.static.ipv6.internode.on.net. [2001:44b8:31ae:7b01:d06c:e63f:c1f0:dfb5]) by smtp.gmail.com with ESMTPSA id wq1sm16984036pbc.49.2015.11.12.17.36.38 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Nov 2015 17:36:39 -0800 (PST) Sender: Kubilay Kocak Reply-To: koobs@FreeBSD.org Subject: Re: Python on FreeBSD is slower than on Linux References: To: Vladimir Bogrecov , python@freebsd.org, freebsd-virtualization@freebsd.org From: Kubilay Kocak X-Enigmail-Draft-Status: N1110 Message-ID: <56453E9D.3040809@FreeBSD.org> Date: Fri, 13 Nov 2015 12:36:29 +1100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:42.0) Gecko/20100101 Thunderbird/42.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 01:36:41 -0000 On 13/11/2015 6:26 AM, Vladimir Bogrecov wrote: > Hello, > > I'm developing a little project on Python 3.5. The server's operating > system is FreeBSD 10.2. Today I decided to do a little test "just for fun" > and the result has confused me. I ran the following code > > import random > import time > > > def test_sort(size): > sequence = [i for i in range(0, size)] > random.shuffle(sequence) > start = time.time() > ordered_sequence = sorted(sequence) > print(time.time() - start) > > > if __name__ == '__main__': > test_sort(1000000) > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > (5$ per month) virtual machines on the Digital Ocean ( > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > sec, on the Debian 1.0 sec. Both machines was created specially for test > and had not any customization. Could you help me to understand why python > is so slower on FreeBSD and may be there are some steps I can perform to > speed up the python to work not slower than on Debian. > > I have found in Google the similar question: > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html so > it has an interest not only for me. > > P.S. I really like FreeBSD and I would be happy to solve this issue. If you > will have an interest to this issue I can provide SSH access for both > machines :) > > Thank You! >From FreeBSD Python's (team) point of view, I can't think of anything obvious off the top of my head that might cause a ~30% performance issue for that workload. Let's get a trace (truss, strace, dtrace) of what's going during the run so we can figure out exactly what's happening and in what context. With respect to the testing environment, certain VPS providers throttle bursts of CPU pretty heavily, so you'll want to account for/isolate that as a potential contributor. Yes both OS's are being run on the same provider, but as Alfred said, one OS may be mitigating/working around certain virtualisation 'issues'. A full trace of what the test case is doing is definitely the next best step I can think of, even before profiling in python, which is probably going to provide insight as well. Personally, I'd love to hear about anything that might result in FreeBSD always topping the charts for Python performance. ./koobs From owner-freebsd-virtualization@freebsd.org Fri Nov 13 08:02:03 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D2C1A2DAF9; Fri, 13 Nov 2015 08:02:03 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5F1C1AD5; Fri, 13 Nov 2015 08:02:02 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by wmec201 with SMTP id c201so68781851wme.0; Fri, 13 Nov 2015 00:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=dAiKPp+eGvhdA6t2xfoWW+f5N0pM1azLbjVpfeVzb2U=; b=vEZ1fUOAu0MqB7LhBH+GJkuW1d/aBMqbJaQRSXMbHYt48CVtIy5AMqXLvNSDi4HoUt 1jTdsBHwoGZZzibvLtveH+dlrMhtgD9HUjlmfdwnI6GlTuMu4TcEgc4PAQS8agLfkh7o XGwqI0mgzqErtwC71aUcCzYd35qsdB593E6KRSlQfHBshc9vMMS6AJHXgDnVm55ulRjn D60pJJUu5ZHMGyYHjboq7is6FSNz0E+DjHTuK7MC7yaK/3SnLsGYem8PrlnggdeuBFvh jSiRZXgayVoUfDq/H+Ade09ZoAhA0GCjki5b9S9gGI0k0XXUkB2t6zHnkrobCWZLryJP eAVw== X-Received: by 10.28.63.22 with SMTP id m22mr2196960wma.58.1447401720514; Fri, 13 Nov 2015 00:02:00 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id q1sm18573643wjy.31.2015.11.13.00.01.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Nov 2015 00:01:59 -0800 (PST) Sender: Baptiste Daroussin Date: Fri, 13 Nov 2015 09:01:57 +0100 From: Baptiste Daroussin To: Kubilay Kocak Cc: Vladimir Bogrecov , python@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Python on FreeBSD is slower than on Linux Message-ID: <20151113080157.GA93991@ivaldir.etoilebsd.net> References: <56453E9D.3040809@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <56453E9D.3040809@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 08:02:03 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 13, 2015 at 12:36:29PM +1100, Kubilay Kocak wrote: > On 13/11/2015 6:26 AM, Vladimir Bogrecov wrote: > > Hello, > >=20 > > I'm developing a little project on Python 3.5. The server's operating > > system is FreeBSD 10.2. Today I decided to do a little test "just for f= un" > > and the result has confused me. I ran the following code > >=20 > > import random > > import time > >=20 > >=20 > > def test_sort(size): > > sequence =3D [i for i in range(0, size)] > > random.shuffle(sequence) > > start =3D time.time() > > ordered_sequence =3D sorted(sequence) > > print(time.time() - start) > >=20 > >=20 > > if __name__ =3D=3D '__main__': > > test_sort(1000000) > >=20 > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > > (5$ per month) virtual machines on the Digital Ocean ( > > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > > sec, on the Debian 1.0 sec. Both machines was created specially for test > > and had not any customization. Could you help me to understand why pyth= on > > is so slower on FreeBSD and may be there are some steps I can perform to > > speed up the python to work not slower than on Debian. > >=20 > > I have found in Google the similar question: > > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.htm= l so > > it has an interest not only for me. > >=20 > > P.S. I really like FreeBSD and I would be happy to solve this issue. If= you > > will have an interest to this issue I can provide SSH access for both > > machines :) > >=20 > > Thank You! >=20 > From FreeBSD Python's (team) point of view, I can't think of anything > obvious off the top of my head that might cause a ~30% performance issue > for that workload. >=20 > Let's get a trace (truss, strace, dtrace) of what's going during the run > so we can figure out exactly what's happening and in what context. >=20 > With respect to the testing environment, certain VPS providers throttle > bursts of CPU pretty heavily, so you'll want to account for/isolate that > as a potential contributor. Yes both OS's are being run on the same > provider, but as Alfred said, one OS may be mitigating/working around > certain virtualisation 'issues'. >=20 > A full trace of what the test case is doing is definitely the next best > step I can think of, even before profiling in python, which is probably > going to provide insight as well. >=20 > Personally, I'd love to hear about anything that might result in FreeBSD > always topping the charts for Python performance. >=20 Well the python devs are aware by themselves of potential performances issu= es on FreeBSD (and non linux in general) for example subprocess will try to close= fds, on linux by getting the list of fd from /proc/fd and only close the one the= y do not want among the existing ones. on freebsd they do the same if /dev/fd is mounted meaning without /dev/fd, perfs will suck. They do not use closefrom= (2) here because on linux it is not async-signal-safe. one could make them use closefrom(2) on non linux for example or even more efficiently but freebsd = only modify the code to use kinfo_getfile(3). https://bugs.python.org/issue11284 Another area is the AIO iirc (needs to be double checked) the python uses l= inux only things for aio which makes this way slower on FreeBSD. I'm kind of surprised given the number of pythonic people we have that no o= ne has had a look at how python perform on FreeBSD and how things are implemen= ted in the python VM to help them. Bapt --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlZFmPUACgkQ8kTtMUmk6EzWBQCfZhE1Y9Dk68y2EZgyXViTe+ac cp8An3S8uaHRkqOfP5cCVK5TuYcD1Zug =66lI -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM-- From owner-freebsd-virtualization@freebsd.org Fri Nov 13 08:15:20 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1B20A2DDAF; Fri, 13 Nov 2015 08:15:20 +0000 (UTC) (envelope-from mark@exonetric.com) Received: from relay.exonetric.net (relay0.exonetric.net [178.250.72.161]) by mx1.freebsd.org (Postfix) with ESMTP id DDB311F78; Fri, 13 Nov 2015 08:15:19 +0000 (UTC) (envelope-from mark@exonetric.com) Received: from [10.8.225.155] (188.29.164.199.threembb.co.uk [188.29.164.199]) by relay.exonetric.net (Postfix) with ESMTPSA id BCDAA2BE9A; Fri, 13 Nov 2015 08:08:11 +0000 (GMT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Python on FreeBSD is slower than on Linux From: Mark Blackman X-Mailer: iPhone Mail (13B143) In-Reply-To: <5644E9F0.40805@freebsd.org> Date: Fri, 13 Nov 2015 08:08:10 +0000 Cc: Vladimir Bogrecov , python@freebsd.org, "freebsd-virtualization@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <5FA6299E-962D-4F4B-B4AB-B2BEF1E3915B@exonetric.com> References: <5644E9F0.40805@freebsd.org> To: Alfred Perlstein X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 08:15:20 -0000 On 12 Nov 2015, at 19:35, Alfred Perlstein wrote: >=20 > I'm adding Freebsd-virtualization to this thread as both problems point to= some possible issue with FreeBSD as a guest. (although a bare metal compar= ison should likely be done as well). >=20 > -Alfred >=20 >> On 11/12/15 11:26 AM, Vladimir Bogrecov wrote: >> Hello, >>=20 >> I'm developing a little project on Python 3.5. The server's operating >> system is FreeBSD 10.2. Today I decided to do a little test "just for fun= " >> and the result has confused me. I ran the following code >>=20 >> import random >> import time >>=20 >>=20 >> def test_sort(size): >> sequence =3D [i for i in range(0, size)] >> random.shuffle(sequence) >> start =3D time.time() >> ordered_sequence =3D sorted(sequence) >> print(time.time() - start) >>=20 >>=20 >> if __name__ =3D=3D '__main__': >> test_sort(1000000) >>=20 >> on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest >> (5$ per month) virtual machines on the Digital Ocean ( >> https://www.digitalocean.com). The average result on the FreeBSD was 1.5 >> sec, on the Debian 1.0 sec. Both machines was created specially for test >> and had not any customization. Could you help me to understand why python= >> is so slower on FreeBSD and may be there are some steps I can perform to >> speed up the python to work not slower than on Debian. >>=20 >> I have found in Google the similar question: >> https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html s= o >> it has an interest not only for me. >>=20 >> P.S. I really like FreeBSD and I would be happy to solve this issue. If y= ou >> will have an interest to this issue I can provide SSH access for both >> machines :) >>=20 >> Thank You! I have some memory that the gettimeofday is quite expensive on FreeBSD as a r= esult of substantially more accuracy and I reckon that test script is callin= g it about 2 million times. Mark= From owner-freebsd-virtualization@freebsd.org Fri Nov 13 08:23:44 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD985A2B09C; Fri, 13 Nov 2015 08:23:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 83711155E; Fri, 13 Nov 2015 08:23:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tAD8NcJC059240 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 13 Nov 2015 10:23:38 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tAD8NcJC059240 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tAD8Ncbn059239; Fri, 13 Nov 2015 10:23:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 13 Nov 2015 10:23:38 +0200 From: Konstantin Belousov To: Baptiste Daroussin Cc: Kubilay Kocak , Vladimir Bogrecov , python@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Python on FreeBSD is slower than on Linux Message-ID: <20151113082338.GT2257@kib.kiev.ua> References: <56453E9D.3040809@FreeBSD.org> <20151113080157.GA93991@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151113080157.GA93991@ivaldir.etoilebsd.net> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 08:23:45 -0000 On Fri, Nov 13, 2015 at 09:01:57AM +0100, Baptiste Daroussin wrote: > On Fri, Nov 13, 2015 at 12:36:29PM +1100, Kubilay Kocak wrote: > > On 13/11/2015 6:26 AM, Vladimir Bogrecov wrote: > > > Hello, > > > > > > I'm developing a little project on Python 3.5. The server's operating > > > system is FreeBSD 10.2. Today I decided to do a little test "just for fun" > > > and the result has confused me. I ran the following code > > > > > > import random > > > import time > > > > > > > > > def test_sort(size): > > > sequence = [i for i in range(0, size)] > > > random.shuffle(sequence) > > > start = time.time() > > > ordered_sequence = sorted(sequence) > > > print(time.time() - start) > > > > > > > > > if __name__ == '__main__': > > > test_sort(1000000) > > > > > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > > > (5$ per month) virtual machines on the Digital Ocean ( > > > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > > > sec, on the Debian 1.0 sec. Both machines was created specially for test > > > and had not any customization. Could you help me to understand why python > > > is so slower on FreeBSD and may be there are some steps I can perform to > > > speed up the python to work not slower than on Debian. > > > > > > I have found in Google the similar question: > > > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html so > > > it has an interest not only for me. > > > > > > P.S. I really like FreeBSD and I would be happy to solve this issue. If you > > > will have an interest to this issue I can provide SSH access for both > > > machines :) > > > > > > Thank You! > > > > From FreeBSD Python's (team) point of view, I can't think of anything > > obvious off the top of my head that might cause a ~30% performance issue > > for that workload. > > > > Let's get a trace (truss, strace, dtrace) of what's going during the run > > so we can figure out exactly what's happening and in what context. > > > > With respect to the testing environment, certain VPS providers throttle > > bursts of CPU pretty heavily, so you'll want to account for/isolate that > > as a potential contributor. Yes both OS's are being run on the same > > provider, but as Alfred said, one OS may be mitigating/working around > > certain virtualisation 'issues'. > > > > A full trace of what the test case is doing is definitely the next best > > step I can think of, even before profiling in python, which is probably > > going to provide insight as well. > > > > Personally, I'd love to hear about anything that might result in FreeBSD > > always topping the charts for Python performance. > > > Well the python devs are aware by themselves of potential performances issues on > FreeBSD (and non linux in general) for example subprocess will try to close fds, > on linux by getting the list of fd from /proc/fd and only close the one they do > not want among the existing ones. on freebsd they do the same if /dev/fd is > mounted meaning without /dev/fd, perfs will suck. They do not use closefrom(2) > here because on linux it is not async-signal-safe. one could make them use > closefrom(2) on non linux for example or even more efficiently but freebsd only > modify the code to use kinfo_getfile(3). > > https://bugs.python.org/issue11284 > > Another area is the AIO iirc (needs to be double checked) the python uses linux > only things for aio which makes this way slower on FreeBSD. > > I'm kind of surprised given the number of pythonic people we have that no one > has had a look at how python perform on FreeBSD and how things are implemented > in the python VM to help them. Note that the code provided does not do any system actions at all. It is, I guess, is pure calculation and probably memory allocation. The later, for the initial warm-up, may have different constant cost between different implementations of malloc/operating system/policy put by hypervisor on the OS access patterns to memory. In other words, to meaningfully compare apples to apples, the testing must isolate each variadic part. Run the tests on the same _real_ hardware, provide the warm-up to isolate the initialization cost, do statistically-meaningful analysis. Do trace the test e.g. using ktrace to see the program<->system iteration, in particular, ktrace allows to see the page faults experienced by the execution. From owner-freebsd-virtualization@freebsd.org Fri Nov 13 08:49:12 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AAD9A2B937; Fri, 13 Nov 2015 08:49:12 +0000 (UTC) (envelope-from mark@exonetric.com) Received: from relay.exonetric.net (relay0.exonetric.net [178.250.72.161]) by mx1.freebsd.org (Postfix) with ESMTP id 794E41603; Fri, 13 Nov 2015 08:49:11 +0000 (UTC) (envelope-from mark@exonetric.com) Received: from [10.11.49.217] (188.29.164.157.threembb.co.uk [188.29.164.157]) by relay.exonetric.net (Postfix) with ESMTPSA id A8E462BEA3; Fri, 13 Nov 2015 08:49:10 +0000 (GMT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Python on FreeBSD is slower than on Linux From: Mark Blackman X-Mailer: iPhone Mail (13B143) In-Reply-To: <5FA6299E-962D-4F4B-B4AB-B2BEF1E3915B@exonetric.com> Date: Fri, 13 Nov 2015 08:49:09 +0000 Cc: Vladimir Bogrecov , python@freebsd.org, "freebsd-virtualization@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <748D249D-51E8-4D56-8331-C9A1408218FB@exonetric.com> References: <5644E9F0.40805@freebsd.org> <5FA6299E-962D-4F4B-B4AB-B2BEF1E3915B@exonetric.com> To: Alfred Perlstein X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 08:49:12 -0000 > On 13 Nov 2015, at 08:08, Mark Blackman wrote: >=20 >> On 12 Nov 2015, at 19:35, Alfred Perlstein wrote: >>=20 >> I'm adding Freebsd-virtualization to this thread as both problems point t= o some possible issue with FreeBSD as a guest. (although a bare metal compa= rison should likely be done as well). >>=20 >> -Alfred >>=20 >>> On 11/12/15 11:26 AM, Vladimir Bogrecov wrote: >>> Hello, >>>=20 >>> I'm developing a little project on Python 3.5. The server's operating >>> system is FreeBSD 10.2. Today I decided to do a little test "just for fu= n" >>> and the result has confused me. I ran the following code >>>=20 >>> import random >>> import time >>>=20 >>>=20 >>> def test_sort(size): >>> sequence =3D [i for i in range(0, size)] >>> random.shuffle(sequence) >>> start =3D time.time() >>> ordered_sequence =3D sorted(sequence) >>> print(time.time() - start) >>>=20 >>>=20 >>> if __name__ =3D=3D '__main__': >>> test_sort(1000000) >>>=20 >>> on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest= >>> (5$ per month) virtual machines on the Digital Ocean ( >>> https://www.digitalocean.com). The average result on the FreeBSD was 1.5= >>> sec, on the Debian 1.0 sec. Both machines was created specially for test= >>> and had not any customization. Could you help me to understand why pytho= n >>> is so slower on FreeBSD and may be there are some steps I can perform to= >>> speed up the python to work not slower than on Debian. >>>=20 >>> I have found in Google the similar question: >>> https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html= so >>> it has an interest not only for me. >>>=20 >>> P.S. I really like FreeBSD and I would be happy to solve this issue. If y= ou >>> will have an interest to this issue I can provide SSH access for both >>> machines :) >>>=20 >>> Thank You! >=20 > I have some memory that the gettimeofday is quite expensive on FreeBSD as a= result of substantially more accuracy and I reckon that test script is call= ing it about 2 million times. Doh, never mind, misread the python. Just twice. :)= From owner-freebsd-virtualization@freebsd.org Fri Nov 13 15:32:15 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B03FA2E7DC; Fri, 13 Nov 2015 15:32:15 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD5314CD; Fri, 13 Nov 2015 15:32:15 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from [IPv6:2601:645:8004:7515:3999:ff28:f8c3:c361] (unknown [IPv6:2601:645:8004:7515:3999:ff28:f8c3:c361]) by elvis.mu.org (Postfix) with ESMTPSA id AB082345A916; Fri, 13 Nov 2015 07:32:13 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Python on FreeBSD is slower than on Linux From: Alfred Perlstein X-Mailer: iPhone Mail (13B143) In-Reply-To: <748D249D-51E8-4D56-8331-C9A1408218FB@exonetric.com> Date: Fri, 13 Nov 2015 07:32:14 -0800 Cc: Vladimir Bogrecov , python@freebsd.org, "freebsd-virtualization@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <16AC997E-EE0B-4906-849D-98187EBA14F4@freebsd.org> References: <5644E9F0.40805@freebsd.org> <5FA6299E-962D-4F4B-B4AB-B2BEF1E3915B@exonetric.com> <748D249D-51E8-4D56-8331-C9A1408218FB@exonetric.com> To: Mark Blackman X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 15:32:15 -0000 Vladimir , Please run truss(1) against the python code and paste a subset here. Maybe i= t is doing many semaphore ops.=20 Sent from my iPhone > On Nov 13, 2015, at 12:49 AM, Mark Blackman wrote: >=20 > Vladimir From owner-freebsd-virtualization@freebsd.org Fri Nov 13 17:29:18 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 452D2A2E525 for ; Fri, 13 Nov 2015 17:29:18 +0000 (UTC) (envelope-from matthieu.cerda@gmail.com) Received: from mail-lf0-x236.google.com (mail-lf0-x236.google.com [IPv6:2a00:1450:4010:c07::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACE50146B for ; Fri, 13 Nov 2015 17:29:17 +0000 (UTC) (envelope-from matthieu.cerda@gmail.com) Received: by lfs39 with SMTP id 39so56467584lfs.3 for ; Fri, 13 Nov 2015 09:29:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=AX+fffgEKnWj0kWh1cYen7FwxJBUNFcbMDSFroSYopg=; b=fliC0ysDp2StTmGF8UjhogUp2l01DfUB0X/9mGW0rpV+YVdDcs+60qW7j1q8otEik6 6ZdwipIGh2HxxIOpKXwxs74KFvUwVOno8Dm+RtUKkrAwyDbvN5bVYbblRpw318OLtyH0 5nGa1yesGPAbmNVZ5CzkA0d5LoH9kRVjp2tLlo67erSTI6hIsZLvULCmWOqwAQIws7Oe cn4vRbL8MiuuKc7e10ccmn9R3b4quwr+C7M0wrHDsSy0JB3hXF/qLsKasSld89moFYMz abSDZQevLEzY6CJokVxWXFiXcpjNcD789xPurPlZTrCb6rI9OAdwBs8iR1d2uzHCSTmA 8PjA== MIME-Version: 1.0 X-Received: by 10.25.156.5 with SMTP id f5mr10729777lfe.46.1447435755645; Fri, 13 Nov 2015 09:29:15 -0800 (PST) Received: by 10.25.164.70 with HTTP; Fri, 13 Nov 2015 09:29:15 -0800 (PST) Date: Fri, 13 Nov 2015 18:29:15 +0100 Message-ID: Subject: Issue with Virtualbox OSE 4.3.32 SATA/AHCI on 11-CURRENT From: Matthieu CERDA To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 17:29:18 -0000 Hello folks! I'm having a little bit of trouble here trying to run Virtualbox 4.3.32 on a 11-CURRENT installation (r290644) Everytime I try to install a GNU/Linux OS (the test case was installing a Ubuntu VM from the Live CD, but it also occurs with a Debian netinstall) with the default SATA disk emulation, the VM stalls as soon as disk I/O are needed on the virtualized disk (the virtual CD from an ISO runs all right). The VM VBox.log gives: ---8<--- (...) 00:02:41.496257 AioMgr0-N: Request 0x0000081025e280 failed with rc=VERR_TRY_AGAIN, migrating endpoint /home/user/VirtualBox VMs/Ubuntu/Ubuntu.vdi to failsafe manager. 00:02:41.692796 AIOMgr: I/O manager 0x000008180623a0 encountered a critical error (rc=VERR_FILE_AIO_NO_REQUEST) during operation. Falling back to failsafe mode. Expect reduced performance 00:02:41.692813 AIOMgr: Error happened in /wrkdirs/usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.32/src/VBox/VMM/VMMR3/PDMAsyncCompletionFileNormal.cpp:(1664){int pdmacFileAioMgrNormal(RTTHREAD, void *)} 00:02:41.692819 AIOMgr: Please contact the product vendor 00:03:12.298528 AHCI#0: Port 0 reset 00:03:12.306370 AHCI#0P0: Cancelled task 0 00:03:12.306383 AHCI#0P0: Cancelled task 1 00:03:12.306388 AHCI#0P0: Cancelled task 2 00:03:12.306391 AHCI#0P0: Cancelled task 3 00:03:12.306406 AHCI#0P0: Cancelled task 4 00:03:12.306409 AHCI#0P0: Cancelled task 5 00:03:12.306413 AHCI#0P0: Cancelled task 6 00:03:12.306416 AHCI#0P0: Cancelled task 7 00:03:12.306419 AHCI#0P0: Cancelled task 8 00:03:12.306422 AHCI#0P0: Cancelled task 9 00:03:12.306426 AHCI#0P0: Cancelled task 10 00:03:12.306429 AHCI#0P0: Cancelled task 11 00:03:12.306432 AHCI#0P0: Cancelled task 12 00:03:12.306434 AHCI#0P0: Cancelled task 13 00:03:12.306437 AHCI#0P0: Cancelled task 14 00:03:12.306439 AHCI#0P0: Cancelled task 15 00:03:12.306441 AHCI#0P0: Cancelled task 16 00:03:12.306444 AHCI#0P0: Cancelled task 17 00:03:12.306446 AHCI#0P0: Cancelled task 18 00:03:12.306448 AHCI#0P0: Cancelled task 19 00:03:12.306451 AHCI#0P0: Cancelled task 20 00:03:12.306453 AHCI#0P0: Cancelled task 21 00:03:12.306455 AHCI#0P0: Cancelled task 22 00:03:12.306458 AHCI#0P0: Cancelled task 24 00:03:12.306460 AHCI#0P0: Cancelled task 25 00:03:12.306462 AHCI#0P0: Cancelled task 26 00:03:12.306465 AHCI#0P0: Cancelled task 27 00:03:12.306467 AHCI#0P0: Cancelled task 28 00:03:12.306470 AHCI#0P0: Cancelled task 29 00:03:12.306472 AHCI#0P0: Cancelled task 30 00:03:43.322453 AHCI#0: Port 0 reset 00:03:43.330286 AHCI#0P0: Cancelled task 0 00:03:43.330298 AHCI#0P0: Cancelled task 1 00:03:43.330323 AHCI#0P0: Cancelled task 2 (...) The resets go on and on until I shoot the VM down... Using the IDE backend seems to work (at least the VM did boot and began to install properly, I'm waiting for it to complete) I'm not blocked since a workaround exists, but I feel that the default backend storage that Virtualbox selected for me should normally misbehave like this. Is there a know way to debug this kind of issues? Thanks in advance, and have a nice week end! From owner-freebsd-virtualization@freebsd.org Fri Nov 13 22:57:42 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A415BA2E531; Fri, 13 Nov 2015 22:57:42 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2E7B91564; Fri, 13 Nov 2015 22:57:42 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from AlfredMacbookAir.local (unknown [IPv6:2601:645:8004:7515:2c06:4fbd:2d2:ee65]) by elvis.mu.org (Postfix) with ESMTPSA id 1B592345A916; Fri, 13 Nov 2015 14:57:36 -0800 (PST) Subject: Re: Python on FreeBSD is slower than on Linux To: Baptiste Daroussin , Kubilay Kocak References: <56453E9D.3040809@FreeBSD.org> <20151113080157.GA93991@ivaldir.etoilebsd.net> Cc: Vladimir Bogrecov , python@freebsd.org, freebsd-virtualization@freebsd.org From: Alfred Perlstein Organization: FreeBSD Message-ID: <56466ADF.5020508@freebsd.org> Date: Fri, 13 Nov 2015 14:57:35 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151113080157.GA93991@ivaldir.etoilebsd.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2015 22:57:42 -0000 On 11/13/15 12:01 AM, Baptiste Daroussin wrote: > I'm kind of surprised given the number of pythonic people we have that > no one has had a look at how python perform on FreeBSD and how things > are implemented in the python VM to help them. Bapt Did this recently in a few places however not in this one. Having a syscall to give the list of open fds would be pretty a++. -Alfred From owner-freebsd-virtualization@freebsd.org Sat Nov 14 00:21:39 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C7D1A2EDDC; Sat, 14 Nov 2015 00:21:39 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C2E441DF6; Sat, 14 Nov 2015 00:21:38 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id tAE0Lb8O050864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 13 Nov 2015 16:21:37 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id tAE0LbUW050863; Fri, 13 Nov 2015 16:21:37 -0800 (PST) (envelope-from jmg) Date: Fri, 13 Nov 2015 16:21:37 -0800 From: John-Mark Gurney To: Alfred Perlstein Cc: Vladimir Bogrecov , python@freebsd.org, "freebsd-virtualization@freebsd.org" Subject: Re: Python on FreeBSD is slower than on Linux Message-ID: <20151114002137.GB65715@funkthat.com> References: <5644E9F0.40805@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5644E9F0.40805@freebsd.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Fri, 13 Nov 2015 16:21:37 -0800 (PST) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2015 00:21:39 -0000 Alfred Perlstein wrote this message on Thu, Nov 12, 2015 at 11:35 -0800: > I'm adding Freebsd-virtualization to this thread as both problems point > to some possible issue with FreeBSD as a guest. (although a bare metal > comparison should likely be done as well). This could simply be a python compiled w/ different compiler and compiler optimization flags issue... One easy way to eliminate FreeBSD is to take the Linux version of python, and run it on FreeBSD using the linux emulator to help ensure that the performance is the same when running the same binary on the two OS's... > On 11/12/15 11:26 AM, Vladimir Bogrecov wrote: > > Hello, > > > > I'm developing a little project on Python 3.5. The server's operating > > system is FreeBSD 10.2. Today I decided to do a little test "just for fun" > > and the result has confused me. I ran the following code > > > > import random > > import time > > > > > > def test_sort(size): > > sequence = [i for i in range(0, size)] > > random.shuffle(sequence) > > start = time.time() > > ordered_sequence = sorted(sequence) > > print(time.time() - start) > > > > > > if __name__ == '__main__': > > test_sort(1000000) > > > > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest > > (5$ per month) virtual machines on the Digital Ocean ( > > https://www.digitalocean.com). The average result on the FreeBSD was 1.5 > > sec, on the Debian 1.0 sec. Both machines was created specially for test > > and had not any customization. Could you help me to understand why python > > is so slower on FreeBSD and may be there are some steps I can perform to > > speed up the python to work not slower than on Debian. > > > > I have found in Google the similar question: > > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html so > > it has an interest not only for me. > > > > P.S. I really like FreeBSD and I would be happy to solve this issue. If you > > will have an interest to this issue I can provide SSH access for both > > machines :) -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."