From owner-freebsd-xen@freebsd.org Mon Aug 15 15:56:08 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAFC1BBAE17 for ; Mon, 15 Aug 2016 15:56:08 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 416C917DC for ; Mon, 15 Aug 2016 15:56:07 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Mon, 15 Aug 2016 17:54:55 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 057937A9-93D9-4763-BB2F-9EF358C438BE.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Mon, 15 Aug 2016 17:54:53 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 15 Aug 2016 17:54:52 +0200 From: rainer@ultra-secure.de To: freebsd-xen@freebsd.org Subject: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 12, bad: 0, connections: 12, history: 12, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 15:56:08 -0000 Hi, I've got a problem. For a customer, I run a VM in Xen that should perform a certain task in PHP (written using the ZendFrameWork). That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB 5.5.0, php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory The "reference" server that the customer uses is somewhere else and manages to perform the same task in 3s. I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it takes about 9s. I've tried it on physical hardware with 10.3, PHP5.5, MariaDB 5.5 and it also takes about 9s (that machine hosts a load of other sites but has lot of cores and memory available). Then, I've installed an Ubuntu 14 VM in XenServer. It comes with PHP5.5 and MariaDB 5.5 by default. It's VM with 2vCPUs and 8GB RAM. There, the script take about 9s, too (just as if it was running on physical FreeBSD). So, is this expected? Did I do something wrong? We don't run XenServer directly, but use it as part of an Apache CloudStack "Private Cloud". Version is 6.5 SP-something (will have to ask if that is important). The template I use for FreeBSD installation I created myself, by installing from an ISO and selection "FreeBSD 10" as OS. Originally, it was using ZFS for the database-directory, but I moved that to UFS (didn't really lead to a performance break-through, though). What else can I do? Best Regards, Rainer From owner-freebsd-xen@freebsd.org Tue Aug 16 08:56:09 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A315BBA4E2 for ; Tue, 16 Aug 2016 08:56:09 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F20A1979 for ; Tue, 16 Aug 2016 08:56:08 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="380349603" Date: Tue, 16 Aug 2016 10:54:55 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160816085455.46a5slqsbgauod5t@mac> References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 08:56:09 -0000 On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer@ultra-secure.de wrote: > Hi, > > I've got a problem. > Hello, > > For a customer, I run a VM in Xen that should perform a certain task in PHP > (written using the ZendFrameWork). > > That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB 5.5.0, > php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory > The "reference" server that the customer uses is somewhere else and manages > to perform the same task in 3s. > > I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it takes > about 9s. In the sentence above, are you running it in a Xen VM or on bare metal? > I've tried it on physical hardware with 10.3, PHP5.5, MariaDB 5.5 and it > also takes about 9s (that machine hosts a load of other sites but has lot of > cores and memory available). > > > Then, I've installed an Ubuntu 14 VM in XenServer. It comes with PHP5.5 and > MariaDB 5.5 by default. It's VM with 2vCPUs and 8GB RAM. > > There, the script take about 9s, too (just as if it was running on physical > FreeBSD). I'm not sure I understood your problem right, is it that FreeBSD on Xen always takes 18-20s to perform a task while on bare metal it only takes ~9s? If that's the case, I would recommend that you first try to disable PV disks and nics, by adding the following to your /boot/loader.conf: hw.xen.disable_pv_disks=1 hw.xen.disable_pv_nics=1 If that still yelds the same performance (or worse), then you could still try to disable all Xen code, by removing: options XENHVM device xenpci >From you kernel config and recompiling the kernel. Roger. From owner-freebsd-xen@freebsd.org Tue Aug 16 09:29:49 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25A1DBBB530 for ; Tue, 16 Aug 2016 09:29:49 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 9ED731304 for ; Tue, 16 Aug 2016 09:29:48 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 11:29:46 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 6FF2FC54-DEFC-4FBF-A720-DFCA948EB875.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 11:29:43 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 11:29:43 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816085455.46a5slqsbgauod5t@mac> References: <20160816085455.46a5slqsbgauod5t@mac> Message-ID: <2a0a5ae2821551935de329b8665834be@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 14, bad: 0, connections: 14, history: 14, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 09:29:49 -0000 Am 2016-08-16 10:54, schrieb Roger Pau Monné: > On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer@ultra-secure.de wrote: >> Hi, >> >> I've got a problem. >> > > Hello, > >> >> For a customer, I run a VM in Xen that should perform a certain task >> in PHP >> (written using the ZendFrameWork). >> >> That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB >> 5.5.0, >> php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory >> The "reference" server that the customer uses is somewhere else and >> manages >> to perform the same task in 3s. >> >> I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it >> takes >> about 9s. > > In the sentence above, are you running it in a Xen VM or on bare metal? > This is both Xen. I think the customer is also running it on some sort of virtualization. >> I've tried it on physical hardware with 10.3, PHP5.5, MariaDB 5.5 and >> it >> also takes about 9s (that machine hosts a load of other sites but has >> lot of >> cores and memory available). >> >> >> Then, I've installed an Ubuntu 14 VM in XenServer. It comes with >> PHP5.5 and >> MariaDB 5.5 by default. It's VM with 2vCPUs and 8GB RAM. >> >> There, the script take about 9s, too (just as if it was running on >> physical >> FreeBSD). > > I'm not sure I understood your problem right, is it that FreeBSD on Xen > always takes 18-20s to perform a task while on bare metal it only takes > ~9s? Well, that in itself is not the problem. The problem is that Linux on Xen is as fast as my bare metal (incidentally, both the physical Xen host (Dom0) and the physical server I ran the script for comparison are the same hardware). > If that's the case, I would recommend that you first try to disable PV > disks > and nics, by adding the following to your /boot/loader.conf: > > hw.xen.disable_pv_disks=1 > hw.xen.disable_pv_nics=1 OK, can I still boot the VM with this or will NICs and disks show up as different devices then? > If that still yelds the same performance (or worse), then you could > still > try to disable all Xen code, by removing: > > options XENHVM > device xenpci > > From you kernel config and recompiling the kernel. I'm using stock FreeBSD 10.3. I was under the assumption that this is the "optimal" configuration. Regards Rainer From owner-freebsd-xen@freebsd.org Tue Aug 16 10:01:26 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22989BBC335 for ; Tue, 16 Aug 2016 10:01:26 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 88A9816AA for ; Tue, 16 Aug 2016 10:01:25 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 12:01:23 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 271A2E59-17CD-42EF-A1F7-8D7C3ED27296.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 12:01:13 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 16 Aug 2016 12:01:12 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816085455.46a5slqsbgauod5t@mac> References: <20160816085455.46a5slqsbgauod5t@mac> Message-ID: <006343a9483a8ed8c3219dfaeb321971@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 15, bad: 0, connections: 15, history: 15, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 10:01:26 -0000 Just a little update. Using FreeBSD 11.0-RC1 (amd64) on a 2vCPU 8GB VM, I can complete the task in 12s. However, using PHP 5.6 and MySQL 5.6 from FreeBSD's pkg-repo. I don't have packages for FreeBSD 11, yet, and the default versions in the official packages are different than mine - my poudriere is still on FreeBSD 10.1. So, it's unclear if FreeBSD 11 is actually faster. From owner-freebsd-xen@freebsd.org Tue Aug 16 10:06:09 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14A21BBC47E for ; Tue, 16 Aug 2016 10:06:09 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A271119DA for ; Tue, 16 Aug 2016 10:06:08 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="380359575" Date: Tue, 16 Aug 2016 12:06:05 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160816100605.la63x2ju5bmtdqhl@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2a0a5ae2821551935de329b8665834be@ultra-secure.de> User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 10:06:09 -0000 On Tue, Aug 16, 2016 at 11:29:43AM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-16 10:54, schrieb Roger Pau Monné: > > On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer@ultra-secure.de wrote: > > > Hi, > > > > > > I've got a problem. > > > > > > > Hello, > > > > > > > > For a customer, I run a VM in Xen that should perform a certain task > > > in PHP > > > (written using the ZendFrameWork). > > > > > > That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB > > > 5.5.0, > > > php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory > > > The "reference" server that the customer uses is somewhere else and > > > manages > > > to perform the same task in 3s. > > > > > > I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it > > > takes > > > about 9s. > > > > In the sentence above, are you running it in a Xen VM or on bare metal? > > > > This is both Xen. > I think the customer is also running it on some sort of virtualization. Hm, so a given workload on Xen takes ~9s, and it also takes ~9s when run on bare metal FreeBSD, is that right? > > > > I've tried it on physical hardware with 10.3, PHP5.5, MariaDB 5.5 > > > and it > > > also takes about 9s (that machine hosts a load of other sites but > > > has lot of > > > cores and memory available). > > > > > > > > > Then, I've installed an Ubuntu 14 VM in XenServer. It comes with > > > PHP5.5 and > > > MariaDB 5.5 by default. It's VM with 2vCPUs and 8GB RAM. > > > > > > There, the script take about 9s, too (just as if it was running on > > > physical > > > FreeBSD). > > > > I'm not sure I understood your problem right, is it that FreeBSD on Xen > > always takes 18-20s to perform a task while on bare metal it only takes > > ~9s? > > > Well, that in itself is not the problem. > The problem is that Linux on Xen is as fast as my bare metal (incidentally, > both the physical Xen host (Dom0) and the physical server I ran the script > for comparison are the same hardware). But from your description above I take that you get the same performance when running FreeBSD on Xen or when running FreeBSD on bare metal, in which case I'm not sure if this problem is related to Xen at all. > > > If that's the case, I would recommend that you first try to disable PV > > disks > > and nics, by adding the following to your /boot/loader.conf: > > > > hw.xen.disable_pv_disks=1 > > hw.xen.disable_pv_nics=1 > > OK, can I still boot the VM with this or will NICs and disks show up as > different devices then? NIC will show up as "re", disks as "ada" (which is what you already have). > > > If that still yelds the same performance (or worse), then you could > > still > > try to disable all Xen code, by removing: > > > > options XENHVM > > device xenpci > > > > From you kernel config and recompiling the kernel. > > I'm using stock FreeBSD 10.3. > I was under the assumption that this is the "optimal" configuration. It should be, I'm just trying to figure out if there's something there that's hampering performance. But please read above because it's still not clear to me that this is related to Xen. Roger. From owner-freebsd-xen@freebsd.org Tue Aug 16 10:14:00 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAED5BBC627 for ; Tue, 16 Aug 2016 10:14:00 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 4358A1DBA for ; Tue, 16 Aug 2016 10:13:59 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 12:13:58 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 31845A37-23BF-4C67-9657-73A6988BD8FB.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 12:13:56 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 12:13:55 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816100605.la63x2ju5bmtdqhl@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 16, bad: 0, connections: 16, history: 16, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 10:14:01 -0000 Am 2016-08-16 12:06, schrieb Roger Pau Monné: > On Tue, Aug 16, 2016 at 11:29:43AM +0200, rainer@ultra-secure.de wrote: >> Am 2016-08-16 10:54, schrieb Roger Pau Monné: >> > On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer@ultra-secure.de wrote: >> > > Hi, >> > > >> > > I've got a problem. >> > > >> > >> > Hello, >> > >> > > >> > > For a customer, I run a VM in Xen that should perform a certain task >> > > in PHP >> > > (written using the ZendFrameWork). >> > > >> > > That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB >> > > 5.5.0, >> > > php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory >> > > The "reference" server that the customer uses is somewhere else and >> > > manages >> > > to perform the same task in 3s. >> > > >> > > I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it >> > > takes >> > > about 9s. >> > >> > In the sentence above, are you running it in a Xen VM or on bare metal? >> > >> >> This is both Xen. >> I think the customer is also running it on some sort of >> virtualization. > > Hm, so a given workload on Xen takes ~9s, and it also takes ~9s when > run on > bare metal FreeBSD, is that right? It only takes 9s with Linux as a Xen-guest. With all things equal (PHP-version, MariaDB-version), FreeBSD is essentially only half as fast as Linux as a Xen-guest. Sorry for the confusion. >> > hw.xen.disable_pv_disks=1 >> > hw.xen.disable_pv_nics=1 >> >> OK, can I still boot the VM with this or will NICs and disks show up >> as >> different devices then? > > NIC will show up as "re", disks as "ada" (which is what you already > have). I tried this with the FreeBSD 11 VM mentioned in my other mail and it only gets a bit slower. Between 5% and 10%, I'd say. From owner-freebsd-xen@freebsd.org Tue Aug 16 11:08:09 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 446D8BBB622 for ; Tue, 16 Aug 2016 11:08:09 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D1D371D9C for ; Tue, 16 Aug 2016 11:08:08 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="372564144" Date: Tue, 16 Aug 2016 13:08:00 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160816110759.6xlvxikw3tziahfd@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 11:08:09 -0000 On Tue, Aug 16, 2016 at 12:13:55PM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-16 12:06, schrieb Roger Pau Monné: > > On Tue, Aug 16, 2016 at 11:29:43AM +0200, rainer@ultra-secure.de wrote: > > > Am 2016-08-16 10:54, schrieb Roger Pau Monné: > > > > On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer@ultra-secure.de wrote: > > > > > Hi, > > > > > > > > > > I've got a problem. > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > For a customer, I run a VM in Xen that should perform a certain task > > > > > in PHP > > > > > (written using the ZendFrameWork). > > > > > > > > > > That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB > > > > > 5.5.0, > > > > > php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory > > > > > The "reference" server that the customer uses is somewhere else and > > > > > manages > > > > > to perform the same task in 3s. > > > > > > > > > > I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it > > > > > takes > > > > > about 9s. > > > > > > > > In the sentence above, are you running it in a Xen VM or on bare metal? > > > > > > > > > > This is both Xen. > > > I think the customer is also running it on some sort of > > > virtualization. > > > > Hm, so a given workload on Xen takes ~9s, and it also takes ~9s when run > > on > > bare metal FreeBSD, is that right? > > > It only takes 9s with Linux as a Xen-guest. > With all things equal (PHP-version, MariaDB-version), FreeBSD is essentially > only half as fast as Linux as a Xen-guest. > Sorry for the confusion. > And FreeBSD on bare metal is equally fast as Linux then? (ie: the slowdown is only noticeable when running FreeBSD on Xen) > > > > > hw.xen.disable_pv_disks=1 > > > > hw.xen.disable_pv_nics=1 > > > > > > OK, can I still boot the VM with this or will NICs and disks show up > > > as > > > different devices then? > > > > NIC will show up as "re", disks as "ada" (which is what you already > > have). > > I tried this with the FreeBSD 11 VM mentioned in my other mail and it only > gets a bit slower. > Between 5% and 10%, I'd say. If you can provide me with some way to synthesize this workload that doesn't involve setting up the full stack plus your app I can try to reproduce it locally and analyze it in order to find the bottlenecks. Roger. From owner-freebsd-xen@freebsd.org Tue Aug 16 13:14:15 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7E15BBB051 for ; Tue, 16 Aug 2016 13:14:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 4F5491BB4 for ; Tue, 16 Aug 2016 13:14:14 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 15:14:12 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 1938CEB6-342A-4A1A-9AEB-D744CFAD7202.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 15:14:06 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 15:14:05 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816110759.6xlvxikw3tziahfd@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 17, bad: 0, connections: 17, history: 17, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:14:16 -0000 Am 2016-08-16 13:08, schrieb Roger Pau Monné: > And FreeBSD on bare metal is equally fast as Linux then? (ie: the > slowdown > is only noticeable when running FreeBSD on Xen) Yes. I should clarify a bit more. The task involves 12000 (simple) MySQL-Queries, where the script basically selects the numbers 1...4000 from a table (it's stupid, I know) and then proceeds to run a for i in ... loop 4000 times which consists of three other sql-queries, where the WHERE-clause is constrained by the value from above. We've now found that indeed MariaDB is much faster on Xen-Linux than Xen-FreeBSD. The tables all use innodb and the DB is sitting on UFS (in the FreeBSD-on-Xen case, the FreeBSD-on-bare-metal has ZFS). Linux is using ext4. Could this really be an UFS vs. ext4 thing? I assume that optimizations in MariaDB 10.1 lead to less disk-activity. I haven't had a chance to try MariaDB on Linux on bare-metal (we don't really have one where I could test this ATM). > If you can provide me with some way to synthesize this workload that > doesn't > involve setting up the full stack plus your app I can try to reproduce > it > locally and analyze it in order to find the bottlenecks. It's a bit of a pain to setup (mariadb, php-fpm, nginx). Also, you need some files from another framework (apparently).... From owner-freebsd-xen@freebsd.org Tue Aug 16 13:29:43 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CE41BBB53A for ; Tue, 16 Aug 2016 13:29:43 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D5F2016BE for ; Tue, 16 Aug 2016 13:29:41 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="380399941" Date: Tue, 16 Aug 2016 15:29:38 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160816132938.d2i4u2y3scpzi2et@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:29:43 -0000 On Tue, Aug 16, 2016 at 03:14:05PM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-16 13:08, schrieb Roger Pau Monné: > > > And FreeBSD on bare metal is equally fast as Linux then? (ie: the > > slowdown > > is only noticeable when running FreeBSD on Xen) > > > Yes. > > I should clarify a bit more. > The task involves 12000 (simple) MySQL-Queries, where the script basically > selects the numbers 1...4000 from a table (it's stupid, I know) and then > proceeds to run a for i in ... loop 4000 times which consists of three other > sql-queries, where the WHERE-clause is constrained by the value from above. > > We've now found that indeed MariaDB is much faster on Xen-Linux than > Xen-FreeBSD. > > The tables all use innodb and the DB is sitting on UFS (in the > FreeBSD-on-Xen case, the FreeBSD-on-bare-metal has ZFS). > Linux is using ext4. Hm, the fact that FreeBSD on bare metal is using ZFS could also make a difference. The ZFS memory caching is quite aggressive, and I expect it should speed up database queries (unless the database itself is fully loaded into RAM, in which case it doesn't matter much). > Could this really be an UFS vs. ext4 thing? Hm, maybe. There are a lot of moving pieces here that make it quite hard to diagnose the issue properly. Could you try to run something like UnixBench (or any other general benchmarking tool) inside of the Linux VM, the FreeBSD VM and a bare metal FreeBSD install? This way we might be able to spot what's causing this slowdown. Roger. From owner-freebsd-xen@freebsd.org Tue Aug 16 13:39:02 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B593BBBB9C3 for ; Tue, 16 Aug 2016 13:39:02 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 786791EC0 for ; Tue, 16 Aug 2016 13:39:01 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id 7093A9DD2A0; Tue, 16 Aug 2016 15:38:52 +0200 (CEST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? From: Borja Marcos In-Reply-To: <20160816132938.d2i4u2y3scpzi2et@mac> Date: Tue, 16 Aug 2016 15:38:51 +0200 Cc: rainer@ultra-secure.de, freebsd-xen@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> To: =?iso-8859-1?Q?Roger_Pau_Monn=E9?= X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:39:02 -0000 > On 16 Aug 2016, at 15:29, Roger Pau Monn=E9 = wrote: >=20 >> Could this really be an UFS vs. ext4 thing? >=20 > Hm, maybe. There are a lot of moving pieces here that make it quite = hard to=20 > diagnose the issue properly. >=20 > Could you try to run something like UnixBench (or any other general=20 > benchmarking tool) inside of the Linux VM, the FreeBSD VM and a bare = metal=20 > FreeBSD install? This way we might be able to spot what's causing this=20= > slowdown. Maybe this is too obvious, my apologies in that case. But, how have the = filesystems been created and mounted? Asynchronous? Synchronous? Journalling? Softupdates = in the case of=20 FreeBSD UFS? It can make quite a difference. Borja. From owner-freebsd-xen@freebsd.org Tue Aug 16 13:39:15 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F263FBBB9E0 for ; Tue, 16 Aug 2016 13:39:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 6427D1EE3 for ; Tue, 16 Aug 2016 13:39:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 15:39:13 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id B2A1C258-3F4B-4DD0-8A36-63565EEB28DC.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 15:39:08 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 15:39:07 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816132938.d2i4u2y3scpzi2et@mac> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 18, bad: 0, connections: 18, history: 18, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:39:16 -0000 Am 2016-08-16 15:29, schrieb Roger Pau Monné: > Could you try to run something like UnixBench (or any other general > benchmarking tool) inside of the Linux VM, the FreeBSD VM and a bare > metal > FreeBSD install? This way we might be able to spot what's causing this > slowdown. I'll have to attach new disks first. Because the FreeBSD VM currently has 16GB RAM, bonnie wants to create a 32GB file... The MySQL Benchmark-differences are in line with what I get from my curl-requests to the servers. I'll look into running benchmarks. Previously, I had to find out the running MyISAM on ZFS on Xen is absolutely killing performance. It's not great on physical hardware either, but on Xen it's really noticeable. From owner-freebsd-xen@freebsd.org Tue Aug 16 13:42:02 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0080DBBBB46 for ; Tue, 16 Aug 2016 13:42:02 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 5A9CB1277 for ; Tue, 16 Aug 2016 13:42:00 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 15:41:59 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 9023DBEC-2442-4D94-9C1C-50936D914B9D.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 15:41:57 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 15:41:56 +0200 From: rainer@ultra-secure.de To: Borja Marcos Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> Message-ID: <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 19, bad: 0, connections: 19, history: 19, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:42:02 -0000 Am 2016-08-16 15:38, schrieb Borja Marcos: >> On 16 Aug 2016, at 15:29, Roger Pau Monné >> wrote: >> >>> Could this really be an UFS vs. ext4 thing? >> >> Hm, maybe. There are a lot of moving pieces here that make it quite >> hard to >> diagnose the issue properly. >> >> Could you try to run something like UnixBench (or any other general >> benchmarking tool) inside of the Linux VM, the FreeBSD VM and a bare >> metal >> FreeBSD install? This way we might be able to spot what's causing this >> slowdown. > > Maybe this is too obvious, my apologies in that case. But, how have > the filesystems been > created and mounted? Asynchronous? Synchronous? Journalling? > Softupdates in the case of > FreeBSD UFS? It can make quite a difference. FreeBSD /dev/ada2p1 on /home/db (ufs, local, soft-updates) Linux: /dev/mapper/system-lvm--home /home ext4 defaults 0 2 What does "defaults" mean, BTW? From owner-freebsd-xen@freebsd.org Tue Aug 16 13:48:24 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4425BBBBC8A for ; Tue, 16 Aug 2016 13:48:24 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05A21180F for ; Tue, 16 Aug 2016 13:48:23 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 1727F9DC8A2; Tue, 16 Aug 2016 15:48:15 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? From: Borja Marcos In-Reply-To: <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> Date: Tue, 16 Aug 2016 15:48:14 +0200 Cc: =?utf-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> To: rainer@ultra-secure.de X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:48:24 -0000 > On 16 Aug 2016, at 15:41, rainer@ultra-secure.de wrote: >=20 > Am 2016-08-16 15:38, schrieb Borja Marcos: >>=20 >> Maybe this is too obvious, my apologies in that case. But, how have >> the filesystems been >> created and mounted? Asynchronous? Synchronous? Journalling? >> Softupdates in the case of >> FreeBSD UFS? It can make quite a difference. >=20 >=20 > FreeBSD >=20 > /dev/ada2p1 on /home/db (ufs, local, soft-updates) >=20 >=20 >=20 > Linux: > /dev/mapper/system-lvm--home /home ext4 defaults 0 = 2 >=20 >=20 > What does "defaults" mean, BTW? That=E2=80=99s the mother of the lamb, we use to say in Spain ;) I guess it depends on the particular distribution, not just on being = ext4. Is there a tool similar to dumpfs on Linux?=20 You can also experiment with the FreeBSD options, maybe it will be a = quicker route. Try to mount as asynchronous. In case it makes a big difference, you got it. Borja. From owner-freebsd-xen@freebsd.org Tue Aug 16 14:05:39 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA32ABBC2F9 for ; Tue, 16 Aug 2016 14:05:39 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 28DB6169F for ; Tue, 16 Aug 2016 14:05:38 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 16:05:37 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id EC28C248-A2F1-4E04-AD54-14C4574F49E3.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 16:05:31 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 16:05:30 +0200 From: rainer@ultra-secure.de To: Borja Marcos Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> References: <20160816085455.46a5slqsbgauod5t@mac> <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 20, bad: 0, connections: 20, history: 20, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 14:05:39 -0000 Am 2016-08-16 15:48, schrieb Borja Marcos: >> On 16 Aug 2016, at 15:41, rainer@ultra-secure.de wrote: >> >> Am 2016-08-16 15:38, schrieb Borja Marcos: >>> >>> Maybe this is too obvious, my apologies in that case. But, how have >>> the filesystems been >>> created and mounted? Asynchronous? Synchronous? Journalling? >>> Softupdates in the case of >>> FreeBSD UFS? It can make quite a difference. >> >> >> FreeBSD >> >> /dev/ada2p1 on /home/db (ufs, local, soft-updates) >> >> >> >> Linux: >> /dev/mapper/system-lvm--home /home ext4 defaults 0 >> 2 >> >> >> What does "defaults" mean, BTW? > > That’s the mother of the lamb, we use to say in Spain ;) > > I guess it depends on the particular distribution, not just on being > ext4. Is there a tool similar to > dumpfs on Linux? Apparently, it's in cat /proc/mounts /dev/mapper/system-lvm--tmp /tmp ext4 rw,relatime,data=ordered 0 0 /dev/mapper/system-lvm--var /var ext4 rw,relatime,data=ordered 0 0 /dev/mapper/system-lvm--home /home ext4 rw,relatime,data=ordered 0 0 /dev/mapper/system-lvm--varlog /var/log ext4 rw,relatime,data=ordered 0 0 > You can also experiment with the FreeBSD options, maybe it will be a > quicker route. Try to mount as asynchronous. > In case it makes a big difference, you got it. But I don't really want to mount it asyncronous. Would it help to have journaling? Or is soft-updates already the "optimum"? https://www.freebsd.org/doc/handbook/configtuning-disk.html From owner-freebsd-xen@freebsd.org Tue Aug 16 14:18:56 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FE10BBC4F2 for ; Tue, 16 Aug 2016 14:18:56 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC55D1BFA for ; Tue, 16 Aug 2016 14:18:55 +0000 (UTC) (envelope-from prvs=029c3005f=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="380415718" Date: Tue, 16 Aug 2016 16:18:26 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: Borja Marcos , Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160816141826.56mxsgx6e7rynxqg@mac> References: <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 14:18:56 -0000 On Tue, Aug 16, 2016 at 04:05:30PM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-16 15:48, schrieb Borja Marcos: > > > On 16 Aug 2016, at 15:41, rainer@ultra-secure.de wrote: > > > > > > Am 2016-08-16 15:38, schrieb Borja Marcos: > > > > > > > > Maybe this is too obvious, my apologies in that case. But, how have > > > > the filesystems been > > > > created and mounted? Asynchronous? Synchronous? Journalling? > > > > Softupdates in the case of > > > > FreeBSD UFS? It can make quite a difference. > > > > > > > > > FreeBSD > > > > > > /dev/ada2p1 on /home/db (ufs, local, soft-updates) > > > > > > > > > > > > Linux: > > > /dev/mapper/system-lvm--home /home ext4 defaults > > > 0 2 > > > > > > > > > What does "defaults" mean, BTW? > > > > That’s the mother of the lamb, we use to say in Spain ;) > > > > I guess it depends on the particular distribution, not just on being > > ext4. Is there a tool similar to > > dumpfs on Linux? > > > Apparently, it's in > cat /proc/mounts > > /dev/mapper/system-lvm--tmp /tmp ext4 rw,relatime,data=ordered 0 0 > /dev/mapper/system-lvm--var /var ext4 rw,relatime,data=ordered 0 0 > /dev/mapper/system-lvm--home /home ext4 rw,relatime,data=ordered 0 0 > /dev/mapper/system-lvm--varlog /var/log ext4 rw,relatime,data=ordered 0 0 > > > > > > You can also experiment with the FreeBSD options, maybe it will be a > > quicker route. Try to mount as asynchronous. > > In case it makes a big difference, you got it. > > But I don't really want to mount it asyncronous. > Would it help to have journaling? > > Or is soft-updates already the "optimum"? I'm not saying it's not possible, but are you sure this slowdown is caused by the disk? It's certainly a possibility, but I would like to make sure it's caused by that before jumping into conclusions. Can you load the full database in RAM and perform the same test. TBH, I don't use MariaDB, so I'm not sure what's the best way to achieve this, but a quick search on google shows there are multiple ways. In any case, make sure with iostat that the database is not read from the disk. Roger. From owner-freebsd-xen@freebsd.org Tue Aug 16 14:24:14 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74A7EBBC730 for ; Tue, 16 Aug 2016 14:24:14 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id B79D011C3 for ; Tue, 16 Aug 2016 14:24:13 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 16 Aug 2016 16:24:12 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id FC9D4824-FBA3-4E38-8A86-973B3B035C6E.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 16 Aug 2016 16:24:10 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 16 Aug 2016 16:24:09 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: Borja Marcos , freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160816141826.56mxsgx6e7rynxqg@mac> References: <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 21, bad: 0, connections: 21, history: 21, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 14:24:14 -0000 Am 2016-08-16 16:18, schrieb Roger Pau Monné: > I'm not saying it's not possible, but are you sure this slowdown is > caused > by the disk? It's certainly a possibility, but I would like to make > sure > it's caused by that before jumping into conclusions. > > Can you load the full database in RAM and perform the same test. TBH, I > don't use MariaDB, so I'm not sure what's the best way to achieve this, > but > a quick search on google shows there are multiple ways. In any case, > make > sure with iostat that the database is not read from the disk. I'll try to do some disk-benchmarks, when I can attach some bigger disks. From owner-freebsd-xen@freebsd.org Wed Aug 17 08:19:17 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45C4ABBC819; Wed, 17 Aug 2016 08:19:17 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 6A9D91410; Wed, 17 Aug 2016 08:19:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Wed, 17 Aug 2016 10:19:08 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 5F1FD6AE-141D-4980-A23F-C84E38C90BD1.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Wed, 17 Aug 2016 10:19:05 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 17 Aug 2016 10:19:05 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: References: <2a0a5ae2821551935de329b8665834be@ultra-secure.de> <20160816100605.la63x2ju5bmtdqhl@mac> <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> Message-ID: <599395934f751784b1f842ed3c8f879c@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 22, bad: 0, connections: 22, history: 22, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 08:19:17 -0000 Am 2016-08-16 16:24, schrieb rainer@ultra-secure.de: > Am 2016-08-16 16:18, schrieb Roger Pau Monné: > >> I'm not saying it's not possible, but are you sure this slowdown is >> caused >> by the disk? It's certainly a possibility, but I would like to make >> sure >> it's caused by that before jumping into conclusions. >> >> Can you load the full database in RAM and perform the same test. TBH, >> I >> don't use MariaDB, so I'm not sure what's the best way to achieve >> this, but >> a quick search on google shows there are multiple ways. In any case, >> make >> sure with iostat that the database is not read from the disk. > > I'll try to do some disk-benchmarks, when I can attach some bigger > disks. on Ubuntu 14 with HVM: dc3dd wipe=/dev/xvdc dc3dd 7.1.614 started at 2016-08-17 09:38:17 +0200 compiled options: command line: dc3dd wipe=/dev/xvdc device size: 104857600 sectors (probed) sector size: 512 bytes (probed) 53687091200 bytes (50 G) copied (100%), 464.642 s, 110 M/s input results for pattern `00': 104857600 sectors in output results for device `/dev/xvdc': 104857600 sectors out dc3dd completed at 2016-08-17 09:46:01 +0200 On FreeBSD 10.3-RELEASE-p6 with HVM: Aug 17 09:57:52 bla-prod kernel: xbd4: 51200MB at device/vbd/51776 on xenbusb_front0 Aug 17 09:57:52 bla-prod kernel: xbd4: features: write_barrier Aug 17 09:57:52 bla-prod kernel: xbd4: synchronize cache commands enabled. dc3dd wipe=/dev/xbd4 and it's showing 8.something MB/s Is this normal? What may I be doing wrong? Rainer From owner-freebsd-xen@freebsd.org Wed Aug 17 09:12:42 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED10FBBC79E; Wed, 17 Aug 2016 09:12:42 +0000 (UTC) (envelope-from prvs=0309661e3=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88B6D1905; Wed, 17 Aug 2016 09:12:42 +0000 (UTC) (envelope-from prvs=0309661e3=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="372793205" Date: Wed, 17 Aug 2016 11:12:38 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: , Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160817091229.hm5a66ftwnfoj7vx@mac> References: <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <599395934f751784b1f842ed3c8f879c@ultra-secure.de> User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 09:12:43 -0000 On Wed, Aug 17, 2016 at 10:19:05AM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-16 16:24, schrieb rainer@ultra-secure.de: > > Am 2016-08-16 16:18, schrieb Roger Pau Monné: > > > > > I'm not saying it's not possible, but are you sure this slowdown is > > > caused > > > by the disk? It's certainly a possibility, but I would like to make > > > sure > > > it's caused by that before jumping into conclusions. > > > > > > Can you load the full database in RAM and perform the same test. > > > TBH, I > > > don't use MariaDB, so I'm not sure what's the best way to achieve > > > this, but > > > a quick search on google shows there are multiple ways. In any case, > > > make > > > sure with iostat that the database is not read from the disk. > > > > I'll try to do some disk-benchmarks, when I can attach some bigger > > disks. > > on Ubuntu 14 with HVM: > > dc3dd wipe=/dev/xvdc > > dc3dd 7.1.614 started at 2016-08-17 09:38:17 +0200 > compiled options: > command line: dc3dd wipe=/dev/xvdc > device size: 104857600 sectors (probed) > sector size: 512 bytes (probed) > 53687091200 bytes (50 G) copied (100%), 464.642 s, 110 M/s > > input results for pattern `00': > 104857600 sectors in > > output results for device `/dev/xvdc': > 104857600 sectors out > > dc3dd completed at 2016-08-17 09:46:01 +0200 > > > On FreeBSD 10.3-RELEASE-p6 with HVM: > > Aug 17 09:57:52 bla-prod kernel: xbd4: 51200MB at > device/vbd/51776 on xenbusb_front0 > Aug 17 09:57:52 bla-prod kernel: xbd4: features: write_barrier > Aug 17 09:57:52 bla-prod kernel: xbd4: synchronize cache commands enabled. > > dc3dd wipe=/dev/xbd4 > > and it's showing 8.something MB/s > > Is this normal? No, I don't think so, this is what I get using a slow USB 2.0 disk as the backend: (on Dom0 I get something between 70-80M/s, so there isn't much difference). # dc3dd wipe=/dev/ada1 dc3dd 7.2.641 started at 2016-08-17 09:03:26 +0000 compiled options: command line: dc3dd wipe=/dev/ada1 device size: 2097152 sectors (probed), 1,073,741,824 bytes sector size: 512 bytes (probed) 1073741824 bytes ( 1 G ) copied ( 100% ), 16 s, 65 M/s input results for pattern `00': 2097152 sectors in output results for device `/dev/ada1': 2097152 sectors out dc3dd completed at 2016-08-17 09:03:42 +0000 > What may I be doing wrong? TBH it's hard to tell, I don't know of any option that could cause this disk performance degradation. Do you also have ada* devices apart from the xbd* ones? I don't think it's going to make any difference, but could you try with the ada* block devices instead? Roger. From owner-freebsd-xen@freebsd.org Wed Aug 17 09:29:22 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAD55BBD0CE; Wed, 17 Aug 2016 09:29:22 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 1FF141A98; Wed, 17 Aug 2016 09:29:21 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Wed, 17 Aug 2016 11:29:20 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id D3EC94CB-85EC-47C6-8DF7-8BB1997516AF.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Wed, 17 Aug 2016 11:29:17 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 17 Aug 2016 11:29:17 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160817091229.hm5a66ftwnfoj7vx@mac> References: <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> <20160817091229.hm5a66ftwnfoj7vx@mac> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 23, bad: 0, connections: 23, history: 23, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 09:29:23 -0000 Am 2016-08-17 11:12, schrieb Roger Pau Monné: > No, I don't think so, this is what I get using a slow USB 2.0 disk as > the > backend: (on Dom0 I get something between 70-80M/s, so there isn't much > difference). > > # dc3dd wipe=/dev/ada1 > > dc3dd 7.2.641 started at 2016-08-17 09:03:26 +0000 > compiled options: > command line: dc3dd wipe=/dev/ada1 > device size: 2097152 sectors (probed), 1,073,741,824 bytes > sector size: 512 bytes (probed) > 1073741824 bytes ( 1 G ) copied ( 100% ), 16 s, 65 M/s > > input results for pattern `00': > 2097152 sectors in > > output results for device `/dev/ada1': > 2097152 sectors out > > dc3dd completed at 2016-08-17 09:03:42 +0000 > >> What may I be doing wrong? > > TBH it's hard to tell, I don't know of any option that could cause this > disk > performance degradation. Do you also have ada* devices apart from the > xbd* > ones? I don't think it's going to make any difference, but could you > try > with the ada* block devices instead? Strange thing is, I have ada devices for the the other disks, but this one didn't show up as ada-device. On my FreeBSD11 Test VM, the disk didn't show up until I rebooted, even though I (believe to) have the xen-guest stuff installed: (freebsd11 ) 0 # ps ax |grep xe-d 694 v0- I 0:00.00 /bin/sh /usr/local/sbin/xe-daemon -p /var/run/xe-daemon.pid 2202 0 R+ 0:00.00 grep xe-d (freebsd11 ) 0 # pkg info|grep xe- xe-guest-utilities-6.2.0_2 FreeBSD VM tools for Citrix XenServer and XCP (freebsd11 ) 0 # pkg info|grep xen xen-guest-tools-4.6.1 Xen tools within FreeBSD domU I have an ada device there and I got about 10MB/s on wipe. At least in the beginning. (freebsd11 ) 0 # sysctl -a |grep xen kern.vm_guest: xen device xenpci vfs.pfs.vncache.maxentries: 0 dev.xenbusb_back.0.%parent: xenstore0 dev.xenbusb_back.0.%pnpinfo: dev.xenbusb_back.0.%location: dev.xenbusb_back.0.%driver: xenbusb_back dev.xenbusb_back.0.%desc: Xen Backend Devices dev.xenbusb_back.%parent: dev.xn.0.xenstore_peer_path: /local/domain/0/backend/vif/245/0 dev.xn.0.xenbus_peer_domid: 0 dev.xn.0.xenbus_connection_state: Connected dev.xn.0.xenbus_dev_type: vif dev.xn.0.xenstore_path: device/vif/0 dev.xn.0.%parent: xenbusb_front0 dev.xbd.1.xenstore_peer_path: /local/domain/0/backend/vbd3/245/768 dev.xbd.1.xenbus_peer_domid: 0 dev.xbd.1.xenbus_connection_state: Connected dev.xbd.1.xenbus_dev_type: vbd dev.xbd.1.xenstore_path: device/vbd/768 dev.xbd.1.%parent: xenbusb_front0 dev.xbd.0.xenstore_peer_path: /local/domain/0/backend/vbd3/245/832 dev.xbd.0.xenbus_peer_domid: 0 dev.xbd.0.xenbus_connection_state: Connected dev.xbd.0.xenbus_dev_type: vbd dev.xbd.0.xenstore_path: device/vbd/832 dev.xbd.0.%parent: xenbusb_front0 dev.xenbusb_front.0.%parent: xenstore0 dev.xenbusb_front.0.%pnpinfo: dev.xenbusb_front.0.%location: dev.xenbusb_front.0.%driver: xenbusb_front dev.xenbusb_front.0.%desc: Xen Frontend Devices dev.xenbusb_front.%parent: dev.xs_dev.0.%parent: xenstore0 dev.xctrl.0.%parent: xenstore0 dev.xenballoon.0.%parent: xenstore0 dev.xenballoon.0.%pnpinfo: dev.xenballoon.0.%location: dev.xenballoon.0.%driver: xenballoon dev.xenballoon.0.%desc: Xen Balloon Device dev.xenballoon.%parent: dev.debug.0.%parent: xenpv0 dev.privcmd.0.%parent: xenpv0 dev.evtchn.0.%parent: xenpv0 dev.xenstore.0.%parent: xenpv0 dev.xenstore.0.%pnpinfo: dev.xenstore.0.%location: dev.xenstore.0.%driver: xenstore dev.xenstore.0.%desc: XenStore dev.xenstore.%parent: dev.xen_et.0.%parent: xenpv0 dev.xen_et.0.%pnpinfo: dev.xen_et.0.%location: dev.xen_et.0.%driver: xen_et dev.xen_et.0.%desc: Xen PV Clock dev.xen_et.%parent: dev.granttable.0.%parent: xenpv0 dev.xenpv.0.%parent: nexus0 dev.xenpv.0.%pnpinfo: dev.xenpv.0.%location: dev.xenpv.0.%driver: xenpv dev.xenpv.0.%desc: Xen PV bus dev.xenpv.%parent: dev.xenpci.0.%parent: pci0 dev.xenpci.0.%pnpinfo: vendor=0x5853 device=0x0001 subvendor=0x5853 subdevice=0x0001 class=0x010000 dev.xenpci.0.%location: slot=3 function=0 dbsf=pci0:0:3:0 handle=\_SB_.PCI0.S18_ dev.xenpci.0.%driver: xenpci dev.xenpci.0.%desc: Xen Platform Device dev.xenpci.%parent: dev.xen.xsd_kva: 18446735281894703104 dev.xen.xsd_port: 3 dev.xen.balloon.high_mem: 0 dev.xen.balloon.low_mem: 0 dev.xen.balloon.hard_limit: 18446744073709551615 dev.xen.balloon.driver_pages: 0 dev.xen.balloon.target: 2097152 dev.xen.balloon.current: 2096128 Do you know what I could check on the dom0 side to make sure it's configured right? From owner-freebsd-xen@freebsd.org Wed Aug 17 10:15:26 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99FE8BBE000; Wed, 17 Aug 2016 10:15:26 +0000 (UTC) (envelope-from prvs=0309661e3=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 428D81D9C; Wed, 17 Aug 2016 10:15:25 +0000 (UTC) (envelope-from prvs=0309661e3=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="scan'208";a="372799843" Date: Wed, 17 Aug 2016 12:15:21 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: CC: , Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? Message-ID: <20160817101521.e3sekmopltmbuujr@mac> References: <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> <20160817091229.hm5a66ftwnfoj7vx@mac> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 10:15:26 -0000 On Wed, Aug 17, 2016 at 11:29:17AM +0200, rainer@ultra-secure.de wrote: > Am 2016-08-17 11:12, schrieb Roger Pau Monné: > > > No, I don't think so, this is what I get using a slow USB 2.0 disk as > > the > > backend: (on Dom0 I get something between 70-80M/s, so there isn't much > > difference). > > > > # dc3dd wipe=/dev/ada1 > > > > dc3dd 7.2.641 started at 2016-08-17 09:03:26 +0000 > > compiled options: > > command line: dc3dd wipe=/dev/ada1 > > device size: 2097152 sectors (probed), 1,073,741,824 bytes > > sector size: 512 bytes (probed) > > 1073741824 bytes ( 1 G ) copied ( 100% ), 16 s, 65 M/s > > > > input results for pattern `00': > > 2097152 sectors in > > > > output results for device `/dev/ada1': > > 2097152 sectors out > > > > dc3dd completed at 2016-08-17 09:03:42 +0000 > > > > > What may I be doing wrong? > > > > TBH it's hard to tell, I don't know of any option that could cause this > > disk > > performance degradation. Do you also have ada* devices apart from the > > xbd* > > ones? I don't think it's going to make any difference, but could you try > > with the ada* block devices instead? > > > Strange thing is, I have ada devices for the the other disks, but this one > didn't show up as ada-device. The fact that it shows up as ada or xbd depends on what you specify in the guest config file (hd* will show up as ada, while xvd* will show up as xbd*). I don't know how/if XenServer allows you to specify the vdev in the guest configuration. > On my FreeBSD11 Test VM, the disk didn't show up until I rebooted, even > though I (believe to) have the xen-guest stuff installed: > > (freebsd11 ) 0 # ps ax |grep xe-d > 694 v0- I 0:00.00 /bin/sh /usr/local/sbin/xe-daemon -p > /var/run/xe-daemon.pid > 2202 0 R+ 0:00.00 grep xe-d > (freebsd11 ) 0 # pkg info|grep xe- > xe-guest-utilities-6.2.0_2 FreeBSD VM tools for Citrix XenServer and XCP > (freebsd11 ) 0 # pkg info|grep xen > xen-guest-tools-4.6.1 Xen tools within FreeBSD domU > > I have an ada device there and I got about 10MB/s on wipe. At least in the > beginning. Hm, so performance is more or less the same. Having the xen-guest stuff should not make a difference regarding disks, this is IIRC only used when migrating a VM. > (freebsd11 ) 0 # sysctl -a |grep xen > kern.vm_guest: xen > device xenpci > vfs.pfs.vncache.maxentries: 0 > dev.xenbusb_back.0.%parent: xenstore0 > dev.xenbusb_back.0.%pnpinfo: > dev.xenbusb_back.0.%location: > dev.xenbusb_back.0.%driver: xenbusb_back > dev.xenbusb_back.0.%desc: Xen Backend Devices > dev.xenbusb_back.%parent: > dev.xn.0.xenstore_peer_path: /local/domain/0/backend/vif/245/0 > dev.xn.0.xenbus_peer_domid: 0 > dev.xn.0.xenbus_connection_state: Connected > dev.xn.0.xenbus_dev_type: vif > dev.xn.0.xenstore_path: device/vif/0 > dev.xn.0.%parent: xenbusb_front0 > dev.xbd.1.xenstore_peer_path: /local/domain/0/backend/vbd3/245/768 > dev.xbd.1.xenbus_peer_domid: 0 > dev.xbd.1.xenbus_connection_state: Connected > dev.xbd.1.xenbus_dev_type: vbd > dev.xbd.1.xenstore_path: device/vbd/768 > dev.xbd.1.%parent: xenbusb_front0 > dev.xbd.0.xenstore_peer_path: /local/domain/0/backend/vbd3/245/832 > dev.xbd.0.xenbus_peer_domid: 0 > dev.xbd.0.xenbus_connection_state: Connected > dev.xbd.0.xenbus_dev_type: vbd > dev.xbd.0.xenstore_path: device/vbd/832 > dev.xbd.0.%parent: xenbusb_front0 > dev.xenbusb_front.0.%parent: xenstore0 > dev.xenbusb_front.0.%pnpinfo: > dev.xenbusb_front.0.%location: > dev.xenbusb_front.0.%driver: xenbusb_front > dev.xenbusb_front.0.%desc: Xen Frontend Devices > dev.xenbusb_front.%parent: > dev.xs_dev.0.%parent: xenstore0 > dev.xctrl.0.%parent: xenstore0 > dev.xenballoon.0.%parent: xenstore0 > dev.xenballoon.0.%pnpinfo: > dev.xenballoon.0.%location: > dev.xenballoon.0.%driver: xenballoon > dev.xenballoon.0.%desc: Xen Balloon Device > dev.xenballoon.%parent: > dev.debug.0.%parent: xenpv0 > dev.privcmd.0.%parent: xenpv0 > dev.evtchn.0.%parent: xenpv0 > dev.xenstore.0.%parent: xenpv0 > dev.xenstore.0.%pnpinfo: > dev.xenstore.0.%location: > dev.xenstore.0.%driver: xenstore > dev.xenstore.0.%desc: XenStore > dev.xenstore.%parent: > dev.xen_et.0.%parent: xenpv0 > dev.xen_et.0.%pnpinfo: > dev.xen_et.0.%location: > dev.xen_et.0.%driver: xen_et > dev.xen_et.0.%desc: Xen PV Clock > dev.xen_et.%parent: > dev.granttable.0.%parent: xenpv0 > dev.xenpv.0.%parent: nexus0 > dev.xenpv.0.%pnpinfo: > dev.xenpv.0.%location: > dev.xenpv.0.%driver: xenpv > dev.xenpv.0.%desc: Xen PV bus > dev.xenpv.%parent: > dev.xenpci.0.%parent: pci0 > dev.xenpci.0.%pnpinfo: vendor=0x5853 device=0x0001 subvendor=0x5853 > subdevice=0x0001 class=0x010000 > dev.xenpci.0.%location: slot=3 function=0 dbsf=pci0:0:3:0 > handle=\_SB_.PCI0.S18_ > dev.xenpci.0.%driver: xenpci > dev.xenpci.0.%desc: Xen Platform Device > dev.xenpci.%parent: > dev.xen.xsd_kva: 18446735281894703104 > dev.xen.xsd_port: 3 > dev.xen.balloon.high_mem: 0 > dev.xen.balloon.low_mem: 0 > dev.xen.balloon.hard_limit: 18446744073709551615 > dev.xen.balloon.driver_pages: 0 > dev.xen.balloon.target: 2097152 > dev.xen.balloon.current: 2096128 > This looks fine AFAICT. > > Do you know what I could check on the dom0 side to make sure it's configured > right? Sadly XenServer uses a completely different disk backend from the Xen Open Source Project (which is what I work with). Could you try to ask on the XenServer mailing lists or forums[0]? I think there's a better chance you will find someone familiar with it there. Feel free to Cc me if you need input regarding the FreeBSD blkfront internals. Roger. [0] http://xenserver.org/ From owner-freebsd-xen@freebsd.org Wed Aug 17 10:17:33 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A82F0BBC158; Wed, 17 Aug 2016 10:17:33 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id DB4E011CF; Wed, 17 Aug 2016 10:17:32 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Wed, 17 Aug 2016 12:17:31 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id A1B98409-9C20-4D45-A251-7626E5E72156.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Wed, 17 Aug 2016 12:17:29 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 17 Aug 2016 12:17:28 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160817091229.hm5a66ftwnfoj7vx@mac> References: <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> <20160817091229.hm5a66ftwnfoj7vx@mac> Message-ID: <23f4fbc340f9cf51ee65bbd148706649@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 24, bad: 0, connections: 24, history: 24, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 10:17:33 -0000 I played a bit with the "OS-Type". If I switch to "Other PV", I can get a bit more throughput (10MB/s). Still too slow :-( From owner-freebsd-xen@freebsd.org Wed Aug 17 10:32:02 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05789BBC7DD; Wed, 17 Aug 2016 10:32:02 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B66C51DAF; Wed, 17 Aug 2016 10:32:00 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id AEBFA9DC9EE; Wed, 17 Aug 2016 12:31:57 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? From: Borja Marcos In-Reply-To: <23f4fbc340f9cf51ee65bbd148706649@ultra-secure.de> Date: Wed, 17 Aug 2016 12:31:57 +0200 Cc: =?utf-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <0CF8B21F-2728-426B-B81E-C9043148CBD9@sarenet.es> References: <20160816110759.6xlvxikw3tziahfd@mac> <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> <20160817091229.hm5a66ftwnfoj7vx@mac> <23f4fbc340f9cf51ee65bbd148706649@ultra-secure.de> To: rainer@ultra-secure.de X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 10:32:02 -0000 > On 17 Aug 2016, at 12:17, rainer@ultra-secure.de wrote: >=20 > I played a bit with the "OS-Type". > If I switch to "Other PV", I can get a bit more throughput (10MB/s). >=20 > Still too slow :-( Can it be a problem with the sync cache commands? Just wondering what = could be so different with Linux vs FreeBSD. At least in the past the Linux crowd = has chosen to=20 be lousy with committing data do disk and relying on fsck. FreeBSD does the opposite. Borja. From owner-freebsd-xen@freebsd.org Wed Aug 17 10:52:18 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 500E3BBCE51; Wed, 17 Aug 2016 10:52:18 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 7E3DF18AC; Wed, 17 Aug 2016 10:52:16 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Wed, 17 Aug 2016 12:52:15 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id EC266A4B-38D3-483C-96C0-0CC4A7414D85.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Wed, 17 Aug 2016 12:52:12 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 17 Aug 2016 12:52:12 +0200 From: rainer@ultra-secure.de To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Cc: freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Subject: Re: Is it me or is FreeBSD slower on Xen than Linux? In-Reply-To: <20160817101521.e3sekmopltmbuujr@mac> References: <20160816132938.d2i4u2y3scpzi2et@mac> <00D22384-BAA7-42E4-A486-4BE07562D011@sarenet.es> <8521aebaa093bcefe5956a71fd879140@ultra-secure.de> <872C5626-F58D-4F84-92AC-88B7352D1DDF@sarenet.es> <20160816141826.56mxsgx6e7rynxqg@mac> <599395934f751784b1f842ed3c8f879c@ultra-secure.de> <20160817091229.hm5a66ftwnfoj7vx@mac> <20160817101521.e3sekmopltmbuujr@mac> Message-ID: <948682720e31ad9f8c0cbd2456ed9d3d@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 25, bad: 0, connections: 25, history: 25, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 10:52:18 -0000 Am 2016-08-17 12:15, schrieb Roger Pau Monné: > I think there's a better chance you will find someone familiar with it > there. Feel free to Cc me if you need input regarding the FreeBSD > blkfront > internals. > > Roger. > > [0] http://xenserver.org/ OK, registered there and I can't login. Resetting my password they say I'm either in an embargoed country or on a sanctioned party list. Yeah, I'm in Switzerland. "Please note: If you have just created a new account and received this message, please try to log in again in 6 hours before contacting Customer service." Well, if it helps to curb spammers, I'm all for it. Thanks for your support. Rainer From owner-freebsd-xen@freebsd.org Wed Aug 17 12:07:49 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50ADABBD48A for ; Wed, 17 Aug 2016 12:07:49 +0000 (UTC) (envelope-from akshay1994.leo@gmail.com) Received: from mail-pa0-x243.google.com (mail-pa0-x243.google.com [IPv6:2607:f8b0:400e:c03::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21A7A1CBF; Wed, 17 Aug 2016 12:07:49 +0000 (UTC) (envelope-from akshay1994.leo@gmail.com) Received: by mail-pa0-x243.google.com with SMTP id ez1so7041149pab.3; Wed, 17 Aug 2016 05:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=LTeGO0RnhUdEkCMQB5esOrXxFI9uPyFlgpg/9T33ItU=; b=nAjUBVzgD2TL4G0t6UCarBD9Ksq2oz2K5PQ8mMevLrwyPugWqfa2KJ5cfwVDuWvH00 JJU2NBwTc0IVAcjnQbHLfImMUEYaKo9MiN2KA1ZmkObtMtxb05PDw7lOwxlT0MtUkAuF DbJ2CvjIyyGUj+AYzUowSVMS7kFnivfeYQGKb/sSuOoV3+gXn5ujAslMny4lt7N1a7wW MmkaBlYvp5rWIgcayBMmVlHcomxHsykLFL9C2AdspzQOwX5qFfgnBSyX+SaYqKLeaQck RZ0gYKFlmQYBVVoZCGOYY7FH+ewKLPKOSml+fn6TdbLVguOxwZ1aAsA3Dvx0Vo6+pVAx EwdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=LTeGO0RnhUdEkCMQB5esOrXxFI9uPyFlgpg/9T33ItU=; b=CYCGHcL7gP4zyBVwwwX2mkALOO5RktwI68DaR99i8cI5bfB/sP9ZqzuC8thZE7waHq g+kRZLwe9/p0DW+z5jqlfLrlulMDO8YTSQ9Wq7vf80Wkkv5vZ+AsArTi3jT5AeZUD/LS Afk9+fw++4bO6PRWUW8rKgDzQuYEJgedECebUtRSRCFXlJBsqcWB/zDtZXCIFF56KX8g uz4Qs5l6c+JvoauEUwXe0Skla15CsZuDDCVaW90a5ujuNb6tCuwL7WwflqZouu5aJ3jp 1dyP6EK5Rnl52df+uXFe3iZBKElcl70aWTy7zBrVN0TUXFssrXhkSJrJDOoNTRfmuOxA k3rg== X-Gm-Message-State: AEkoout0wNbflyuqVUFb+HKl6GmluP0jUe2PC7tyg6MpuKWWJQ+PBg24uIX1MbsbVcYzCQ== X-Received: by 10.66.150.202 with SMTP id uk10mr4053443pab.10.1471435668304; Wed, 17 Aug 2016 05:07:48 -0700 (PDT) Received: from localhost.localdomain ([122.180.156.10]) by smtp.gmail.com with ESMTPSA id l82sm46842760pfk.8.2016.08.17.05.07.43 (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 17 Aug 2016 05:07:47 -0700 (PDT) From: Akshay Jaggi To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Andrew Cooper , George Dunlap , Ian Jackson , Konrad Rzeszutek Wilk , Stefano Stabellini , Tim Deegan , FreeBSD-Xen , Akshay Jaggi Subject: [PATCH v2] gnttab: Add gntdev device mappings for FreeBSD Date: Wed, 17 Aug 2016 17:36:59 +0530 Message-Id: <1471435619-8747-1-git-send-email-akshay1994.leo@gmail.com> X-Mailer: git-send-email 2.8.2 X-Mailman-Approved-At: Wed, 17 Aug 2016 13:46:31 +0000 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 12:07:49 -0000 Add grant table userspace device mappings for FreeBSD (enables support for qdisk backend on FreeBSD Dom0). Signed-off-by: Akshay Jaggi --- Changed since v1: * fix coding style * remove O_CLOEXEC * remove SET_MAX_GRANTS ioctl * update freebsd/gntdev.h to latest version * replace alloca with malloc --- tools/include/xen-sys/FreeBSD/gntdev.h | 191 ++++++++++++++++++++ tools/libs/gnttab/Makefile | 2 +- tools/libs/gnttab/freebsd.c | 318 +++++++++++++++++++++++++++++++++ 3 files changed, 510 insertions(+), 1 deletion(-) create mode 100644 tools/include/xen-sys/FreeBSD/gntdev.h create mode 100644 tools/libs/gnttab/freebsd.c diff --git a/tools/include/xen-sys/FreeBSD/gntdev.h b/tools/include/xen-sys/FreeBSD/gntdev.h new file mode 100644 index 0000000..5f31e21 --- /dev/null +++ b/tools/include/xen-sys/FreeBSD/gntdev.h @@ -0,0 +1,191 @@ +/*- + * Copyright (c) 2016 Akshay Jaggi + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * gntdev.h + * + * Interface to /dev/xen/gntdev. + * + * This device provides the user with two kinds of functionalities: + * 1. Grant Allocation + * Allocate a page of our own memory, and share it with a foreign domain. + * 2. Grant Mapping + * Map a grant allocated by a foreign domain, into our own memory. + * + * + * Grant Allocation + * + * Steps to allocate a grant: + * 1. Do an `IOCTL_GNTDEV_ALLOC_GREF ioctl`, with + * - `domid`, as the domain-id of the foreign domain + * - `flags`, ORed with GNTDEV_ALLOC_FLAG_WRITABLE if you want the foreign + * domain to have write access to the shared memory + * - `count`, with the number of pages to share with the foreign domain + * + * Ensure that the structure you allocate has enough memory to store + * all the allocated grant-refs, i.e., you need to allocate + * (sizeof(struct ioctl_gntdev_alloc_gref) + (count - 1)*sizeof(uint32_t)) + * bytes of memory. + * + * 2. Mmap the address given in `index` after a successful ioctl. + * This will give you access to the granted pages. + * + * Note: + * 1. The grant is not removed until all three of the following conditions + * are met + * - The region is not mmaped. That is, munmap() has been called if + * the region was mmapped previously. + * - IOCTL_GNTDEV_DEALLOC_GREF ioctl has been performed. After you + * perform this ioctl, you can no longer mmap or set notify on + * the grant. + * - The foreign domain has stopped using the grant. + * 2. Granted pages can only belong to one mmap region. + * 3. Every page of granted memory is a unit in itself. What this means + * is that you can set a unmap notification for each of the granted + * pages, individually; you can mmap and dealloc-ioctl a contiguous + * range of allocated grants (even if alloc-ioctls were performed + * individually), etc. + * + * + * Grant Mapping + * + * Steps to map a grant: + * 1. Do a `IOCTL_GNTDEV_MAP_GRANT_REF` ioctl, with + * - `count`, as the number of foreign grants to map + * - `refs[i].domid`, as the domain id of the foreign domain + * - `refs[i].ref`, as the grant-ref for the grant to be mapped + * + * 2. Mmap the address given in `index` after a successful ioctl. + * This will give you access to the mapped pages. + * + * Note: + * 1. The map hypercall is not made till the region is mmapped. + * 2. The unit is defined by the map ioctl. This means that only one + * unmap notification can be set on a group of pages that were + * mapped together in one ioctl, and also no single mmaping of contiguous + * grant-maps is possible. + * 3. You can mmap the same grant-map region multiple times. + * 4. The grant is not unmapped until both of the following conditions are met + * - The region is not mmaped. That is, munmap() has been called for + * as many times as the grant was mmapped. + * - IOCTL_GNTDEV_UNMAP_GRANT_REF ioctl has been called. + * 5. IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR ioctl gives index and count of + * a grant-map from the virtual address of the location where the grant + * is mmapped. + * + * + * IOCTL_GNTDEV_SET_UNMAP_NOTIFY + * This ioctl allows us to set notifications to be made when the grant is + * either unmapped (in case of a mapped grant), or when it is ready to be + * deallocated by us, ie, the grant is no more mmapped, and the dealloc + * ioctl has been called (in case of an allocated grant). OR `action` with + * the required notification masks, and fill in the appropriate fields. + * - UNMAP_NOTIFY_CLEAR_BYTE clears the byte at `index`, where index is + * the address of the byte in file address space. + * - UNMAP_NOTIFY_SEND_EVENT sends an event channel notification on + * `event_channel_port` + * In case of multiple notify ioctls, only the last one survives. + * + */ + +#ifndef __XEN_GNTDEV_H__ +#define __XEN_GNTDEV_H__ + +#include + +#define IOCTL_GNTDEV_SET_UNMAP_NOTIFY \ + _IOW('E', 0, struct ioctl_gntdev_unmap_notify) +struct ioctl_gntdev_unmap_notify { + /* IN parameters */ + uint64_t index; + uint32_t action; + uint32_t event_channel_port; +}; + +#define UNMAP_NOTIFY_CLEAR_BYTE 0x1 +#define UNMAP_NOTIFY_SEND_EVENT 0x2 + +/*-------------------- Grant Allocation IOCTLs ------------------------------*/ + +#define IOCTL_GNTDEV_ALLOC_GREF \ + _IOWR('E', 1, struct ioctl_gntdev_alloc_gref) +struct ioctl_gntdev_alloc_gref { + /* IN parameters */ + uint16_t domid; + uint16_t flags; + uint32_t count; + /* OUT parameters */ + uint64_t index; + /* Variable OUT parameter */ + uint32_t gref_ids[1]; +}; + +#define GNTDEV_ALLOC_FLAG_WRITABLE 1 + +#define IOCTL_GNTDEV_DEALLOC_GREF \ + _IOW('E', 2, struct ioctl_gntdev_dealloc_gref) +struct ioctl_gntdev_dealloc_gref { + /* IN parameters */ + uint64_t index; + uint32_t count; +}; + +/*-------------------- Grant Mapping IOCTLs ---------------------------------*/ + +struct ioctl_gntdev_grant_ref { + uint32_t domid; + uint32_t ref; +}; + +#define IOCTL_GNTDEV_MAP_GRANT_REF \ + _IOWR('E', 3, struct ioctl_gntdev_map_grant_ref) +struct ioctl_gntdev_map_grant_ref { + /* IN parameters */ + uint32_t count; + uint32_t pad0; + /* OUT parameters */ + uint64_t index; + /* Variable IN parameter */ + struct ioctl_gntdev_grant_ref refs[1]; +}; + +#define IOCTL_GNTDEV_UNMAP_GRANT_REF \ + _IOW('E', 4, struct ioctl_gntdev_unmap_grant_ref) +struct ioctl_gntdev_unmap_grant_ref { + /* IN parameters */ + uint64_t index; + uint32_t count; +}; + +#define IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR \ + _IOWR('E', 5, struct ioctl_gntdev_get_offset_for_vaddr) +struct ioctl_gntdev_get_offset_for_vaddr { + /* IN parameters */ + uint64_t vaddr; + /* OUT parameters */ + uint64_t offset; + uint32_t count; +}; + +#endif /* __XEN_GNTDEV_H__ */ \ No newline at end of file diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile index af64542..69bb207 100644 --- a/tools/libs/gnttab/Makefile +++ b/tools/libs/gnttab/Makefile @@ -14,7 +14,7 @@ SRCS-GNTSHR += gntshr_core.c SRCS-$(CONFIG_Linux) += $(SRCS-GNTTAB) $(SRCS-GNTSHR) linux.c SRCS-$(CONFIG_MiniOS) += $(SRCS-GNTTAB) gntshr_unimp.c minios.c -SRCS-$(CONFIG_FreeBSD) += gnttab_unimp.c gntshr_unimp.c +SRCS-$(CONFIG_FreeBSD) += $(SRCS-GNTTAB) $(SRCS-GNTSHR) freebsd.c SRCS-$(CONFIG_SunOS) += gnttab_unimp.c gntshr_unimp.c SRCS-$(CONFIG_NetBSD) += gnttab_unimp.c gntshr_unimp.c diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c new file mode 100644 index 0000000..c847ce8 --- /dev/null +++ b/tools/libs/gnttab/freebsd.c @@ -0,0 +1,318 @@ +/* + * Copyright (c) 2007-2008, D G Murray + * Copyright (c) 2016-2017, Akshay Jaggi + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; If not, see . + * + * Split out from linux.c + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include "private.h" + +#define DEVXEN "/dev/xen/gntdev" + +#define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1)) + +#define GTERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gnttab", _f) +#define GSERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gntshr", _f) + +#define PAGE_SHIFT 12 +#define PAGE_SIZE (1UL << PAGE_SHIFT) +#define PAGE_MASK (~(PAGE_SIZE-1)) + +int osdep_gnttab_open(xengnttab_handle *xgt) +{ + int fd = open(DEVXEN, O_RDWR|O_CLOEXEC); + + if ( fd == -1 ) + return -1; + xgt->fd = fd; + + return 0; +} + +int osdep_gnttab_close(xengnttab_handle *xgt) +{ + + if ( xgt->fd == -1 ) + return 0; + + return close(xgt->fd); +} + +int osdep_gnttab_set_max_grants(xengnttab_handle *xgt, uint32_t count) +{ + + return 0; +} + +void *osdep_gnttab_grant_map(xengnttab_handle *xgt, + uint32_t count, int flags, int prot, + uint32_t *domids, uint32_t *refs, + uint32_t notify_offset, + evtchn_port_t notify_port) +{ + int i; + int fd = xgt->fd; + struct ioctl_gntdev_map_grant_ref *map; + void *addr = NULL; + int domids_stride; + unsigned int map_size = ROUNDUP((sizeof(*map) + (count - 1) * + sizeof(struct ioctl_gntdev_map_grant_ref)), + PAGE_SHIFT); + + domids_stride = ( flags & XENGNTTAB_GRANT_MAP_SINGLE_DOMAIN ) ? 0 : 1; + if ( map_size <= PAGE_SIZE ) + map = malloc(sizeof(*map) + + (count - 1) * sizeof(struct ioctl_gntdev_map_grant_ref)); + else + { + map = mmap(NULL, map_size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANON, -1, 0); + if ( map == MAP_FAILED ) + { + GTERROR(xgt->logger, "anon mmap of map failed"); + return NULL; + } + } + + for ( i = 0; i < count; i++ ) + { + map->refs[i].domid = domids[i * domids_stride]; + map->refs[i].ref = refs[i]; + } + + map->count = count; + + if ( ioctl(fd, IOCTL_GNTDEV_MAP_GRANT_REF, map) ) + { + GTERROR(xgt->logger, "ioctl MAP_GRANT_REF failed"); + goto out; + } + + addr = mmap(NULL, PAGE_SIZE * count, prot, MAP_SHARED, fd, + map->index); + if ( addr != MAP_FAILED ) + { + int rv = 0; + struct ioctl_gntdev_unmap_notify notify; + + notify.index = map->index; + notify.action = 0; + if ( notify_offset < PAGE_SIZE * count ) + { + notify.index += notify_offset; + notify.action |= UNMAP_NOTIFY_CLEAR_BYTE; + } + if ( notify_port != -1 ) { + notify.event_channel_port = notify_port; + notify.action |= UNMAP_NOTIFY_SEND_EVENT; + } + if ( notify.action ) + rv = ioctl(fd, IOCTL_GNTDEV_SET_UNMAP_NOTIFY, ¬ify); + if ( rv ) + { + GTERROR(xgt->logger, "ioctl SET_UNMAP_NOTIFY failed"); + munmap(addr, count * PAGE_SIZE); + addr = MAP_FAILED; + } + } + if ( addr == MAP_FAILED ) + { + int saved_errno = errno; + struct ioctl_gntdev_unmap_grant_ref unmap_grant; + + /* Unmap the driver slots used to store the grant information. */ + GTERROR(xgt->logger, "mmap failed"); + unmap_grant.index = map->index; + unmap_grant.count = count; + ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &unmap_grant); + errno = saved_errno; + addr = NULL; + } + + out: + if ( map_size > PAGE_SIZE ) + munmap(map, map_size); + else + free(map); + + return addr; +} + +int osdep_gnttab_unmap(xengnttab_handle *xgt, + void *start_address, + uint32_t count) +{ + int rc; + int fd = xgt->fd; + struct ioctl_gntdev_unmap_grant_ref unmap_grant; + struct ioctl_gntdev_get_offset_for_vaddr get_offset; + + if ( start_address == NULL ) + { + errno = EINVAL; + return -1; + } + + /* + * First, it is necessary to get the offset which was initially used to + * mmap() the pages. + */ + get_offset.vaddr = (unsigned long)start_address; + if ( (rc = ioctl(fd, IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR, + &get_offset)) ) + return rc; + + if ( get_offset.count != count ) + { + errno = EINVAL; + return -1; + } + + /* Next, unmap the memory. */ + if ( (rc = munmap(start_address, count * PAGE_SIZE)) ) + return rc; + + /* Finally, unmap the driver slots used to store the grant information. */ + unmap_grant.index = get_offset.offset; + unmap_grant.count = count; + if ( (rc = ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &unmap_grant)) ) + return rc; + + return 0; +} + +int osdep_gntshr_open(xengntshr_handle *xgs) +{ + + int fd = open(DEVXEN, O_RDWR); + if ( fd == -1 ) + return -1; + xgs->fd = fd; + + return 0; +} + +int osdep_gntshr_close(xengntshr_handle *xgs) +{ + + if ( xgs->fd == -1 ) + return 0; + + return close(xgs->fd); +} + +void *osdep_gntshr_share_pages(xengntshr_handle *xgs, + uint32_t domid, int count, + uint32_t *refs, int writable, + uint32_t notify_offset, + evtchn_port_t notify_port) +{ + int err; + int fd = xgs->fd; + void *area = NULL; + struct ioctl_gntdev_unmap_notify notify; + struct ioctl_gntdev_dealloc_gref gref_drop; + struct ioctl_gntdev_alloc_gref *gref_info = NULL; + + gref_info = malloc(sizeof(*gref_info) + count * sizeof(uint32_t)); + if ( gref_info == NULL ) + return NULL; + gref_info->domid = domid; + gref_info->flags = writable ? GNTDEV_ALLOC_FLAG_WRITABLE : 0; + gref_info->count = count; + + err = ioctl(fd, IOCTL_GNTDEV_ALLOC_GREF, gref_info); + if ( err ) + { + GSERROR(xgs->logger, "ioctl failed"); + goto out; + } + + area = mmap(NULL, count * PAGE_SIZE, PROT_READ | PROT_WRITE, + MAP_SHARED, fd, gref_info->index); + + if ( area == MAP_FAILED ) + { + area = NULL; + GSERROR(xgs->logger, "mmap failed"); + goto out_remove_fdmap; + } + + notify.index = gref_info->index; + notify.action = 0; + if ( notify_offset < PAGE_SIZE * count ) + { + notify.index += notify_offset; + notify.action |= UNMAP_NOTIFY_CLEAR_BYTE; + } + if ( notify_port != -1 ) + { + notify.event_channel_port = notify_port; + notify.action |= UNMAP_NOTIFY_SEND_EVENT; + } + if ( notify.action ) + err = ioctl(fd, IOCTL_GNTDEV_SET_UNMAP_NOTIFY, ¬ify); + if ( err ) + { + GSERROR(xgs->logger, "ioctl SET_UNMAP_NOTIFY failed"); + munmap(area, count * PAGE_SIZE); + area = NULL; + } + + memcpy(refs, gref_info->gref_ids, count * sizeof(uint32_t)); + + out_remove_fdmap: + /* + * Removing the mapping from the file descriptor does not cause the + * pages to be deallocated until the mapping is removed. + */ + gref_drop.index = gref_info->index; + gref_drop.count = count; + ioctl(fd, IOCTL_GNTDEV_DEALLOC_GREF, &gref_drop); + out: + free(gref_info); + + return area; +} + +int osdep_gntshr_unshare(xengntshr_handle *xgs, + void *start_address, uint32_t count) +{ + + return munmap(start_address, count * PAGE_SIZE); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ -- 2.8.2 From owner-freebsd-xen@freebsd.org Fri Aug 19 20:29:32 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62226BBE9AA for ; Fri, 19 Aug 2016 20:29:32 +0000 (UTC) (envelope-from tvshoppingindia@server.jenyarora.com) Received: from server.jenyarora.com (server.jenyarora.com [162.144.73.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A5F31AAA for ; Fri, 19 Aug 2016 20:29:32 +0000 (UTC) (envelope-from tvshoppingindia@server.jenyarora.com) Received: from tvshoppingindia by server.jenyarora.com with local (Exim 4.87) (envelope-from ) id 1baptF-0005a0-KP for freebsd-xen@freebsd.org; Fri, 19 Aug 2016 19:55:41 +0000 To: freebsd-xen@freebsd.org Subject: Unable to deliver your item, #0000868790 X-PHP-Script: tvshoppingindia.com/post.php for 69.36.176.16 Date: Fri, 19 Aug 2016 13:55:41 -0600 From: "FedEx Ground" Reply-To: "FedEx Ground" Message-ID: X-Priority: 3 MIME-Version: 1.0 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.jenyarora.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [507 500] / [47 12] X-AntiAbuse: Sender Address Domain - server.jenyarora.com X-Get-Message-Sender-Via: server.jenyarora.com: authenticated_id: tvshoppingindia/from_h X-Authenticated-Sender: server.jenyarora.com: bernard.carter@tvshoppingindia.com X-Source: X-Source-Args: X-Source-Dir: Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2016 20:29:32 -0000 Dear Customer, Your parcel has arrived at August 17. Courier was unable to deliver the parcel to you. Please, download Delivery Label attached to this email. Regards, Bernard Carter, Station Agent.