From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 25 15:44:48 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91BE8A45 for ; Wed, 25 Mar 2015 15:44:48 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9DF8E0 for ; Wed, 25 Mar 2015 15:44:47 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id D2E25280F61 for ; Thu, 26 Mar 2015 01:44:38 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zAs27BruUC8O for ; Thu, 26 Mar 2015 01:44:38 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-76-126-65-88.hsd1.ca.comcast.net [76.126.65.88]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 974F7280F5C; Thu, 26 Mar 2015 01:44:36 +1000 (EST) Message-ID: <5512D7E3.9060401@freebsd.org> Date: Wed, 25 Mar 2015 08:44:35 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Julian Hsiao Subject: Re: Several bhyve quirks References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2015 15:44:48 -0000 Hi Julian, I'll let Neel take care of the time questions. > 3. Even moderate guest disk I/O completely kills guest network > performance. For example, whenever security(8) (security(7) in FreeBSD) > runs, guest network throughput drops from 150+ Mbps to ~20 Mbps, and > jitter from ping jumps from <0.01 ms to 100+ ms. If I try to build > something in the guest, then network becomes almost unusable. > > The network performance degradation only affects the guest that's > generating the I/O; high I/O on guest B doesn't affect guest A, nor > would high I/O on the host. > > I'm using both virtio-blk and virio-net drivers, and the guests' disk > images are backed by zvol+geli. Removing geli has no effect. > > There are some commits in CURRENT that suggests improved virtio > performance, but I'm not comfortable running CURRENT. Is there a > workaround I could use for 10.1? In 10.1, virtio-blk i/o is done sychronously in the context of the guest vCPU exit. If it's a single vCPU guest, or the virtio-net interrupt happens to be delivered to that vCPU, performance will suffer. A workaround is to use ahci-hd for the disk emulation and not virtio-blk. The AHCI emulation does i/o in a dedicated thread and doesn't block the vCPU thread. > 4. virtio-blk always reports the virtual disk as having 512-byte > sectors, and so I get I/O errors on OpenBSD guests when the disk image > is backed by zvol+geli with 4K sector size. Curiously, this only seems > to affect zvol+geli; with just zvol it seems to work. Also, it works > either way on Linux guests. > > ATM I changed the zvol / geli sector size to 512 bytes, which probably > made #2 worse. I think this bug / feature is addressed by: > , > but again is there a workaround to force a specific sector size for 10.1? The only workaround for 10.1 would be to use ahci-hd instead of virtio-blk. The correct sector size will be reported there. > 5. This may be better directed at OpenBSD but I'll ask here anyway: if I > enable virtio-rnd then OpenBSD would not boot with "couldn't map > interrupt" error. The kernel in bsd.rd will boot, but not the installed > kernel (or the one built from STABLE; I forgot). Again, Linux seems > unaffected, but I couldn't tell if it's actually working. Try using the -W option to bhyve. This will force the bhyve virtio code to advertize (non-standard) MSI interrupt capability which OpenBSD will then use to allocate vectors. later, Peter.