Date: Wed, 25 Mar 2015 08:44:35 -0700 From: Peter Grehan <grehan@freebsd.org> To: Julian Hsiao <madoka@nyanisore.net> Cc: freebsd-virtualization@freebsd.org Subject: Re: Several bhyve quirks Message-ID: <5512D7E3.9060401@freebsd.org> In-Reply-To: <metut1$b1f$1@ger.gmane.org> References: <metut1$b1f$1@ger.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Julian, I'll let Neel take care of the time questions. > 3. Even moderate guest disk I/O completely kills guest network > performance. For example, whenever security(8) (security(7) in FreeBSD) > runs, guest network throughput drops from 150+ Mbps to ~20 Mbps, and > jitter from ping jumps from <0.01 ms to 100+ ms. If I try to build > something in the guest, then network becomes almost unusable. > > The network performance degradation only affects the guest that's > generating the I/O; high I/O on guest B doesn't affect guest A, nor > would high I/O on the host. > > I'm using both virtio-blk and virio-net drivers, and the guests' disk > images are backed by zvol+geli. Removing geli has no effect. > > There are some commits in CURRENT that suggests improved virtio > performance, but I'm not comfortable running CURRENT. Is there a > workaround I could use for 10.1? In 10.1, virtio-blk i/o is done sychronously in the context of the guest vCPU exit. If it's a single vCPU guest, or the virtio-net interrupt happens to be delivered to that vCPU, performance will suffer. A workaround is to use ahci-hd for the disk emulation and not virtio-blk. The AHCI emulation does i/o in a dedicated thread and doesn't block the vCPU thread. > 4. virtio-blk always reports the virtual disk as having 512-byte > sectors, and so I get I/O errors on OpenBSD guests when the disk image > is backed by zvol+geli with 4K sector size. Curiously, this only seems > to affect zvol+geli; with just zvol it seems to work. Also, it works > either way on Linux guests. > > ATM I changed the zvol / geli sector size to 512 bytes, which probably > made #2 worse. I think this bug / feature is addressed by: > <https://github.com/freebsd/freebsd/commit/02e846756ee99b849987a9bb6f57566fc70360c7>, > but again is there a workaround to force a specific sector size for 10.1? The only workaround for 10.1 would be to use ahci-hd instead of virtio-blk. The correct sector size will be reported there. > 5. This may be better directed at OpenBSD but I'll ask here anyway: if I > enable virtio-rnd then OpenBSD would not boot with "couldn't map > interrupt" error. The kernel in bsd.rd will boot, but not the installed > kernel (or the one built from STABLE; I forgot). Again, Linux seems > unaffected, but I couldn't tell if it's actually working. Try using the -W option to bhyve. This will force the bhyve virtio code to advertize (non-standard) MSI interrupt capability which OpenBSD will then use to allocate vectors. later, Peter.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5512D7E3.9060401>