Date: Thu, 17 Nov 2016 11:36:17 +0100 From: "Patrick M. Hausen" <hausen@punkt.de> To: freebsd-emulation@freebsd.org Subject: Re: bhyve: zvols for guest disk - yes or no? Message-ID: <46273DBA-06F8-44F2-88AD-C4812E249ED8@punkt.de> In-Reply-To: <5be68f57-c9c5-7c20-f590-1beed55fd6bb@rlwinm.de> References: <D991D88D-1327-4580-B6E5-2D59338147C0@punkt.de> <b775f684-98a2-b929-2b13-9753c95fd4f2@rlwinm.de> <D5A6875B-A2AE-4DD9-B941-71146AEF2578@punkt.de> <5be68f57-c9c5-7c20-f590-1beed55fd6bb@rlwinm.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, all, > Am 17.11.2016 um 11:16 schrieb Jan Bramkamp <crest@rlwinm.de>: > Bhyve doesn't support page deduplication but it doesn't wire down = guest memory unless you asked it to, but you have to wire down guest = memory to use PCI passthrough. If I were picking VM hosts today I would = go with LGA 2011 v3 boards having least eight DDR4 slots per socket. = Maybe use some nice >=3D 2TB NVMe SSDs and suddenly your limited by CPU = cycles and storage space instead of IOPS and RAM. Perhaps I'll describe the intention of this project. All your = suggestions are great for a heavy virtualisation platform. In our case we just want a cheap, yet "reliable enough" platform to host a handful of rocket.chat instances. While rocket.chat claims FreeBSD support that's way behind compared to running the product on Linux. We found Ubuntu to be a good choice in this case. Unfortunately the application's architecture doesn't easily fit running multiple instances on one kernel/OS. The absence of current FreeBSD support rules out jails. So a full-fledged hypervisor is called for. And we will use a system with two mirrored SATA disks to run no more than 10 Ubuntu guests with at most 4 GB each and 100 GB of guest disk space. 1 or 2 TB of spinning rust in a mirror, that's all. At least rocket.chat is not performance hungry. The only issue I saw is RAM (CPU supports at most 32 GB) and the ZVOL "issue" I initially asked about. We will now provision them = "sparse" with a small block size ... > An other thing I learned the hard way is that ZVOL are set in stone at = the ZVOL creation. You have to (cam)dd everything to change the block = size. The default ZVOL block size is 8K which isn't wrong but your = guests need to align their file systems (and swap) correctly or you'll = suffer from write amplification. And ZFS RAID-Z really sucks for such = small block sizes. Use mirrored VDEVs in your pools or you will suffer = from massive metadata overhead and disappointing IOPS. I attended Kirk's kernel class on last EuroBSDCon ;-) You are = essentially preaching to the choir - but for the list archives and later searches = sake, your work will probably do somebody else some good. Thanks again, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=C3=BCrgen Egeling AG Mannheim 108285
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46273DBA-06F8-44F2-88AD-C4812E249ED8>