Date: Thu, 17 Nov 2016 13:32:04 +0100 From: Jan Bramkamp <crest@rlwinm.de> To: freebsd-emulation@freebsd.org Subject: Re: bhyve: zvols for guest disk - yes or no? Message-ID: <c5e125b0-5022-dd3e-f86d-ec7f8588f624@rlwinm.de> In-Reply-To: <582D97BC.8030801@b1t.name> References: <D991D88D-1327-4580-B6E5-2D59338147C0@punkt.de> <b775f684-98a2-b929-2b13-9753c95fd4f2@rlwinm.de> <D5A6875B-A2AE-4DD9-B941-71146AEF2578@punkt.de> <5be68f57-c9c5-7c20-f590-1beed55fd6bb@rlwinm.de> <582D97BC.8030801@b1t.name>
next in thread | previous in thread | raw e-mail | index | archive | help
On 17/11/2016 12:42, Volodymyr Kostyrko wrote: > Jan Bramkamp wrote: >> An other thing I learned the hard way is that ZVOL are set in stone at >> the ZVOL creation. You have to (cam)dd everything to change the block >> size. The default ZVOL block size is 8K which isn't wrong but your >> guests need to align their file systems (and swap) correctly or you'll >> suffer from write amplification. And ZFS RAID-Z really sucks for such >> small block sizes. Use mirrored VDEVs in your pools or you will suffer >> from massive metadata overhead and disappointing IOPS. > > This pole has two ends though. > > When you are working with file system default 8k block size is too > small. Setting it up to 64k will save a lot writes for host. This is due > to most current filesystems do work correctly with extents/big blocks: > > * Linux: ext4, xfs - full support; > * BSD: ufs - disabled, need to be enabled on format, zfs - full support; > * NTFS: no support, but you can use 64k blocks with Win XP - Win 10 > (though Win XP can't boot off such partition). > > Setting block size bigger makes fragmentation less common (check your > `zpool list`) and saves writes. When guest writes one 64k chunk ZFS > writes 8 separate 8k blocks (+ metadata and stuff) and this is not good > for the speed. > > On the opposite when you have some database inside VM you need to > prepare disk for it accordingly. Guides for using PostgreSQL and MySQL > apply to VM's too. MSSQL on the other hand uses 64k extents to work with > database internally so raising block size to 64k would be good for it. IIRC MySQL with InnoDB uses 16kiB blocks and Postgres uses 8kiB blocks. UFS2 in FreeBSD defaults to 32kiB blocks with 4kiB fragments and you can't raise the block size to more than 64kiB and the documentation recommends keeping the block size to fragement size ratio at 8/1, but I don't know why. This gets even more complicated if you want to use GELI because GELI can't reliably deal with blocks larger than the physical page size (4kiB on i386/amd64). The defaults work out perfectly in this case and changing them can break your systems in interesting ways like GELI trying to allocate a second page worth of memory after processing the first page and failing... If you your main use case for the ZVOLs is MySQL inside the guests you're probably best of with a 16KiB blocksize for the ZVOLs, aligning your partitions to multiples of 16KiB and checking your guest file system parameters as well. Keep in mind that I didn't test this myself. -- Jan Bramkamp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c5e125b0-5022-dd3e-f86d-ec7f8588f624>