Date: Wed, 20 Mar 2019 09:34:41 +0000 From: Matt Churchyard <matt.churchyard@userve.net> To: Victor Sudakov <vas@mpeks.tomsk.su>, "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: RE: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor Message-ID: <6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com> In-Reply-To: <20190320015236.GA35511@admin.sibptus.ru> References: <20190319024638.GA8193@admin.sibptus.ru> <AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de> <20190320015236.GA35511@admin.sibptus.ru>
index | next in thread | previous in thread | raw e-mail
Patrick M. Hausen wrote: > > > 1. Does ARC actually cache zfs volumes (not files/datasets)? > > Yes it does. I find this distinction between volumes/files/etc and what is cached causes confusion (as well as "volumes not datasets"). Both ZVOLs and Z file systems are types of dataset. A dataset stores data in records (usually up to 128kb in size). It's these records that are cached (and that most ZFS functions such as compression/raidz/zil/etc work with) As far as the ZFS lower levels are concerned, there is no difference between a volume and a file system. > > > 2. If ARC does cache volumes, does this cache make sense on a > > hypervisor, because guest OSes will probably have their own disk cache anyway. > > IMHO not much, because the guest OS is relying on the fact that when > it writes it’s own cached data out to „disk“, it will be committed to > stable storage. Maybe I've missed something but I don't quite get the link between read cache (ARC) and guest writes here? > This is an important point. > > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less > > of total RAM, so that most RAM is available to guest machines? > > Yes if you build your own solution on plain FreeBSD. No if you are > running FreeNAS which already tries to autotune the ARC size according > to the memory committed to VMs. > > > 4. What other zfs tuning measures can you suggest for a bhyve > > hypervisor? > > e.g. > zfs set sync=always zfs/vm > > if zfs/vm is the dataset under which you create the ZVOLs for your > emulated disks. >Well, bhyve already has an option for this: >The block-device-options are: >nocache Open the file with O_DIRECT. >direct Open the file using O_SYNC. >ro Force the file to be opened read-only. >I think something like >"-s 4:0,virtio-blk,/dev/zvol/zroot/vm/mail/disk0,direct" >would do the same? > > I’m using this for all my VM „disks“ and have added a 16 GB SLOG > device to my spinning disk pool - seems to work great. This is on a home system. > Is SLOG also used by zfs volumes? As above, the core of ZFS doesn't care what type of dataset it is working with. ARC/ZIL/etc all work exactly the same. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6388ebd5a12949179ff6da33eb107205>