From owner-freebsd-virtualization@freebsd.org Wed Mar 20 01:52:38 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B0CF15443B4 for ; Wed, 20 Mar 2019 01:52:38 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D30A80451 for ; Wed, 20 Mar 2019 01:52:37 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=53CR8DbjxNXHgU1PD9OnucoU9ZALOyPksYzTnZIcDJs=; b=Epy191gURcghI2j4DnbfZgpMpP 5Rw3A8fOUF6Ley8Km8obtjXXiQ/P2lwTn/bXUxNJxTySiMVvfjmptRrHlkHsh9XcG//lMxjBThMuT hImLtpyTr0SK0hvUiZNn69uCtv0FGacgpyExxOoHDsLmlJt+xPmyaPKUnsg6WFC37Avk=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1h6QPE-0009Hf-Dn for freebsd-virtualization@freebsd.org; Wed, 20 Mar 2019 08:52:36 +0700 Date: Wed, 20 Mar 2019 08:52:36 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor Message-ID: <20190320015236.GA35511@admin.sibptus.ru> References: <20190319024638.GA8193@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YZ5djTAD1cGYuMQK" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.3 (2019-02-01) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2019 01:52:38 -0000 --YZ5djTAD1cGYuMQK Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Patrick M. Hausen wrote: >=20 > > 1. Does ARC actually cache zfs volumes (not files/datasets)? >=20 > Yes it does. >=20 > > 2. If ARC does cache volumes, does this cache make sense on a hyperviso= r, > > because guest OSes will probably have their own disk cache anyway. >=20 > IMHO not much, because the guest OS is relying on the fact that when > it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, it = will be committed to > stable storage. This is an important point. > > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of > > total RAM, so that most RAM is available to guest machines? >=20 > Yes if you build your own solution on plain FreeBSD. No if you are running > FreeNAS which already tries to autotune the ARC size according to the > memory committed to VMs. >=20 > > 4. What other zfs tuning measures can you suggest for a bhyve > > hypervisor? >=20 > e.g. > zfs set sync=3Dalways zfs/vm >=20 > if zfs/vm is the dataset under which you create the ZVOLs for your emulat= ed > disks. Well, bhyve already has an option for this: The block-device-options are: nocache Open the file with O_DIRECT. direct Open the file using O_SYNC. ro Force the file to be opened read-only. I think something like=20 "-s 4:0,virtio-blk,/dev/zvol/zroot/vm/mail/disk0,direct" would do the same? >=20 > I=E2=80=99m using this for all my VM =E2=80=9Edisks=E2=80=9C and have add= ed a 16 GB SLOG device > to my spinning disk pool - seems to work great. This is on a home system. Is SLOG also used by zfs volumes? --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --YZ5djTAD1cGYuMQK Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEbBAEBAgAGBQJckZzkAAoJEA2k8lmbXsY0uukH+JC3YmtDNjPduavG8UcnK1Dy nMPQwavDdOyKK8v37n1K11ftUbdEOaHC+IWrrszuUxc3Ui/hPG6K3FAaLtLtwllp 6p+64AJ9w9t/nIoMfIGJfmU1BMQuE0JeXcJCaohoDXvyMfCxXApdxlK3iRRYa97T 8ch7Bc+f77cYuwvJkj0eW2pSmANObTsj5kcCLDhnWX9EuBoDpE1rmhfB1R1zxJaa 5PYpNChx8tqyb+O/Usex6pnZ9B7pKzO4lf4mDB13yg6ysWYnCGhECi54yZbE146h OMaJsLpE1DxOe3aOCGsx19o5Fi+/mXb4qm1nwWTE5Jv+/zvtlVxjla+UySB0Dg== =dqCO -----END PGP SIGNATURE----- --YZ5djTAD1cGYuMQK--