From nobody Sat Nov 5 17:39:53 2022 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N4Ps30v9Vz4gl6L for ; Sat, 5 Nov 2022 17:40:07 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N4Ps16Q5Hz3XFm for ; Sat, 5 Nov 2022 17:40:05 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1667669998; bh=gWAUP+1qvhkyweUuMzdB/FyCcQVDACvH5MwBARn35zM=; h=Received:Message-ID:Date:MIME-Version:User-Agent:Subject: Content-Language:To:References:From:In-Reply-To:Content-Type: Content-Transfer-Encoding; b=GdRXJad5RtLzeb+PxBHQgALbBgD9xU1QGRvmME7BgFPXE+F/KDA7iocUJmiqav38H J4mYHph1iXKKVJmu4Jyo5UUlcQVISIYw4Ieiae2yGWQAvdVER9HgZVFbLWzw2uUAIB uZNsVdPDe1tShMPo86E8K33guk622/92+9P76Y/PE0Kq08qcvk6EQe65fxNdRnO4JI dy2x0kOZwMIqmUChn3iKdze6mPirO7Ie/11+vcdPUQ2KnpFicyk/x6ohoTHj9U1aeF dXl/kwxpkNE6h1HFKoxK7PcLczO2s5u4MYV5uiU+MzUHZ+oU30Gv0Seu3DVmepqkrj gj3caAtXYbJCnYqcAO4OHgmHWiY3ltk2By+nBa8jP2fgPJ8UbvxUzL+p7lsuAkWx4C ngt3cNRGwtIzmKW7BYlrSF1cp6L+YyhAtiIsUGZ+tZ7M2h5vNRYP9zFSpmCdq4pB9H cVOGDE2GI/S3QPQtdSSZKUkuAp/o0+u/PIHuyqLpHarklfmnAONWs3I9ey3wL4UmMy O0Hd2DMGYazaPOvk3hjFAuJGEcvISjBGUYnFXWSQ5obfRndGUVGp2odjijWywp5R9E v0h65FPindy0OL5wDRTpOqAN916EWEsuaRY7mfhAOipF+U/KLGWBzt71ZaXLFFH9KC nF4LQuA6oemOsQGhKOhALDKE= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Sat, 5 Nov 2022 10:39:57 -0700 Message-ID: Date: Sat, 5 Nov 2022 10:39:53 -0700 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.0 Subject: Re: Setting up ZFS L2ARC on a zvol Content-Language: en-US To: questions@freebsd.org References: From: David Christensen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4N4Ps16Q5Hz3XFm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=GdRXJad5; dmarc=pass (policy=none) header.from=holgerdanske.com; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 2001:470:0:19b::b869:801b as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; R_SPF_ALLOW(-0.20)[+a:november.he.net]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; MLMMJ_DEST(0.00)[questions@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[holgerdanske.com:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 10/26/22 06:32, julio@meroh.net wrote: > Hello, > > I'm setting up a new machine in which I have an NVMe drive and a bunch of hard disks. The hard disks are going to be a ZFS pool and the NVMe drive is going to be the root file system + the L2ARC for the pool. > > Now... I'm considering using ZFS as well for the root file system (as a separate single-drive pool) in order to simplify admin operations: I want to host some VMs on the NVMe for speed, and using zvols will be very helpful as I don't have to come up with the partition sizes upfront. > > And here comes the question: can the L2ARC of the hard disk pool be backed by a zvol on the NVMe pool (again, so that I don't have to use fixed-size partitions)? > > I gave a simple try to this setup and it's not working, so I'm wondering if this is just not a good idea and thus is unsupported, or if there is a bug: > > root@think:~ # zfs create -V 16G -o primarycache=none zroot/l2arg > root@think:~ # zpool add scratch cache zvol/zroot/l2arc > cannot add to 'scratch': no such pool or dataset > root@think:~ # Oct 26 05:45:28 think ZFS[3677]: vdev problem, zpool=scratch path=/dev/zvol/zroot/l2arc type=ereport.fs.zfs.vdev.open_failed > root@think:~ # ls /dev/zvol/zroot/l2arc > /dev/zvol/zroot/l2arc > > Thanks! ZPOOLCONCEPTS(7) for both 12.3-R [1] and 13.1-R [2]: "Virtual devices cannot be nested, so a mirror or raidz virtual device can only contain files or disks. Mirrors of mirrors (or other combinations) are not allowed. David [1] https://www.freebsd.org/cgi/man.cgi?query=zpoolconcepts&apropos=0&sektion=0&manpath=FreeBSD+12.3-RELEASE+and+Ports&arch=default&format=html [2] https://www.freebsd.org/cgi/man.cgi?query=zpoolconcepts&apropos=0&sektion=0&manpath=FreeBSD+13.1-RELEASE+and+Ports&arch=default&format=html