From owner-freebsd-stable@freebsd.org Sun Oct 15 11:38:34 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97288E40E51 for ; Sun, 15 Oct 2017 11:38:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36ADA73EA2 for ; Sun, 15 Oct 2017 11:38:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v9FBcWMM083249 for ; Sun, 15 Oct 2017 13:38:32 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 02788A1C; Sun, 15 Oct 2017 13:38:31 +0200 (CEST) Message-ID: <59E348B7.3060102@omnilan.de> Date: Sun, 15 Oct 2017 13:38:31 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: FreeBSD Stable Subject: Re: reboot-less zfs volmode property refresh? References: <59E32B56.6050203@omnilan.de> <59E33109.6010101@omnilan.de> In-Reply-To: <59E33109.6010101@omnilan.de> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Greylist: ACL 129 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Sun, 15 Oct 2017 13:38:32 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Oct 2017 11:38:34 -0000 Bezüglich Harry Schmalzbauer's Nachricht vom 15.10.2017 11:57 (localtime): > Bezüglich Harry Schmalzbauer's Nachricht vom 15.10.2017 11:33 (localtime): … >> 3.) Modify existing volmode=dev dataset and write new GPT >> >> zfs set volmode=geom >> hostPsys/bhyveVOL/sys/test >> >> zfs get volmode >> hostPsys/bhyveVOL/sys/test >> >> NAME PROPERTY VALUE >> SOURCE >> >> hostPsys/bhyveVOL/sys/test volmode geom local >> gpart create -s gpt >> /dev/zvol/hostPsys/bhyveVOL/sys/test >> >> gpart: arg0 'zvol/hostPsys/bhyveVOL/sys/test': Invalid argument >> >> (fails unexpected) >> >> What can I do to let geom(4) know that there's a new device? > Device should read provider. > > I found that last sentence in the zfs(8) man page, describing the > volmode property: > »This property can be changed any time, but > so far it is processed only during volume creation and pool import.« > > So it seems to be a limitation by design. > > re-importing the pool is no option for me, so I'll keep in mind that > changing volmode means outage. > > I'm aware that I can utilize ctl(8) to get access to a volmode=dev > volume, also md(4) might help in that case, but for changing > volmode=geom a reboot / re-import was required. I'ts not correct that md(4) could help here since ms(4) can't use character devices as vnode backend. Just to correct myself. > I guess the benefit of extending the implementation design is much to > small to justify the effort. > But I think making "volmode" a creation-only property (like utf8only) > should be considered. On the other hand, a reboot/re-import might be preferably over zfs send|recv, which would be the only way in that case and which was a regression; better to not consider making it creation-only... I'd like to share two workarounds: The first makes it possible to easily create backups from (bhyve(8)) guests which have ZVOLs as ahci/virtio-block backend. Like described, changing "volmode" property for the corresponding volume dataset from "dev" to "geom" doesn't help without reboot/re-import. But you can clone: zfs snapshot hostPsys/bhyveVOL/sys/guest@offline zfs clone -o volmode=geom hostPsys/bhyveVOL/sys/guest@offline hostPsys/bhyveVOL/sys/guest.gc glabel status gpt/guestSWAP N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp3 gpt/guestBOOT N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp4 gpt/guestSAFE N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp5 gpt/guestVAR N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp6 gpt/guestLOCAL N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp7 gpt/guestDATA N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp8 gpt/guestJbase N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp9 gpt/guestENTITIES N/A zvol/hostPsys/bhyveVOL/sys/guest.gcp10 After you did what ever you need, simply destroy the snapshot and it's clone: zfs destroy -R hostPsys/bhyveVOL/sys/guest@offline This is just a way to work around the volmode limitation for one special case (reading data from the guest's volume). If you need to access the guest volume for other things than non-destructive ones (writing data), you have to go the ctl(4) way. Of course, also for read-only tasks ctl(4) would be appropriate, but since I'm more used to `zfs` than to `ctladm`, the former is easier for me. The latter has more dynamics. Here's the ctl(4) example: ctladm create -b block -d guest-zvol -o file=/dev/zvol/hostPsys/bhyveVOL/sys/guest ctladm port -p `ctladm port -l | grep "camsim.*naa" | cut -w -f 1` -o on geom disk list Geom name: da11 Providers: 1. Name: da11 Mediasize: 8589934592 (8.0G) Sectorsize: 512 Stripesize: 8192 Stripeoffset: 0 Mode: r0w0e0 descr: FREEBSD CTLDISK lunname: FREEBSD guest-zvol lunid: FREEBSD guest-zvol ident: MYSERIAL 0 rotationrate: 0 fwsectors: 63 fwheads: 255 glabel status gpt/guestSWAP N/A da11p3 gpt/guestBOOT N/A da11p4 gpt/guestSAFE N/A da11p5 gpt/guestVAR N/A da11p6 gpt/guestLOCAL N/A da11p7 gpt/guestDATA N/A da11p8 gpt/guestJbase N/A da11p9 gpt/guestENTITIES N/A da11p10 ctladm devlist and ctladm portlist give more info. After you did what ever needed, switch camsim off and remove target: ctladm port -p `ctladm port -l | grep "camsim.*naa" | cut -w -f 1` -o off ctladm remove -b block -l `ctladm devl | grep "guest-zvol" | cut -w -f 2` -harry