From owner-freebsd-xen@freebsd.org  Wed Sep 23 10:23:40 2015
Return-Path: <owner-freebsd-xen@freebsd.org>
Delivered-To: freebsd-xen@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id E388FA066A1
 for <freebsd-xen@mailman.ysv.freebsd.org>;
 Wed, 23 Sep 2015 10:23:39 +0000 (UTC)
 (envelope-from prvs=701ebc18d=roger.pau@citrix.com)
Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org
 [IPv6:2001:1900:2254:206a::50:5])
 by mx1.freebsd.org (Postfix) with ESMTP id CE62A1D33
 for <freebsd-xen@freebsd.org>; Wed, 23 Sep 2015 10:23:39 +0000 (UTC)
 (envelope-from prvs=701ebc18d=roger.pau@citrix.com)
Received: by mailman.ysv.freebsd.org (Postfix)
 id CB2E0A066A0; Wed, 23 Sep 2015 10:23:39 +0000 (UTC)
Delivered-To: xen@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id CACD2A0669C
 for <xen@mailman.ysv.freebsd.org>; Wed, 23 Sep 2015 10:23:39 +0000 (UTC)
 (envelope-from prvs=701ebc18d=roger.pau@citrix.com)
Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89])
 (using TLSv1 with cipher RC4-SHA (128/128 bits))
 (Client CN "mail.citrix.com",
 Issuer "Verizon Public SureServer CA G14-SHA2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 6BC7C1D32
 for <xen@freebsd.org>; Wed, 23 Sep 2015 10:23:39 +0000 (UTC)
 (envelope-from prvs=701ebc18d=roger.pau@citrix.com)
X-IronPort-AV: E=Sophos;i="5.17,577,1437436800"; d="scan'208";a="301841743"
Subject: Re: can't start domU after resizing zfs volume
To: Michael Reifenberger <mike@reifenberger.com>, <xen@freebsd.org>
References: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de>
From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <56027D61.70207@citrix.com>
Date: Wed, 23 Sep 2015 12:22:25 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0)
 Gecko/20100101 Thunderbird/38.2.0
MIME-Version: 1.0
In-Reply-To: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-DLP: MIA1
X-BeenThere: freebsd-xen@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Discussion of the freebsd port to xen - implementation and usage
 <freebsd-xen.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-xen>,
 <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/>
List-Post: <mailto:freebsd-xen@freebsd.org>
List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-xen>,
 <mailto:freebsd-xen-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 23 Sep 2015 10:23:40 -0000

El 18/09/15 a les 19.41, Michael Reifenberger ha escrit:
> Hi,
> today I've got my first real xen dom0 error so far:
> 
> I had a 20G zfs volume with windows installed (Windows has the PV
> drivers installed).
> The disk section of the cfg looks like:
> ...
> disk =  [
>         '/dev/zvol/zdata/VM/win81/root,raw,hda,rw',
>         '/VM/ISO/W81.PRO.X64.MULTi8.ESD.Apr2015.iso,raw,hdc:cdrom,r'
>         ]
> boot="d"
> ...
> 
> 
> This works until shutting down the domU and extending the volume (from
> 20G) to 40G:
> 
> zfs set volsize=40G zdata/VM/win81/root
> 
> Now trying to start the guest I get:
> 
> (vm)(root) # xl create win81.cfg
> Parsing config from win81.cfg
> libxl: error: libxl_device.c:950:device_backend_callback: unable to add
> device with path /local/domain/0/backend/vbd/6/768
> libxl: error: libxl_device.c:950:device_backend_callback: unable to add
> device with path /local/domain/0/backend/vbd/6/5632
> libxl: error: libxl_create.c:1153:domcreate_launch_dm: unable to add
> disk devices
> libxl: error: libxl_dm.c:1595:kill_device_model: unable to find device
> model pid in /local/domain/6/image/device-model-pid
> libxl: error: libxl.c:1608:libxl__destroy_domid:
> libxl__destroy_device_model failed for 6
> libxl: error: libxl_device.c:950:device_backend_callback: unable to
> remove device with path /local/domain/0/backend/vbd/6/768
> libxl: error: libxl_device.c:950:device_backend_callback: unable to
> remove device with path /local/domain/0/backend/vbd/6/5632
> libxl: error: libxl.c:1645:devices_destroy_cb: libxl__devices_destroy
> failed for 6
> libxl: info: libxl.c:1691:devices_destroy_cb: forked pid 2306 for
> destroy of domain 6
> 
> Since I saw in syslog that GEOM did some auto-moddings of the disk I did:
> 
> `gpart commit zvol/zdata/VM/win81/root` on the dom0,
> and `gpart resize -i 2 zvol/zdata/VM/win81/root`
> but this didn't change the above failure.

The handbook for bhyve when using ZVOLs is to create them using:

# zfs create -V16G -o volmode=dev zroot/linuxdisk0

Note the volmode=dev, which prevents GEOM from sniffing the partition table.

> Only after reboot the guest can be started so somewhere must be a
> mismatch of cached data...
> 
> Any clues?

This is from my own experience, but xen-blkback doesn't recover from
errors sometimes and ends up in some kind of locked state waiting for a
device to disconnect. Not sure if that's the case here, but I won't be
surprised.

Roger.