From owner-freebsd-xen@freebsd.org Sun Sep 20 21:33:49 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 956B6A06744 for ; Sun, 20 Sep 2015 21:33:49 +0000 (UTC) (envelope-from thinker.lp@gmail.com) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33A661C98 for ; Sun, 20 Sep 2015 21:33:49 +0000 (UTC) (envelope-from thinker.lp@gmail.com) Received: by wicge5 with SMTP id ge5so92389333wic.0 for ; Sun, 20 Sep 2015 14:33:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=XIMEFFcHVc5hqvqDb4JiZLd9Ih+8yzQvjtSLjkfivEE=; b=Fr+HwCu3jxuY25wETCfOnnkdP/1oXlEdkYq6CrXL2HjXcatfd2QM2xXCHTfnPdBu7X nQYu28dH6LnE0soZwFkvKZY1CUY5QeQ8jp2UYQooIO3x6CIGZ8Iap74jdCCD5kdnvdzJ N1TnslxHRY0QQsT0nkbOhgYhdGcf4Zp9Ds7SCTIJ1xOKDOaeeHjNEZtgaHfxq/wNB5Xc Buo1zK3J88b7TBhGkQv/Vf9FhLgDPCYZ6NTWwACXXBBZ3H6YiHkMkXC0FxnX8bPxNA6c YN0E5nQ+jiiEftzQiks+ujvaXE1uAqRGQlQb2XlhRjo4sBLOOl6DpjXXnlDz0nPK64D0 2s4Q== MIME-Version: 1.0 X-Received: by 10.194.114.37 with SMTP id jd5mr18882049wjb.34.1442784827404; Sun, 20 Sep 2015 14:33:47 -0700 (PDT) Received: by 10.28.49.197 with HTTP; Sun, 20 Sep 2015 14:33:47 -0700 (PDT) Date: Mon, 21 Sep 2015 00:33:47 +0300 Message-ID: Subject: kern/154428: xn0 network interface and PF - Massive performance drop From: Jonas Liepuonius To: freebsd-xen@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 21:33:49 -0000 Hello, facing the same problem with FreeBSD 10.2 on XenServer 6.5. Disabling PF entirely fixes the problem, and the workarounds I found ( "ifconfig xn0 -lro -tso" and "sysctl net.inet.tcp.tso=0" ) don't help unfortunately. -- Regards, Jonas From owner-freebsd-xen@freebsd.org Sun Sep 20 23:04:17 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 983B2A054E4 for ; Sun, 20 Sep 2015 23:04:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84CB919AF for ; Sun, 20 Sep 2015 23:04:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8KN4H43059999 for ; Sun, 20 Sep 2015 23:04:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-xen@FreeBSD.org Subject: [Bug 154428] [xen] xn0 network interface and PF - Massive performance drop Date: Sun, 20 Sep 2015 23:04:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: thinker.lp@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-xen@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2015 23:04:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154428 Jonas Liepuonius changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |thinker.lp@gmail.com --- Comment #12 from Jonas Liepuonius --- Hello, so I've encountered the same problem on FreeBSD 10.2 on XenServer 6.5. Enabling pf basically shuts off my interfaces, pings go through but nothing else. I tried setting net.inet.tcp.tso=0 on boot in /etc/sysctl.conf and setting it manually with sysctl but with no luck. ifconfig xn0 -tso -lro didn't help me either. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-xen@freebsd.org Mon Sep 21 10:36:23 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58F4CA05680 for ; Mon, 21 Sep 2015 10:36:23 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from smtp.krpservers.com (smtp.krpservers.com [62.13.128.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.krpservers.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 002661AF5 for ; Mon, 21 Sep 2015 10:36:22 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from [10.12.30.106] (vpn01-01.tdx.co.uk [62.13.130.213] (may be forged)) (authenticated bits=0) by smtp.krpservers.com (8.14.9/8.14.9) with ESMTP id t8LAZM6j014823 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 21 Sep 2015 11:35:23 +0100 (BST) (envelope-from kpielorz_lst@tdx.co.uk) Date: Mon, 21 Sep 2015 11:35:52 +0100 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , Adam McDougall , freebsd-xen@freebsd.org Subject: Re: XenServer 6.5(SP1) - HVM 're0: watchdog timeout' errors - summary... Message-ID: <706DF24C7398EB8E313504F5@[10.12.30.106]> In-Reply-To: <34C596D4CD79963C95725C41@[10.12.30.106]> References: <9CC4073DE17E3C8BD01EF7D3@[10.12.30.106]> <55EDA470.1000007@citrix.com> <55EDC55B.8000004@egr.msu.edu> <762F895FF087C3BBB6DB94DA@[10.12.30.106]> <55EFF61A.3020409@citrix.com> <34C596D4CD79963C95725C41@[10.12.30.106]> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 10:36:23 -0000 In the vain hope this'll help others having this issue... Having looked at this now for some time - and run a lot of tests, the current best solution to allow a FreeBSD domU under XenServer 6.5 to act as a gateway, or run OpenVPN (or dhcpd etc.) and remain agile - is to switch to VirtIO NIC's. em1000 (em) causes migrations to fail at the destination end. Realtek (re) causes 'watchdog errors' - and if you get rid of the watchdog code from the driver, you don't see the errors - but the NIC's won't pass traffic after a migrate either. PV (xn) have the original problem of not being able to provide routed traffic from one FreeBSD domU to other domU guests, don't work with OpenVPN (apparent 'packet size issues' from packet collation), and don't work for hosting dhcpd. virtio (vtnet) Work for routing for other domU's, work for OpenVPN, work for dhcpd. The *only* disadvantage [aside from possibly performance] I've found is that if you migrate a PV (xn) based FreeBSD domU you suffer around 2-3 seconds of network disconnect. With a VirtIO (vtnet) interface this can be as high as 18 seconds of network disconnect when you migrate. This means there's more of a chance of disrupting sessions that were active either on, or through the DomU. But it does work. So until netfront is fixed (where we can hopefully go back to the undoubtedly more efficient PV 'xn' NIC's) - this is the best workaround for us - to use virtio (vtnet). -Karl From owner-freebsd-xen@freebsd.org Mon Sep 21 19:50:47 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32DC6A06782 for ; Mon, 21 Sep 2015 19:50:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F32411B0 for ; Mon, 21 Sep 2015 19:50:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t8LJokb2091241 for ; Mon, 21 Sep 2015 19:50:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-xen@FreeBSD.org Subject: [Bug 154428] [xen] xn0 network interface and PF - Massive performance drop Date: Mon, 21 Sep 2015 19:50:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bdrewery@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-xen@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 19:50:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154428 --- Comment #13 from Bryan Drewery --- (In reply to Bryan Drewery from comment #11) > I just hit this on 10.1-GENERIC on EC2. Empty pf.conf with pf enabled = > horrible performance. Disabling pf or TSO with pf fixes it. My exact issue with with ifconfig having TSO4 on the interfaces. Just changing the sysctl to 0 fixes the issue. Setting to 1 brings back the extreme packet loss/latency issues. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-xen@freebsd.org Wed Sep 23 10:23:40 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E388FA066A1 for ; Wed, 23 Sep 2015 10:23:39 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CE62A1D33 for ; Wed, 23 Sep 2015 10:23:39 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: by mailman.ysv.freebsd.org (Postfix) id CB2E0A066A0; Wed, 23 Sep 2015 10:23:39 +0000 (UTC) Delivered-To: xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CACD2A0669C for ; Wed, 23 Sep 2015 10:23:39 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Verizon Public SureServer CA G14-SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6BC7C1D32 for ; Wed, 23 Sep 2015 10:23:39 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.17,577,1437436800"; d="scan'208";a="301841743" Subject: Re: can't start domU after resizing zfs volume To: Michael Reifenberger , References: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de> From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Message-ID: <56027D61.70207@citrix.com> Date: Wed, 23 Sep 2015 12:22:25 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 10:23:40 -0000 El 18/09/15 a les 19.41, Michael Reifenberger ha escrit: > Hi, > today I've got my first real xen dom0 error so far: > > I had a 20G zfs volume with windows installed (Windows has the PV > drivers installed). > The disk section of the cfg looks like: > ... > disk = [ > '/dev/zvol/zdata/VM/win81/root,raw,hda,rw', > '/VM/ISO/W81.PRO.X64.MULTi8.ESD.Apr2015.iso,raw,hdc:cdrom,r' > ] > boot="d" > ... > > > This works until shutting down the domU and extending the volume (from > 20G) to 40G: > > zfs set volsize=40G zdata/VM/win81/root > > Now trying to start the guest I get: > > (vm)(root) # xl create win81.cfg > Parsing config from win81.cfg > libxl: error: libxl_device.c:950:device_backend_callback: unable to add > device with path /local/domain/0/backend/vbd/6/768 > libxl: error: libxl_device.c:950:device_backend_callback: unable to add > device with path /local/domain/0/backend/vbd/6/5632 > libxl: error: libxl_create.c:1153:domcreate_launch_dm: unable to add > disk devices > libxl: error: libxl_dm.c:1595:kill_device_model: unable to find device > model pid in /local/domain/6/image/device-model-pid > libxl: error: libxl.c:1608:libxl__destroy_domid: > libxl__destroy_device_model failed for 6 > libxl: error: libxl_device.c:950:device_backend_callback: unable to > remove device with path /local/domain/0/backend/vbd/6/768 > libxl: error: libxl_device.c:950:device_backend_callback: unable to > remove device with path /local/domain/0/backend/vbd/6/5632 > libxl: error: libxl.c:1645:devices_destroy_cb: libxl__devices_destroy > failed for 6 > libxl: info: libxl.c:1691:devices_destroy_cb: forked pid 2306 for > destroy of domain 6 > > Since I saw in syslog that GEOM did some auto-moddings of the disk I did: > > `gpart commit zvol/zdata/VM/win81/root` on the dom0, > and `gpart resize -i 2 zvol/zdata/VM/win81/root` > but this didn't change the above failure. The handbook for bhyve when using ZVOLs is to create them using: # zfs create -V16G -o volmode=dev zroot/linuxdisk0 Note the volmode=dev, which prevents GEOM from sniffing the partition table. > Only after reboot the guest can be started so somewhere must be a > mismatch of cached data... > > Any clues? This is from my own experience, but xen-blkback doesn't recover from errors sometimes and ends up in some kind of locked state waiting for a device to disconnect. Not sure if that's the case here, but I won't be surprised. Roger. From owner-freebsd-xen@freebsd.org Wed Sep 23 14:36:53 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 259BAA06336 for ; Wed, 23 Sep 2015 14:36:53 +0000 (UTC) (envelope-from mike@reifenberger.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0B4DF1FF1 for ; Wed, 23 Sep 2015 14:36:53 +0000 (UTC) (envelope-from mike@reifenberger.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0A954A06334; Wed, 23 Sep 2015 14:36:53 +0000 (UTC) Delivered-To: xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4716A06333 for ; Wed, 23 Sep 2015 14:36:52 +0000 (UTC) (envelope-from mike@reifenberger.com) Received: from mail.eeeit.de (mail.eeeit.de [37.120.160.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77F881FED for ; Wed, 23 Sep 2015 14:36:51 +0000 (UTC) (envelope-from mike@reifenberger.com) Received: from localhost (localhost [127.0.0.1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mike@reifenberger.com) by mail.eeeit.de (Postfix) with ESMTPSA id 2A35E35BB; Wed, 23 Sep 2015 16:36:43 +0200 (CEST) Received: from 212.144.53.198 ([212.144.53.198]) by mail.eeeit.de (Horde Framework) with HTTP; Wed, 23 Sep 2015 16:36:42 +0200 Date: Wed, 23 Sep 2015 16:36:42 +0200 Message-ID: <20150923163642.Horde.C2gq8tfwkC45mOK8NwCIAj-@mail.eeeit.de> From: Michael Reifenberger To: Roger Pau =?utf-8?b?TW9ubsOp?= Cc: xen@freebsd.org Subject: Re: can't start domU after resizing zfs volume References: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de> <56027D61.70207@citrix.com> In-Reply-To: <56027D61.70207@citrix.com> User-Agent: Horde Application Framework 5 Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 14:36:53 -0000 Zitat von Roger Pau Monné : > El 18/09/15 a les 19.41, Michael Reifenberger ha escrit: >> Hi, >> today I've got my first real xen dom0 error so far: >> >> I had a 20G zfs volume with windows installed (Windows has the PV >> drivers installed). >> The disk section of the cfg looks like: >> ... >> disk = [ >> '/dev/zvol/zdata/VM/win81/root,raw,hda,rw', >> '/VM/ISO/W81.PRO.X64.MULTi8.ESD.Apr2015.iso,raw,hdc:cdrom,r' >> ] >> boot="d" >> ... >> >> >> This works until shutting down the domU and extending the volume (from >> 20G) to 40G: >> >> zfs set volsize=40G zdata/VM/win81/root >> >> Now trying to start the guest I get: >> >> (vm)(root) # xl create win81.cfg >> Parsing config from win81.cfg >> libxl: error: libxl_device.c:950:device_backend_callback: unable to add >> device with path /local/domain/0/backend/vbd/6/768 >> libxl: error: libxl_device.c:950:device_backend_callback: unable to add >> device with path /local/domain/0/backend/vbd/6/5632 >> libxl: error: libxl_create.c:1153:domcreate_launch_dm: unable to add >> disk devices >> libxl: error: libxl_dm.c:1595:kill_device_model: unable to find device >> model pid in /local/domain/6/image/device-model-pid >> libxl: error: libxl.c:1608:libxl__destroy_domid: >> libxl__destroy_device_model failed for 6 >> libxl: error: libxl_device.c:950:device_backend_callback: unable to >> remove device with path /local/domain/0/backend/vbd/6/768 >> libxl: error: libxl_device.c:950:device_backend_callback: unable to >> remove device with path /local/domain/0/backend/vbd/6/5632 >> libxl: error: libxl.c:1645:devices_destroy_cb: libxl__devices_destroy >> failed for 6 >> libxl: info: libxl.c:1691:devices_destroy_cb: forked pid 2306 for >> destroy of domain 6 >> >> Since I saw in syslog that GEOM did some auto-moddings of the disk I did: >> >> `gpart commit zvol/zdata/VM/win81/root` on the dom0, >> and `gpart resize -i 2 zvol/zdata/VM/win81/root` >> but this didn't change the above failure. > > The handbook for bhyve when using ZVOLs is to create them using: > > # zfs create -V16G -o volmode=dev zroot/linuxdisk0 > > Note the volmode=dev, which prevents GEOM from sniffing the partition table. > Thats at least a workaround! Sometimes it would be nice to be able to access/pre-fill domU slices/partitions on dom0 as well... >> Only after reboot the guest can be started so somewhere must be a >> mismatch of cached data... >> >> Any clues? > > This is from my own experience, but xen-blkback doesn't recover from > errors sometimes and ends up in some kind of locked state waiting for a > device to disconnect. Not sure if that's the case here, but I won't be > surprised. How does xen-blkback construct this Path: /local/domain/0/backend/vbd/6/768 or /local/domain/0/backend/vbd/6/5632? Is the volmode=dev changable after creation or only at creation time? BTW: Many thanks for supporting XEN-dom0 under FreeBSD/ZFS. So far it works surprisingly stable (Except some minor glitches like the above) :-) Thanks! Greetings --- Michael Gruß --- Michael Reifenberger From owner-freebsd-xen@freebsd.org Wed Sep 23 14:54:01 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07754A070FB for ; Wed, 23 Sep 2015 14:54:01 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E47E41D84 for ; Wed, 23 Sep 2015 14:54:00 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: by mailman.ysv.freebsd.org (Postfix) id E3B37A070F9; Wed, 23 Sep 2015 14:54:00 +0000 (UTC) Delivered-To: xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C86A8A070F7 for ; Wed, 23 Sep 2015 14:54:00 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Verizon Public SureServer CA G14-SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E4081D81 for ; Wed, 23 Sep 2015 14:54:00 +0000 (UTC) (envelope-from prvs=701ebc18d=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.17,577,1437436800"; d="scan'208";a="301905679" Subject: Re: can't start domU after resizing zfs volume To: Michael Reifenberger References: <20150918194154.Horde.PQcchwucJFPQY4U0K75MgpW@mail.eeeit.de> <56027D61.70207@citrix.com> <20150923163642.Horde.C2gq8tfwkC45mOK8NwCIAj-@mail.eeeit.de> CC: From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= X-Enigmail-Draft-Status: N1110 Message-ID: <5602BD02.7050004@citrix.com> Date: Wed, 23 Sep 2015 16:53:54 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20150923163642.Horde.C2gq8tfwkC45mOK8NwCIAj-@mail.eeeit.de> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2015 14:54:01 -0000 El 23/09/15 a les 16.36, Michael Reifenberger ha escrit: > > Zitat von Roger Pau Monné : > >> El 18/09/15 a les 19.41, Michael Reifenberger ha escrit: >>> Hi, >>> today I've got my first real xen dom0 error so far: >>> >>> I had a 20G zfs volume with windows installed (Windows has the PV >>> drivers installed). >>> The disk section of the cfg looks like: >>> ... >>> disk = [ >>> '/dev/zvol/zdata/VM/win81/root,raw,hda,rw', >>> '/VM/ISO/W81.PRO.X64.MULTi8.ESD.Apr2015.iso,raw,hdc:cdrom,r' >>> ] >>> boot="d" >>> ... >>> >>> >>> This works until shutting down the domU and extending the volume (from >>> 20G) to 40G: >>> >>> zfs set volsize=40G zdata/VM/win81/root >>> >>> Now trying to start the guest I get: >>> >>> (vm)(root) # xl create win81.cfg >>> Parsing config from win81.cfg >>> libxl: error: libxl_device.c:950:device_backend_callback: unable to add >>> device with path /local/domain/0/backend/vbd/6/768 >>> libxl: error: libxl_device.c:950:device_backend_callback: unable to add >>> device with path /local/domain/0/backend/vbd/6/5632 >>> libxl: error: libxl_create.c:1153:domcreate_launch_dm: unable to add >>> disk devices >>> libxl: error: libxl_dm.c:1595:kill_device_model: unable to find device >>> model pid in /local/domain/6/image/device-model-pid >>> libxl: error: libxl.c:1608:libxl__destroy_domid: >>> libxl__destroy_device_model failed for 6 >>> libxl: error: libxl_device.c:950:device_backend_callback: unable to >>> remove device with path /local/domain/0/backend/vbd/6/768 >>> libxl: error: libxl_device.c:950:device_backend_callback: unable to >>> remove device with path /local/domain/0/backend/vbd/6/5632 >>> libxl: error: libxl.c:1645:devices_destroy_cb: libxl__devices_destroy >>> failed for 6 >>> libxl: info: libxl.c:1691:devices_destroy_cb: forked pid 2306 for >>> destroy of domain 6 >>> >>> Since I saw in syslog that GEOM did some auto-moddings of the disk I >>> did: >>> >>> `gpart commit zvol/zdata/VM/win81/root` on the dom0, >>> and `gpart resize -i 2 zvol/zdata/VM/win81/root` >>> but this didn't change the above failure. >> >> The handbook for bhyve when using ZVOLs is to create them using: >> >> # zfs create -V16G -o volmode=dev zroot/linuxdisk0 >> >> Note the volmode=dev, which prevents GEOM from sniffing the partition >> table. >> > > Thats at least a workaround! > Sometimes it would be nice to be able to access/pre-fill domU > slices/partitions on > dom0 as well... I agree, I've never used volmode=dev and never had problems on Xen Dom0, but I've also never tried to resize a volume. I will try to reproduce this on my setup in order to figure out what's going on, but it won't be today :(. > >>> Only after reboot the guest can be started so somewhere must be a >>> mismatch of cached data... >>> >>> Any clues? >> >> This is from my own experience, but xen-blkback doesn't recover from >> errors sometimes and ends up in some kind of locked state waiting for a >> device to disconnect. Not sure if that's the case here, but I won't be >> surprised. > > How does xen-blkback construct this Path: > /local/domain/0/backend/vbd/6/768 or > /local/domain/0/backend/vbd/6/5632? Those are created and mostly populated by the toolstack (xl), xen-blkback only writes the set of features it supports there, but the physical disk information is provided by the toolstack. > Is the volmode=dev changable after creation or only at creation time? I would expect so, but I'm no ZFS guru, so it's probably best to search the man pages. I guess as a workaround, or unless why this happens is better understood to me, it's better to set volmode=dev and unset it if you wish to access the partitions inside the ZVOL. > > BTW: Many thanks for supporting XEN-dom0 under FreeBSD/ZFS. > So far it works surprisingly stable (Except some minor glitches like the > above) :-) Thanks for testing it! Roger.