From owner-freebsd-xen@freebsd.org Tue Dec 6 03:48:05 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8216DC6889C for ; Tue, 6 Dec 2016 03:48:05 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7449B0 for ; Tue, 6 Dec 2016 03:48:05 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x22f.google.com with SMTP id a124so632955873ioe.2 for ; Mon, 05 Dec 2016 19:48:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=YxXKdA8znRxPPHfB8nq2d/XJYjmn5UhWja6qtF4LmEM=; b=iVbuh/sJrIWDJQAkIr/isHXxKOIyLGu4Y/R+PydS01t50HNVh6nqKeXn/W5swMy5fk fRZ9ncy5cJ8WAg1zGtolTgUTXC+HZs27jToNc0Ij3w+3ZelNTL2V2pvlDGrhPZrsedrh bi0NOQ4PGGJ61rKZ5bZhuoq5EdyDXP5iVvzhRVTF2V3YFqOhaauq0oY+A2FYU9XVfdEw nftrZnjVB9BpGHw7ZQB8MF0nau2L6GhgUjNtZkcqROsnNfW4cdNt8Y+bliV7nXUd+419 VvzLD/sl/LmLGDcJ+ajwLKzBVPldf4PLHyBVctB3/VKbiv8W55zs7LozwsIeX5jOjNPz kJtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=YxXKdA8znRxPPHfB8nq2d/XJYjmn5UhWja6qtF4LmEM=; b=AfpHAAUHuRbc3Dkf98zg6AU+FB4yYxvsOrzZWdJcE6A+TPQ6Zl4suUDEeXKW/PwDmJ sUmJAHV2DyCHsjvkBQV1AkxOY9UlZsWWWxohCKBhM1J+h+FJFuII0lGri97QaaCyX+9z uP6ZJjkc+b0JDh0fw9leYiGTjvYZZc0MtjpZ5FC/EO1tVFWxRM9KB7aAbvoYIcrtSx4J vUoXpsxR9sL/9oexVNaRLK9NIBPH6KkpQkgj6sCaJ3sKgr3nXuZoAjMDXKla+HOEkyZ/ jsdetTIqZTGttaB9H0qloyldZvPfIg95pRXhhKCYIPvkMVVHkxE6dxK184BwHIOgFjgC EqeQ== X-Gm-Message-State: AKaTC00Os+FrgHRudDwZCuG80MdfcMfGyqy4PnmLwhocAK0clUyvVOQhAJZQ8bRrAS3y+g== X-Received: by 10.107.8.105 with SMTP id 102mr49023232ioi.154.1480996084526; Mon, 05 Dec 2016 19:48:04 -0800 (PST) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [50.66.15.125]) by smtp.gmail.com with ESMTPSA id p71sm8360695ioe.33.2016.12.05.19.48.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Dec 2016 19:48:04 -0800 (PST) To: freebsd-xen@freebsd.org From: Nathan Friess Subject: 11-RELEASE acting as vbd backend Message-ID: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> Date: Mon, 5 Dec 2016 20:48:00 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2016 03:48:05 -0000 Hi, After upgrading a FreeBSD domU from 10.3-RELEASE to 11-RELEASE, it seems that FreeBSD can no longer act as a vbd backend for other domUs. *** My setup before upgrading: Xen 4.4 dom0: Linux (Debian) with a custom compiled 4.2 kernel FreeBSD domU: FreeBSD 10.3-RELEASE-p7 in hvm mode. Filesystem is ZFS, with zvols created for other domUs. xen-guest-tools-4.2.2_1 is installed through pkg. Other domU: Linux with the same compiled 4.2 kernel in pv mode. Xen disk configuration is 'format=raw,vdev=xvda1,backendtype=phy,backend=FREEBSD,access=w,target=/dev/zvol/.../rootfs' *** *** Now that FreeBSD 11-RELEASE is out I upgraded it in place. After upgrading FreeBSD the other Linux domU can no longer find /dev/xvda1 to mount as its root. It hangs at boot waiting for the device to appear. While the Linux domU is waiting for the vbd I ran xenstore-ls in dom0 to see what is going on. The backend appears in FreeBSD and the frontend in the Linux domU, but FreeBSD shows state=2 and Linux shows state=3. FreeBSD backend: /local/domain/6/backend/vbd = "" (n0,r6) /local/domain/6/backend/vbd/7 = "" (n0,r6) /local/domain/6/backend/vbd/7/51713 = "" (n6,r7) /local/domain/6/backend/vbd/7/51713/frontend = "/local/domain/7/device/vbd/51713" (n6,r7) /local/domain/6/backend/vbd/7/51713/params = "/dev/zvol/.../rootfs" (n6,r7) /local/domain/6/backend/vbd/7/51713/script = "/etc/xen/scripts/block" (n6,r7) /local/domain/6/backend/vbd/7/51713/frontend-id = "7" (n6,r7) /local/domain/6/backend/vbd/7/51713/online = "1" (n6,r7) /local/domain/6/backend/vbd/7/51713/removable = "0" (n6,r7) /local/domain/6/backend/vbd/7/51713/bootable = "1" (n6,r7) /local/domain/6/backend/vbd/7/51713/state = "2" (n6,r7) /local/domain/6/backend/vbd/7/51713/dev = "xvda1" (n6,r7) /local/domain/6/backend/vbd/7/51713/type = "phy" (n6,r7) /local/domain/6/backend/vbd/7/51713/mode = "w" (n6,r7) /local/domain/6/backend/vbd/7/51713/device-type = "disk" (n6,r7) /local/domain/6/backend/vbd/7/51713/feature-barrier = "1" (n6,r7) /local/domain/6/backend/vbd/7/51713/feature-flush-cache = "1" (n6,r7) /local/domain/6/backend/vbd/7/51713/max-ring-page-order = "5" (n6,r7) Linux frontend: /local/domain/7/device/vbd = "" (n0,r7) /local/domain/7/device/vbd/51713 = "" (n7,r6) /local/domain/7/device/vbd/51713/backend = "/local/domain/6/backend/vbd/7/51713" (n7,r6) /local/domain/7/device/vbd/51713/backend-id = "6" (n7,r6) /local/domain/7/device/vbd/51713/state = "3" (n7,r6) /local/domain/7/device/vbd/51713/virtual-device = "51713" (n7,r6) /local/domain/7/device/vbd/51713/device-type = "disk" (n7,r6) /local/domain/7/device/vbd/51713/protocol = "x86_64-abi" (n7,r6) /local/domain/7/device/vbd/51713/ring-ref = "8" (n7,r6) /local/domain/7/device/vbd/51713/event-channel = "15" (n7,r6) /local/domain/7/device/vbd/51713/feature-persistent = "1" (n7,r6) When the vbd was working in 10.3, the backend also shows the following: /local/domain/8/backend/vbd/9/51713/hotplug-status = "connected" (n8,r9) /local/domain/8/backend/vbd/9/51713/sectors = "41943040" (n8,r9) /local/domain/8/backend/vbd/9/51713/info = "0" (n8,r9) /local/domain/8/backend/vbd/9/51713/sector-size = "512" (n8,r9) *** In case there is a version incompatibility with the new FreeBSD, I later upgraded Xen to 4.7 and dom0 to a Linux 4.4 kernel. This did not help. How might I get this setup working again? Thanks, Nathan From owner-freebsd-xen@freebsd.org Tue Dec 6 10:05:37 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B770C69808 for ; Tue, 6 Dec 2016 10:05:37 +0000 (UTC) (envelope-from prvs=14170b091=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 012AD1F3C for ; Tue, 6 Dec 2016 10:05:36 +0000 (UTC) (envelope-from prvs=14170b091=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,309,1477958400"; d="scan'208";a="36245621" Date: Tue, 6 Dec 2016 10:04:14 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Nathan Friess CC: Subject: Re: 11-RELEASE acting as vbd backend Message-ID: <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> User-Agent: NeoMutt/20161126 (1.7.1) X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2016 10:05:37 -0000 On Mon, Dec 05, 2016 at 08:48:00PM -0700, Nathan Friess wrote: > Hi, > > After upgrading a FreeBSD domU from 10.3-RELEASE to 11-RELEASE, it seems > that FreeBSD can no longer act as a vbd backend for other domUs. > > > *** > My setup before upgrading: > > Xen 4.4 > > dom0: Linux (Debian) with a custom compiled 4.2 kernel > > FreeBSD domU: FreeBSD 10.3-RELEASE-p7 in hvm mode. > Filesystem is ZFS, with zvols created for other domUs. > xen-guest-tools-4.2.2_1 is installed through pkg. > > Other domU: Linux with the same compiled 4.2 kernel in pv mode. Xen disk > configuration is 'format=raw,vdev=xvda1,backendtype=phy,backend=FREEBSD,access=w,target=/dev/zvol/.../rootfs' > *** > > > *** > Now that FreeBSD 11-RELEASE is out I upgraded it in place. After upgrading > FreeBSD the other Linux domU can no longer find /dev/xvda1 to mount as its > root. It hangs at boot waiting for the device to appear. > > While the Linux domU is waiting for the vbd I ran xenstore-ls in dom0 to see > what is going on. The backend appears in FreeBSD and the frontend in the > Linux domU, but FreeBSD shows state=2 and Linux shows state=3. > > > FreeBSD backend: > > /local/domain/6/backend/vbd = "" (n0,r6) > /local/domain/6/backend/vbd/7 = "" (n0,r6) > /local/domain/6/backend/vbd/7/51713 = "" (n6,r7) > /local/domain/6/backend/vbd/7/51713/frontend = > "/local/domain/7/device/vbd/51713" (n6,r7) > /local/domain/6/backend/vbd/7/51713/params = "/dev/zvol/.../rootfs" > (n6,r7) > /local/domain/6/backend/vbd/7/51713/script = "/etc/xen/scripts/block" > (n6,r7) > /local/domain/6/backend/vbd/7/51713/frontend-id = "7" (n6,r7) > /local/domain/6/backend/vbd/7/51713/online = "1" (n6,r7) > /local/domain/6/backend/vbd/7/51713/removable = "0" (n6,r7) > /local/domain/6/backend/vbd/7/51713/bootable = "1" (n6,r7) > /local/domain/6/backend/vbd/7/51713/state = "2" (n6,r7) > /local/domain/6/backend/vbd/7/51713/dev = "xvda1" (n6,r7) > /local/domain/6/backend/vbd/7/51713/type = "phy" (n6,r7) > /local/domain/6/backend/vbd/7/51713/mode = "w" (n6,r7) > /local/domain/6/backend/vbd/7/51713/device-type = "disk" (n6,r7) > /local/domain/6/backend/vbd/7/51713/feature-barrier = "1" (n6,r7) > /local/domain/6/backend/vbd/7/51713/feature-flush-cache = "1" (n6,r7) > /local/domain/6/backend/vbd/7/51713/max-ring-page-order = "5" (n6,r7) > > > Linux frontend: > > /local/domain/7/device/vbd = "" (n0,r7) > /local/domain/7/device/vbd/51713 = "" (n7,r6) > /local/domain/7/device/vbd/51713/backend = > "/local/domain/6/backend/vbd/7/51713" (n7,r6) > /local/domain/7/device/vbd/51713/backend-id = "6" (n7,r6) > /local/domain/7/device/vbd/51713/state = "3" (n7,r6) > /local/domain/7/device/vbd/51713/virtual-device = "51713" (n7,r6) > /local/domain/7/device/vbd/51713/device-type = "disk" (n7,r6) > /local/domain/7/device/vbd/51713/protocol = "x86_64-abi" (n7,r6) > /local/domain/7/device/vbd/51713/ring-ref = "8" (n7,r6) > /local/domain/7/device/vbd/51713/event-channel = "15" (n7,r6) > /local/domain/7/device/vbd/51713/feature-persistent = "1" (n7,r6) > > > > When the vbd was working in 10.3, the backend also shows the following: > > /local/domain/8/backend/vbd/9/51713/hotplug-status = "connected" (n8,r9) > /local/domain/8/backend/vbd/9/51713/sectors = "41943040" (n8,r9) > /local/domain/8/backend/vbd/9/51713/info = "0" (n8,r9) > /local/domain/8/backend/vbd/9/51713/sector-size = "512" (n8,r9) > *** > > > In case there is a version incompatibility with the new FreeBSD, I later > upgraded Xen to 4.7 and dom0 to a Linux 4.4 kernel. This did not help. > > How might I get this setup working again? Hello, There have been some blkback changes in FreeBSD 11.0 in order to support using more disk types from blkback (like iSCSI). In order to achieve this, if you want to use blkback from a driver domain you will have to install the xen-tools package in the driver domain, and execute the xendriverdomain at domain startup (add xendriverdomain_enable="YES" to your rc.conf). After that, everything should work as expected. Roger. From owner-freebsd-xen@freebsd.org Wed Dec 7 05:53:25 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFFC3C6BE66 for ; Wed, 7 Dec 2016 05:53:25 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B684F11E0 for ; Wed, 7 Dec 2016 05:53:25 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x234.google.com with SMTP id c21so639053098ioj.1 for ; Tue, 06 Dec 2016 21:53:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=9ZVnPSjnC8JiRhn77tvpPMaHSDBSw878mLdpCtKltO4=; b=sfnv7ApINr+dSkJcUmWbBR0XYMU9F1K7LVgxZEaq6NQ4/td/QfJkNqQdIsjNYshATc ijBjOO1Ju7Ty5unrH0lsUfjUapOmppJ1Z3REhgO4FXfjRj4vfAvCqjIH8CDCXXD1r0q2 1/i15X39CJGPDs2CpWJL6ANSmXEFQkiDkCmhAnA5Udf7rxhM1p+o9q3n8yv7GppFRgtj XkGAcY5TFE6YxS00zeqRoET7qNFaCmX98M53qRxBCkJePBYI2ChpZwErjJyCZaZ3LUDu XTyCEhDE5dHnvnHAqKIfKEJer+BYL7Jeh4yo7FDELVfvnGYxASrfPaxFDVv/mimuqTVA Vvhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=9ZVnPSjnC8JiRhn77tvpPMaHSDBSw878mLdpCtKltO4=; b=hcHhHPbx0tsTU/+/u5HJMXOf3Bhk8UrhkQISOGalDG7tz6728W6jcD+iHxik5A4I/w cjGvT33uBeWJOv41xaoO7E8hjavAWU9XMqVMd3h3hfuqYNovW7BV8qgCV4+fKfWCUQ8M 0UMwBL32ahD171Z8JVJZKlJe3ZLFfons1IYywtm+iD9chkyJJtRagl1WOsAvKnLViMmr t7jqTDAAB197oo3wX4OAFuQfve3fytumpNTnZJ8DCgSaNlDBbZWRPet7N+mcMY7iQQY8 GPlgQxXWy0q3x0Vr3CdMxRJpGniTRWrnkPOHi8ZADorIz70Xx5N62rW6PV8xvFz4/Ia8 gejA== X-Gm-Message-State: AKaTC02YSSYRk39aLEqCP2yuie50DFN99AtQ2uCGjDWKjCzef3YJKn1cZoBgLpGmHAujaw== X-Received: by 10.107.35.147 with SMTP id j141mr7171033ioj.41.1481090004925; Tue, 06 Dec 2016 21:53:24 -0800 (PST) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [50.66.15.125]) by smtp.gmail.com with ESMTPSA id i62sm2862385itb.12.2016.12.06.21.53.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Dec 2016 21:53:24 -0800 (PST) Subject: Re: 11-RELEASE acting as vbd backend To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> Cc: freebsd-xen@freebsd.org From: Nathan Friess Message-ID: <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> Date: Tue, 6 Dec 2016 22:53:18 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 05:53:26 -0000 On 12/6/2016 3:04 AM, Roger Pau Monné wrote: > > Hello, > > There have been some blkback changes in FreeBSD 11.0 in order to support using > more disk types from blkback (like iSCSI). In order to achieve this, if you want > to use blkback from a driver domain you will have to install the xen-tools > package in the driver domain, and execute the xendriverdomain at domain startup > (add xendriverdomain_enable="YES" to your rc.conf). After that, everything > should work as expected. Hi Roger, I installed xen-tools and added xendriverdomain_enable="YES" to rc.conf. After rebooting this is what I see: Dec 6 19:14:20 host xenstored: Checking store ... Dec 6 19:14:20 host xenstored: Checking store complete. Dec 6 19:14:20 host xenstored: Checking store ... Dec 6 19:14:20 host xenstored: Checking store complete. Dec 6 19:14:21 host kernel: Failed to initialize dom0 state: No such file or directory Dec 6 19:14:22 host kernel: . Dec 6 19:14:51 host last message repeated 28 times Dec 6 19:14:52 host xenconsoled: Failed to contact xenstore (No such file or directory). Is it running? Trying to start xenstoredomain manually results in: Starting xenservices: xl devd.libxl: error: libxl.c:124:libxl_ctx_alloc: cannot connect to xenstore: No such file or directory cannot init xl context The kernel is showing a xenstore device on boot ... kernel: xenstore0: on xenpv0 I see that there is a xencommons rc script too so I also tried to add xencommons_enable="YES" and start that first. No luck. I must still be missing a step... Thanks, Nathan From owner-freebsd-xen@freebsd.org Wed Dec 7 16:22:10 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78429C6C42C for ; Wed, 7 Dec 2016 16:22:10 +0000 (UTC) (envelope-from prvs=1149a0a087=jwest@ezwind.net) Received: from ezwind.net (booby.ezwind.net [199.188.211.150]) by mx1.freebsd.org (Postfix) with ESMTP id 52C0618BC for ; Wed, 7 Dec 2016 16:22:10 +0000 (UTC) (envelope-from prvs=1149a0a087=jwest@ezwind.net) X-MDAV-Result: clean X-MDAV-Processed: ezwind.net, Wed, 07 Dec 2016 10:21:45 -0600 X-Spam-Processed: ezwind.net, Wed, 07 Dec 2016 10:21:45 -0600 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on Booby X-Spam-Level: X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,NO_RELAYS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from OwnerTHINK by ezwind.net (MDaemon PRO v16.0.4) with ESMTPA id 51-md50000015551.msg for ; Wed, 07 Dec 2016 10:21:44 -0600 X-MDRemoteIP: 68.184.8.34 X-MDHelo: OwnerTHINK X-MDArrival-Date: Wed, 07 Dec 2016 10:21:44 -0600 X-Authenticated-Sender: jwest@ezwind.net X-Return-Path: prvs=1149a0a087=jwest@ezwind.net X-Envelope-From: jwest@ezwind.net X-MDaemon-Deliver-To: freebsd-xen@freebsd.org From: "Jay West" To: Subject: 11-RELEASE and live migration Date: Wed, 7 Dec 2016 10:21:53 -0600 Message-ID: <001f01d250a6$05ca4160$115ec420$@ezwind.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AdJQpQTLkUaPmYqeQdWL9Zw7f2WwFg== X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 16:22:10 -0000 Not sure if anyone else has seen this.... Fresh install of xenserver 7 on two dell PE's, local storage only for xenserver 7 installation on each. All VM storage via iscsi to a nas (multipath, split between two stacked dell powerconnect switches that are properly configured for iscsi traffic). Freebsd11-Release iso used to spin up a few freebsd VM's, and xe-guest-utilities-6.2.0_2 added via 'pkg install'. Live migration from one host to the other works fine (doesn't matter which of the two hosts is the source and which is the destination). But after it is done migrating if you then do a live migration back to the server where it was a few minutes ago.... on the recipient host the VM reboots upon migration. It gets to the bios screen and then just hangs. So... 1) Why does it reboot, 2) Why does it then get stuck just after bios post but before OS load, and 3) Has anyone else seen this? J From owner-freebsd-xen@freebsd.org Wed Dec 7 17:07:00 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C22CC6B09D; Wed, 7 Dec 2016 17:07:00 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id A9044130A; Wed, 7 Dec 2016 17:06:58 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Wed, 07 Dec 2016 18:06:03 +0100 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 98FD77B7-D624-46FF-8C3C-D1CA48C90991.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Wed, 07 Dec 2016 18:06:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 07 Dec 2016 18:06:00 +0100 From: rainer@ultra-secure.de To: Jay West Cc: freebsd-xen@freebsd.org, owner-freebsd-xen@freebsd.org Subject: Re: 11-RELEASE and live migration In-Reply-To: <001f01d250a6$05ca4160$115ec420$@ezwind.net> References: <001f01d250a6$05ca4160$115ec420$@ezwind.net> Message-ID: X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 78, bad: 0, connections: 78, history: 78, pass:all_good, relaying X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 17:07:00 -0000 Am 2016-12-07 17:21, schrieb Jay West: > Not sure if anyone else has seen this.... > > Fresh install of xenserver 7 on two dell PE's, local storage only for > xenserver 7 installation on each. All VM storage via iscsi to a nas > (multipath, split between two stacked dell powerconnect switches that > are > properly configured for iscsi traffic). > > Freebsd11-Release iso used to spin up a few freebsd VM's, and > xe-guest-utilities-6.2.0_2 added via 'pkg install'. > > Live migration from one host to the other works fine (doesn't matter > which > of the two hosts is the source and which is the destination). But after > it > is done migrating if you then do a live migration back to the server > where > it was a few minutes ago.... on the recipient host the VM reboots upon > migration. It gets to the bios screen and then just hangs. > > So... 1) Why does it reboot, 2) Why does it then get stuck just after > bios > post but before OS load, and 3) Has anyone else seen this? We've also seen this (unexplainable reboots, I think they happen after migrations). But it never hangs. It seems to actually "reboot", as if somebody had pressed CTRL-ALT-DEL or typed "reboot". XenServer 6.5SP2 From owner-freebsd-xen@freebsd.org Wed Dec 7 17:26:19 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EE1CC6B64C; Wed, 7 Dec 2016 17:26:19 +0000 (UTC) (envelope-from prvs=1149a0a087=jwest@ezwind.net) Received: from ezwind.net (booby.ezwind.net [199.188.211.150]) by mx1.freebsd.org (Postfix) with ESMTP id F07D81ADB; Wed, 7 Dec 2016 17:26:18 +0000 (UTC) (envelope-from prvs=1149a0a087=jwest@ezwind.net) X-MDAV-Result: clean X-MDAV-Processed: ezwind.net, Wed, 07 Dec 2016 11:26:07 -0600 X-Spam-Processed: ezwind.net, Wed, 07 Dec 2016 11:26:07 -0600 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on Booby X-Spam-Level: X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,NO_RELAYS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from OwnerTHINK by ezwind.net (MDaemon PRO v16.0.4) with ESMTPA id 02-md50000015608.msg; Wed, 07 Dec 2016 11:26:06 -0600 X-MDRemoteIP: 68.184.8.34 X-MDHelo: OwnerTHINK X-MDArrival-Date: Wed, 07 Dec 2016 11:26:06 -0600 X-Authenticated-Sender: jwest@ezwind.net X-Return-Path: prvs=1149a0a087=jwest@ezwind.net X-Envelope-From: jwest@ezwind.net From: "Jay West" To: Cc: , References: <001f01d250a6$05ca4160$115ec420$@ezwind.net> In-Reply-To: Subject: RE: 11-RELEASE and live migration Date: Wed, 7 Dec 2016 11:26:14 -0600 Message-ID: <003501d250af$032dd580$09898080$@ezwind.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AQH1zUDfKZwhlQ3oullSr3GwEGJmYQG1SrqioKgYI0A= X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 17:26:19 -0000 Rainer wrote... ------ We've also seen this (unexplainable reboots, I think they happen after migrations). But it never hangs. It seems to actually "reboot", as if somebody had pressed CTRL-ALT-DEL or typed "reboot". --------- I should also point out... on the same pair of hosts, there are a few windows server 2012 r2 and windows 7 pro VM's. None of them have problems migrating back and forth. It's only the F11 ones that do. First migration goes fine, and vm can be logged in to and excercised. But then migrate it back... and it goes to the fake bios post screen (I forget... cirrus bios or something).. and then just hangs without loading OS so there is no message from the OS. Strange... I have older versions of freebsd on an older version of xenserver... and the above definitely is not a problem there. Perhaps it's something amiss with my new setup, but the windows vm's migrate fine.... J From owner-freebsd-xen@freebsd.org Thu Dec 8 09:40:56 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 838C3C6DE02; Thu, 8 Dec 2016 09:40:56 +0000 (UTC) (envelope-from Christian.Hoyer-Reuther@cac-chem.de) Received: from mail.cac-chem.de (mail.cac-chem.de [213.187.69.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.cac-chem.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43FF11DEA; Thu, 8 Dec 2016 09:40:55 +0000 (UTC) (envelope-from Christian.Hoyer-Reuther@cac-chem.de) Received: from localhost (localhost [127.0.0.1]) by mail.cac-chem.de (Postfix) with ESMTP id 310364094B; Thu, 8 Dec 2016 10:33:49 +0100 (CET) Received: from mail.cac-chem.de ([127.0.0.1]) by localhost (mail.cac-chem.de [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ELxnwvHFQJ9R; Thu, 8 Dec 2016 10:33:49 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mh01.cac.loc (Postfix) with ESMTP id 2AAB66000C; Thu, 8 Dec 2016 10:33:49 +0100 (CET) X-Virus-Scanned: This message was checked by ESET Mail Security for Linux at cac-chem.de Received: from mh01.cac.loc ([127.0.0.1]) by localhost (mh01.cac.loc [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7znWGTZFAgrr; Thu, 8 Dec 2016 10:33:43 +0100 (CET) Received: from EX1.cac.loc (ex1.cac.loc [172.30.77.30]) by mh01.cac.loc (Postfix) with ESMTP; Thu, 8 Dec 2016 10:33:43 +0100 (CET) Received: from EX1.cac.loc ([172.30.77.30]) by ex1 ([172.30.77.30]) with mapi; Thu, 8 Dec 2016 10:33:39 +0100 From: "Hoyer-Reuther, Christian" To: "freebsd-xen@freebsd.org" , "owner-freebsd-xen@freebsd.org" Date: Thu, 8 Dec 2016 10:33:37 +0100 Subject: Re: 11-RELEASE and live migration Thread-Topic: 11-RELEASE and live migration Thread-Index: AQH1zUDfKZwhlQ3oullSr3GwEGJmYQG1SrqioKgYI0CAAQqcMA== Message-ID: <41E487BC91654544B2B8F31096F2D9D4D1B060A925@ex1> References: <001f01d250a6$05ca4160$115ec420$@ezwind.net> <003501d250af$032dd580$09898080$@ezwind.net> In-Reply-To: <003501d250af$032dd580$09898080$@ezwind.net> Accept-Language: de-DE Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: de-DE Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2016 09:40:56 -0000 I did some tests and I see the problem too. XenServer 6.5 SP1 with almost all patches (3 hosts in pool), FreeBSD 11.0-R= ELEASE-p2, xe-guest-utilities-6.2.0_2 installed via pkg. First migration from host 3 to host 1 is ok. Some seconds later I start a new migration from host 1 to host 2 and when m= igration finishes (as seen in XenCenter) then the VM switches to the VGABio= s screen ("Plex86/Bochs VGABios (PCI) current-cvs 01 Sep 2016 ... cirrus-co= mpatible VGA is detected"). The VM seem to hang and does not respond. In Xe= nCenter I see that all the CPU's of the VM go up to 100 percent. Then after 17 minutes the VGABios screen disappears and I see the console, = the CPU usage as seen in XenCenter goes down. I logged in as root before I = started the first migration and root is still logged in. So it was a hang a= nd not a reboot. 20 minutes later I start a new migration from host 2 to host 3 and the prob= lem occurs again. This problem does not exist with 10.3-RELEASE on the same hosts. > -----Original Message----- > From: owner-freebsd-xen@freebsd.org [mailto:owner-freebsd-xen@freebsd.org= ] > On Behalf Of Jay West > Sent: Wednesday, December 07, 2016 6:26 PM > To: rainer@ultra-secure.de > Cc: freebsd-xen@freebsd.org; owner-freebsd-xen@freebsd.org > Subject: RE: 11-RELEASE and live migration >=20 > Rainer wrote... > ------ > We've also seen this (unexplainable reboots, I think they happen after > migrations). >=20 > But it never hangs. It seems to actually "reboot", as if somebody had > pressed CTRL-ALT-DEL or typed "reboot". > --------- >=20 > I should also point out... on the same pair of hosts, there are a few > windows server 2012 r2 and windows 7 pro VM's. None of them have problems > migrating back and forth. It's only the F11 ones that do. >=20 > First migration goes fine, and vm can be logged in to and excercised. But > then migrate it back... and it goes to the fake bios post screen (I > forget... cirrus bios or something).. and then just hangs without loading= OS > so there is no message from the OS. >=20 > Strange... I have older versions of freebsd on an older version of > xenserver... and the above definitely is not a problem there. >=20 > Perhaps it's something amiss with my new setup, but the windows vm's migr= ate > fine.... From owner-freebsd-xen@freebsd.org Fri Dec 9 14:46:45 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69646C6D0BB for ; Fri, 9 Dec 2016 14:46:45 +0000 (UTC) (envelope-from prvs=144a00162=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.ctxuk.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1D8B1587 for ; Fri, 9 Dec 2016 14:46:43 +0000 (UTC) (envelope-from prvs=144a00162=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,324,1477958400"; d="scan'208";a="36517915" Date: Fri, 9 Dec 2016 14:46:30 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Nathan Friess CC: Subject: Re: 11-RELEASE acting as vbd backend Message-ID: <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> User-Agent: NeoMutt/20161126 (1.7.1) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 14:46:45 -0000 On Tue, Dec 06, 2016 at 10:53:18PM -0700, Nathan Friess wrote: > On 12/6/2016 3:04 AM, Roger Pau Monné wrote: > > > > Hello, > > > > There have been some blkback changes in FreeBSD 11.0 in order to support using > > more disk types from blkback (like iSCSI). In order to achieve this, if you want > > to use blkback from a driver domain you will have to install the xen-tools > > package in the driver domain, and execute the xendriverdomain at domain startup > > (add xendriverdomain_enable="YES" to your rc.conf). After that, everything > > should work as expected. > > > Hi Roger, > > I installed xen-tools and added xendriverdomain_enable="YES" to rc.conf. > After rebooting this is what I see: > > Dec 6 19:14:20 host xenstored: Checking store ... > Dec 6 19:14:20 host xenstored: Checking store complete. > Dec 6 19:14:20 host xenstored: Checking store ... > Dec 6 19:14:20 host xenstored: Checking store complete. > Dec 6 19:14:21 host kernel: Failed to initialize dom0 state: No such file > or directory > Dec 6 19:14:22 host kernel: . > Dec 6 19:14:51 host last message repeated 28 times > Dec 6 19:14:52 host xenconsoled: Failed to contact xenstore (No such file > or directory). Is it running? > > > Trying to start xenstoredomain manually results in: > > Starting xenservices: xl devd.libxl: error: libxl.c:124:libxl_ctx_alloc: > cannot connect to xenstore: No such file or directory > cannot init xl context > > > The kernel is showing a xenstore device on boot > > ... kernel: xenstore0: on xenpv0 > > > I see that there is a xencommons rc script too so I also tried to add > xencommons_enable="YES" and start that first. No luck. > > I must still be missing a step... Hello, Sorry for the delay, there was a huge can of worms behind this. I have a solution, but it requires patching both Xen and the FreeBSD kernel. Please bear with me, I will send those patches once ready. Roger. From owner-freebsd-xen@freebsd.org Fri Dec 9 16:32:18 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89E83C6E718 for ; Fri, 9 Dec 2016 16:32:18 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5890A73 for ; Fri, 9 Dec 2016 16:32:18 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x232.google.com with SMTP id d9so64633684ioe.0 for ; Fri, 09 Dec 2016 08:32:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=Ju9HqHk9tOrevRGN18lG7ozYu2dPrlzxDFThP2sp59U=; b=NrUkVGPoyEbE86IrcTU/z1bsik7Pnc2YgZ1f3fJYjIaMiqOj0OFgq0fnHGpg4C3w1T QUu5kVvDIBWTSWYiqVRVn5kKyNy3eXoIhLyK91RBlQQHtEJNV5ZXo9Z8aRiXKZ/5zr6o g7N/WOvic3IcuxWqYS+siExmmWauKLxszMu3HcYq8V7sbuitbsk6Kj3PDHdRvL+WfON0 yNI8fQtEs9ysi/gAIRUzN6Wla9NEXCqie6wGmDO0y9RtpxB6j8EAiP3j0EkdRlmNUjcG RKvL46+XBOZffgiwlnrvRL4n1P6gLaSoZlrRIBBXiy1mpbtEdqDFpPDTMPH0P3lXDnF8 ihqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=Ju9HqHk9tOrevRGN18lG7ozYu2dPrlzxDFThP2sp59U=; b=leTYEBtEPFtLBQbZyWWVY7fhkI9vTKja+LAHkB9przl4QV+K/rkclwPq6LFty69w+E inCGpXea4Yb/wwAveLytnsH/iTDWODn+qw27j0d/99fQkRKEfGIaob9sHpVhxEsGob7a nWpu4wczhz1E3Kbf8In+OrH8mJQWRzg3Meaq+ENspjfAiLTiI1rVjddk314cSPKhpuOq gM0FBBRg3n4CqYNKuyzm4H8tHdPYzdBgZE0e3pD51PGx1/ChPFK+M44aqJI/k6Gegdv7 qSJC5t06SIDW1/HFv/hNXH0t7NmsszqgOqqLMfK+KBBwTwBkBIlt03dmf3aPfyUXHGW+ AD8g== X-Gm-Message-State: AKaTC03BEHoKJ01+LeXfYgcqVcQp20lo+BNSL0rUVb8xS+i5oCBEESuLnWIjip5z2RUh/w== X-Received: by 10.107.181.19 with SMTP id e19mr65917346iof.104.1481301084065; Fri, 09 Dec 2016 08:31:24 -0800 (PST) Received: from [10.1.1.61] ([96.53.4.234]) by smtp.googlemail.com with ESMTPSA id v75sm7330136ita.12.2016.12.09.08.31.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Dec 2016 08:31:23 -0800 (PST) Subject: Re: 11-RELEASE acting as vbd backend To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> Cc: freebsd-xen@freebsd.org From: Nathan Friess Message-ID: <584ADC91.8050909@gmail.com> Date: Fri, 9 Dec 2016 09:32:17 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 16:32:18 -0000 On 16-12-09 07:46 AM, Roger Pau Monné wrote: > Hello, > > Sorry for the delay, there was a huge can of worms behind this. I have a > solution, but it requires patching both Xen and the FreeBSD kernel. Please bear > with me, I will send those patches once ready. It sounds like you might know this already, but I did some more digging and here is what I can see so far... The important command is "xl devd". Running that under truss, it is trying to open /dev/xen/xenbus and that was the cause of the No such file error. It looks like the 10.3 tools used to open /dev/xen/xenstore instead. I can get around this by exporting XENSTORED_PATH="/dev/xen/xenstore". Now xl devd is stuck at: libxl: error: libxl_event.c:639:libxl__ev_xswatch_register: create watch for path /local/domain/7/backend: Invalid argument So it seems like an API issue with asking the kernel or dom0 to set a watch for new backend devices. Whenever the patches are ready I can test them on my system. Nathan