From owner-freebsd-xen@freebsd.org Mon May 21 19:51:17 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6A17EFCECB for ; Mon, 21 May 2018 19:51:17 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-pf0-x243.google.com (mail-pf0-x243.google.com [IPv6:2607:f8b0:400e:c00::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21E4B6DB0E; Mon, 21 May 2018 19:51:17 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-pf0-x243.google.com with SMTP id p12-v6so7565366pff.13; Mon, 21 May 2018 12:51:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=YMSr36fmh9IuqrNTeoP9YDkQOCxqk9m/DbJd7YcYRxQ=; b=mWLBUSeTBsCjx+0YRGqi0OGpsZjGi+orBktLRjCR+bb/Kb7KyUA4QKSDFmxm6v8HK9 B5CDVk5NKmkpZCIDQ+w7Q8Q5wNu8cOeohz3yS/Q/7TS63x9Bj7y3H1zHm8iEh4PQSWBR RPjMfMetx3rk6NiM2W8/zXNQfoREEW+EVj+p3e5aOKJAmhaphRKnsqGQFRF2S0X+7NW2 6FKGDc+A54He8tJQSPfv9ZpFnWAJUyW6r2J2AeFvYj+14nNhF4K41kqL8DOcQzJ9SGaa 3HxWy7ahRriNAy9jL7RDyS+hjuS5iLU/gdLhGoDXWhPYDCauZho9i2VNGBE+QQcVmoep GjFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=YMSr36fmh9IuqrNTeoP9YDkQOCxqk9m/DbJd7YcYRxQ=; b=OpUjP/FbOQmj99cRo8r8O/yDSL1kpfOHqJF5Uy8JenwqB+9mDGNsQkHJxYokrbryCh KkV+C6fdk49YmLiNlrGrat3nUqHMtddH4uTr55WGoO1ieGRTdrDJX+tL5nRKk4fUjTjR tDaZeQj/uIaYniRU/GGXFi6ITwCfSKUJRa8xi19aTbVGUnI9V1U/rRLA1sJgKDmX94I0 PYDHWI8XcGWJrHaEePxEc0QUsx7MYcx4G7QTnLQ+eSYIpYY7dP5mEGywO1UFj4eyKuqJ WTRotv4SPd3qwwA7SX/WWRipbGjYgP+B4V6gzW1TBgpGbH2MSpeL64hqKe11hb0YDRpb 6PGQ== X-Gm-Message-State: ALKqPwcH0hEyArcjetE1gUeiVBBl01wujGsEdjH/AkBhfU2DcdNkPLnR C/+G0egT+5l39BdGqyjrlP+smSmJ X-Google-Smtp-Source: AB8JxZrMRORMag9rwKiFgQzpi080m+tYruYCbgJXMa6caZzo0du9EqFZKOgdbuRK+nRe5GUfaMw7EA== X-Received: by 2002:a62:66dd:: with SMTP id s90-v6mr21041099pfj.123.1526932275857; Mon, 21 May 2018 12:51:15 -0700 (PDT) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [70.72.182.108]) by smtp.gmail.com with ESMTPSA id k193-v6sm18692950pgc.39.2018.05.21.12.51.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 May 2018 12:51:15 -0700 (PDT) Subject: Re: Linux domU only works with xen_platform_pci=0 ? To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Cc: freebsd-xen@freebsd.org References: <20180513151649.4ls73myegkhm3cep@MacBook-Pro-de-Roger.local> <0749df4b-1614-dcdf-1bf2-1bbad1ae5743@duckster.net> <20180514130445.ahqk5ol3kdhriqju@MacBook-Pro-de-Roger.local> <6c0e1f5a-3e7d-054e-298c-5ec3d97e6141@gmail.com> <20180515080836.kufr3q3mk5mxwx4q@MacBook-Pro-de-Roger.local> <20180519080250.67fl66gqbew7xfzp@MacBook-Pro-de-Roger.local> From: Nathan Friess Message-ID: <723f10d0-62c0-e48e-e8f4-b137378f0a25@gmail.com> Date: Mon, 21 May 2018 13:51:13 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180519080250.67fl66gqbew7xfzp@MacBook-Pro-de-Roger.local> Content-Type: multipart/mixed; boundary="------------8503B0B82DD575571FB6E3F0" Content-Language: en-US X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2018 19:51:18 -0000 This is a multi-part message in MIME format. --------------8503B0B82DD575571FB6E3F0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 2018-05-19 02:02 AM, Roger Pau Monné wrote: > On Thu, May 17, 2018 at 06:10:15PM -0600, Nathan Friess wrote: >> I tried the patch on and I my Linux domU did was not able to complete the >> attachment to the FreeBSD backend. I applied the patch to the 11-RELEASE >> kernel. (I couldn't get 11-STABLE to compile.) >> >> With xen_platform_pci=0 the frontend and backend vbds were both stuck in >> state 1. With xen_platform_pci=1 the frontend was in state 1 and backend in >> state 3. > > Thanks for the testing! I however think you had some issue applying > the patch or building/installing the kernel, with the attached patch > the backend should never go into state 3. > > Can you confirm that you have the patch correctly applied to the > kernel you are running? > > And if so, can you try to debug why the backend goes into state 3? > > FWIW, the patch works fine for me and I've been able to boot Debian > and Alpine Linux guests without having to add xen_platform_pci=0. > > Thanks, Roger. > Sorry, I don't which run I was looking at there. Here is the order of events. There are two zvols exposed to the domU, /dev/zvol/zroot/vmachines/smyth/rootfs and .../swap, so the messages are interspersed between the two. xbb(xbb_attach:3738): Attaching to backend/vbd/10/51713 xbb(xbb_attach:3802): Attach done, set state InitWait xbb(xbb_attach_disk:3612): Enter xbb_attach_disk xbb(xbb_attach_disk:3620): Read physical-device-path failed xbb(xbb_frontend_changed:3918): frontend_state=Initialising, xbb_state=InitWait xbb(xbb_attach:3738): Attaching to backend/vbd/10/51714 xbb(xbb_attach:3802): Attach done, set state InitWait xbb(xbb_attach_disk:3612): Enter xbb_attach_disk xbb(xbb_attach_disk:3620): Read physical-device-path failed xbb(xbb_frontend_changed:3918): frontend_state=Initialising, xbb_state=InitWait (Shell script) Start block script /local/domain/9/backend/vbd/10/51714 add (Shell script) Start block script /local/domain/9/backend/vbd/10/51713 add [1] xbb(xbb_frontend_changed:3918): frontend_state=Initialised, xbb_state=InitWait xbb(xbb_connect:3316): Enter xbb_connect: hotplug=0, get_state=2, collect_frontend=0 [1] xbb(xbb_frontend_changed:3918): frontend_state=Initialised, xbb_state=InitWait xbb(xbb_connect:3316): Enter xbb_connect: hotplug=0, get_state=2, collect_frontend=0 (Shell script) Write /dev/zvol/zroot/vmachines/smyth/rootfs to /local/domain/9/backend/vbd/10/51713/physical-device-path xbb(xbb_attach_disk:3612): Enter xbb_attach_disk xbb(xbb_open_backend:2687): opening dev=/dev/zvol/zroot/vmachines/smyth/rootfs xbb(xbb_open_backend:2757): opened dev=/dev/zvol/zroot/vmachines/smyth/rootfs sector_size=512 media_size=21474836480 xbb(xbb_attach_disk:3718): Set hotplug done, attach_disk done (Shell script) Write /dev/zvol/zroot/vmachines/smyth/swap to /local/domain/9/backend/vbd/10/51714/physical-device-path xbb(xbb_attach_disk:3612): Enter xbb_attach_disk xbb(xbb_open_backend:2687): opening dev=/dev/zvol/zroot/vmachines/smyth/swap xbb(xbb_open_backend:2757): opened dev=/dev/zvol/zroot/vmachines/smyth/swap sector_size=512 media_size=2147483648 xbb(xbb_attach_disk:3718): Set hotplug done, attach_disk done ... is stuck here... At [1] the frontend has transitioned to Initialised but the block script has not completed yet so hotplug_done is 0 and the notification about the frontend status change is lost. Calling xbb_connect at the end of xbb_attach_disk like in the attached patch will catch this case. Now with your recent change and the attached file I can use Linux domUs successfully! On a related note, did these patches ever make it into 11-stable? I don't see them at svn.freebsd.org/base/stable/11 but I may have missed something. https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002918.html There were 4 patches, the third one being required for xl devd to trigger block scripts. I have been using them as well for over a year with no issues: https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002918.html Thanks, Nathan --------------8503B0B82DD575571FB6E3F0 Content-Type: text/x-patch; name="missed-watch2.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="missed-watch2.patch" --- blkback.c.orig2 2018-05-21 06:37:22.421962000 -0600 +++ blkback.c 2018-05-21 06:38:27.571800000 -0600 @@ -3694,6 +3694,12 @@ } xbb->hotplug_done = true; + + /* The front end may have finished first, so we should proceed as well. */ + if (xenbus_get_otherend_state(xbb->dev) == XenbusStateInitialised) { + xbb_connect(xbb); + } + } /** --------------8503B0B82DD575571FB6E3F0--