From owner-freebsd-xen@freebsd.org Sun Dec 18 14:32:17 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22F89C872BA for ; Sun, 18 Dec 2016 14:32:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE1611F3E for ; Sun, 18 Dec 2016 14:32:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBIEWGeW068949 for ; Sun, 18 Dec 2016 14:32:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-xen@FreeBSD.org Subject: [Bug 215209] Kernel panic at boot under XEN: integer divide fault while in kernel mode Date: Sun, 18 Dec 2016 14:32:16 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-xen@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2016 14:32:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215209 --- Comment #9 from commit-hook@freebsd.org --- A commit references this bug: Author: dim Date: Sun Dec 18 14:31:12 UTC 2016 New revision: 310228 URL: https://svnweb.freebsd.org/changeset/base/310228 Log: MFC r310013 (by cperciva): Check that blkfront devices have a non-zero number of sectors and a non-zero sector size. Such a device would be a virtual disk of zero bytes; clearly not useful, and not something we should try to attach. As a fortuitous side effect, checking that these values are non-zero here results in them not *becoming* zero later on the function. This odd behaviour began with r309124 (clang 3.9.0) but is challenging to debug; making any changes to this function whatsoever seems to affect the llvm optimizer behaviour enough to make the unexpected zeroing of the sector_size variable cease. PR: 215209 Security: The potential for variables to unexpectedly become zero has worrying consequences for security in general, but not so much in this particular context. MFC r310086: In xbd_connect(), use correct scanf conversion specifiers for the feature_barrier and feature_flush variables. Otherwise, adjacent variables on the stack, such as sector_size, may be overwritten, with disastrous results. Note that I did not see a good reason to revert the addition of zero checks introduced in r310013. Better safe than sorry. PR: 215209 Tested by: royger Changes: _U stable/10/ stable/10/sys/dev/xen/blkfront/blkfront.c _U stable/11/ stable/11/sys/dev/xen/blkfront/blkfront.c _U stable/9/ _U stable/9/sys/ stable/9/sys/dev/xen/blkfront/blkfront.c --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-xen@freebsd.org Sun Dec 18 14:32:19 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0E76C872C5 for ; Sun, 18 Dec 2016 14:32:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8671B1F50 for ; Sun, 18 Dec 2016 14:32:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBIEWIFb069589 for ; Sun, 18 Dec 2016 14:32:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-xen@FreeBSD.org Subject: [Bug 215209] Kernel panic at boot under XEN: integer divide fault while in kernel mode Date: Sun, 18 Dec 2016 14:32:19 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-xen@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2016 14:32:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215209 --- Comment #10 from commit-hook@freebsd.org --- A commit references this bug: Author: dim Date: Sun Dec 18 14:31:12 UTC 2016 New revision: 310228 URL: https://svnweb.freebsd.org/changeset/base/310228 Log: MFC r310013 (by cperciva): Check that blkfront devices have a non-zero number of sectors and a non-zero sector size. Such a device would be a virtual disk of zero bytes; clearly not useful, and not something we should try to attach. As a fortuitous side effect, checking that these values are non-zero here results in them not *becoming* zero later on the function. This odd behaviour began with r309124 (clang 3.9.0) but is challenging to debug; making any changes to this function whatsoever seems to affect the llvm optimizer behaviour enough to make the unexpected zeroing of the sector_size variable cease. PR: 215209 Security: The potential for variables to unexpectedly become zero has worrying consequences for security in general, but not so much in this particular context. MFC r310086: In xbd_connect(), use correct scanf conversion specifiers for the feature_barrier and feature_flush variables. Otherwise, adjacent variables on the stack, such as sector_size, may be overwritten, with disastrous results. Note that I did not see a good reason to revert the addition of zero checks introduced in r310013. Better safe than sorry. PR: 215209 Tested by: royger Changes: _U stable/10/ stable/10/sys/dev/xen/blkfront/blkfront.c _U stable/11/ stable/11/sys/dev/xen/blkfront/blkfront.c _U stable/9/ _U stable/9/sys/ stable/9/sys/dev/xen/blkfront/blkfront.c --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-xen@freebsd.org Sun Dec 18 19:36:36 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AA90C86334 for ; Sun, 18 Dec 2016 19:36:36 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D344F1333 for ; Sun, 18 Dec 2016 19:36:35 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x22d.google.com with SMTP id 136so137906185iou.3 for ; Sun, 18 Dec 2016 11:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=ZuQRuJbl1em0urs0pasYo81PwwKUdMCkzqQ5UyvdfAk=; b=twmGTkMCLppotOXVKLOaCfgFeduR/GjBTBxHOS29u7zL2LuQ9x8BuRRA+GtnlWx5Fd YCmWELeIPG61gTAOXS+Xp4+RNsxfw0e+mRddWp1KvBWl4FvHDuINUvEnC8CBHU2Xqs2u GVReowHurkyR6y0DAfArn100eb6ciPm6PpVTE7jYW+9mhJ/hYugVvNsFO62hbBB1/Szw j5ytDvM/bFObl3krdW6Pl9sQ2Vt/u3c0aHZraL74VH85wv2WS94K4qcIeUpimZjugIk9 b0S7+kvUbTJig+75AlH2Eg+Xi0WDyOv96M+VRwxEA9VoiX4gEnJFzCMSQ8SfqiC/E18I yP9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=ZuQRuJbl1em0urs0pasYo81PwwKUdMCkzqQ5UyvdfAk=; b=XcfZlz+4ITH05haIarimlMje4iGpvDPBbo2qR4ZJZlA4wztXVap4TsYrTZ5vPFrhja +HMndAv1K0HyT6rpVtSE5K3pR2JvO/ztRl7mLin765p4PcNy857s7oG6i65ipSRozYnW eoiudmLCQpQGeCpAKqutqysq9UWq7LYKkMZSBVdEbCe4W0hezoZS8n8g7gYzbV/ryBbK l2MHNruw02vcIKR7KS0XuoTGvIl9w4U4nluAn9LASfExcZTT5BUMqNch8XCJqoplgbsN 9WGrKp6hYqXgru4Cugyc/RNDbXo8aDvKWWlcJuIMiBKEgy2GiGhUc44qAmgDgHrRTeF+ +RLA== X-Gm-Message-State: AIkVDXLJBdyuqS28HzLPMpoY3buyjBCNtO53kOtjLSX38s5hmzmdnRObV4Wb7s3rc6yXlg== X-Received: by 10.107.46.227 with SMTP id u96mr13928454iou.58.1482089795115; Sun, 18 Dec 2016 11:36:35 -0800 (PST) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [50.66.15.125]) by smtp.gmail.com with ESMTPSA id j143sm1940785ita.1.2016.12.18.11.36.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Dec 2016 11:36:34 -0800 (PST) Subject: Re: 11-RELEASE acting as vbd backend To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , freebsd-xen@freebsd.org References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> <584ADC91.8050909@gmail.com> <20161212103257.tg46q2s3j5riixp5@dhcp-3-221.uk.xensource.com> <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> From: Nathan Friess Message-ID: Date: Sun, 18 Dec 2016 12:36:32 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2016 19:36:36 -0000 On 12/15/2016 10:34 AM, Roger Pau Monné wrote: > OK, I finally have this ready, you will need the following patches for FreeBSD, > which I pushed to my git repo on top of current HEAD: > > git://xenbits.xen.org/people/royger/freebsd.git add_watches_xenstore > > You can either compile a kernel from this branch or pickup the 4 top patches > and apply them to your local tree (I don't think you are going to find any > conflicts even if the tree is stable/11 or 11.0-RELEASE). > > You should also update your xen-tools package to the latest version, I've just > updated it to contain the xenstore device path fix: > > https://svnweb.freebsd.org/ports?view=revision&revision=428628 > > After this is done, it should just be a matter of launching xendriverdomain at > system boot and it should work out of the box. Let me know how this goes. Hi Roger, Thank you! With your patches I've made some progress but not quite there yet. I've been trying to trace through xl devd but I haven't been able to pinpoint the underlying issue. Here is where I'm at. I applied the 4 patches from your add_watches_xenstore branch to the 11.0-RELEASE kernel and built xen-tools from svn. The xendriverdomain rc script now starts xl devd successfully. However, when the other domU starts and the backend should be set up in FreeBSD, xl devd attempts to run /etc/xen/scripts/block, which is in /usr/local instead. My guess is that dom0 is generating that path and because dom0 is Linux the paths don't match. (I'm not specifying the script in the domU config directly.) Adding a symlink in /etc to /usr/local/etc/xen gets around this for now. Secondly, when /etc/xen/scripts/block runs, it doesn't have /usr/local/bin or sbin in its PATH, so it cannot execute xenstore-read or write. That was also easy enough to fix. So, now the block script runs but after that nothing seems to happen. This is where I started to trace the code in libxl. I think what needs to happen is that xl devd (or the block script?) needs to set local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and sector-size. The .../state variable is also still set to 3 after the block script runs. It looks like the Linux hotplug scripts don't set those variables directly either, so I'm not sure where that should be happening. (Aside from that, I did see that Linux sets .../physical-device to a major:minor and some feature- variables that FreeBSD does not set.) The xldevd.log doesn't show anything interesting and running with -vvv only shows some cleanup being done after the script has finished. Just to test, I set the above variables in xenstore and then set state to 4 manually, and that did kick the domU into trying to set up the vbd frontend but the kernel crashed in blkfront_gather_backend_features, which is the first thing that happens after reading the sectors, sector-size, and info variables. I was copying directly from what 10.3 was doing for the sectors and sector-size so they should be the same so I'm not sure what I did wrong. Sorry for the long email but hopefully that gives you some idea of where to look next. If nothing seems obvious, then if you could point me in the direction of what should be happening in xl after the block script runs I can try to trace things further. Nathan From owner-freebsd-xen@freebsd.org Mon Dec 19 12:27:40 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 588F8C86A2E for ; Mon, 19 Dec 2016 12:27:40 +0000 (UTC) (envelope-from prvs=154abc353=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.ctxuk.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF9541DC1 for ; Mon, 19 Dec 2016 12:27:38 +0000 (UTC) (envelope-from prvs=154abc353=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,373,1477958400"; d="scan'208";a="37066361" Date: Mon, 19 Dec 2016 12:27:27 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Nathan Friess CC: Subject: Re: 11-RELEASE acting as vbd backend Message-ID: <20161219122727.pd4dzo2hb2m6ld7d@MacBook-Pro-de-Roger.local> References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> <584ADC91.8050909@gmail.com> <20161212103257.tg46q2s3j5riixp5@dhcp-3-221.uk.xensource.com> <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: NeoMutt/20161126 (1.7.1) X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 12:27:40 -0000 On Sun, Dec 18, 2016 at 12:36:32PM -0700, Nathan Friess wrote: > On 12/15/2016 10:34 AM, Roger Pau Monné wrote: > > OK, I finally have this ready, you will need the following patches for FreeBSD, > > which I pushed to my git repo on top of current HEAD: > > > > git://xenbits.xen.org/people/royger/freebsd.git add_watches_xenstore > > > > You can either compile a kernel from this branch or pickup the 4 top patches > > and apply them to your local tree (I don't think you are going to find any > > conflicts even if the tree is stable/11 or 11.0-RELEASE). > > > > You should also update your xen-tools package to the latest version, I've just > > updated it to contain the xenstore device path fix: > > > > https://svnweb.freebsd.org/ports?view=revision&revision=428628 > > > > After this is done, it should just be a matter of launching xendriverdomain at > > system boot and it should work out of the box. Let me know how this goes. > > Hi Roger, > > Thank you! With your patches I've made some progress but not quite there > yet. I've been trying to trace through xl devd but I haven't been able to > pinpoint the underlying issue. Here is where I'm at. > > I applied the 4 patches from your add_watches_xenstore branch to the > 11.0-RELEASE kernel and built xen-tools from svn. > > The xendriverdomain rc script now starts xl devd successfully. However, > when the other domU starts and the backend should be set up in FreeBSD, xl > devd attempts to run /etc/xen/scripts/block, which is in /usr/local instead. > My guess is that dom0 is generating that path and because dom0 is Linux the > paths don't match. (I'm not specifying the script in the domU config > directly.) Adding a symlink in /etc to /usr/local/etc/xen gets around this > for now. That's expected, this path is set by the Dom0, and it uses the Linux default value. It might make more sense to simply don't set it when the backend is a driver domain, but in any case you can easily solve it by using the "script" disk option[0]. You should add script=/usr/local/etc/xen/scripts/block to your disk configuration line. > Secondly, when /etc/xen/scripts/block runs, it doesn't have /usr/local/bin > or sbin in its PATH, so it cannot execute xenstore-read or write. That was > also easy enough to fix. Hm, I guess I haven't seen this because I was running xl devd manually from the command line. AFAICT the patch I'm attaching at the bottom of this email should solve it. > So, now the block script runs but after that nothing seems to happen. This > is where I started to trace the code in libxl. I think what needs to happen > is that xl devd (or the block script?) needs to set > local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and > sector-size. The .../state variable is also still set to 3 after the block > script runs. Backend in state 3 means it's ready and waiting for the frontend to connect. That's quite weird because I've just tested a FreeBSD driver domain from a Linux Dom0 and it seems to work fine, other guests can properly see and attach the devices served by the FreeBSD driver domain. Do you have driver_domain=1 set in your FreeBSD xl config file? > It looks like the Linux hotplug scripts don't set those variables directly > either, so I'm not sure where that should be happening. (Aside from that, I > did see that Linux sets .../physical-device to a major:minor and some > feature- variables that FreeBSD does not set.) The xldevd.log doesn't show > anything interesting and running with -vvv only shows some cleanup being > done after the script has finished. This should be set by blkback itself IIRC. Roger. [0] https://xenbits.xen.org/docs/unstable/misc/xl-disk-configuration.txt ---8<--- >From 0e2e6b6f3892d83548dd1d5c3644972ee5fe9847 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Mon, 19 Dec 2016 12:16:33 +0000 Subject: [PATCH] init/FreeBSD: set correct PATH for xl devd MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit FreeBSD init scripts don't have /usr/local/{bin/sbin} in it's PATH, which prevents `xl devd` from working properly since hotplug scripts require the set of xenstore cli tools to be in PATH. While there also fix the usage of --pidfile, which according to the xl help doesn't use "=", and add braces around XLDEVD_PIDFILE. Signed-off-by: Roger Pau Monné --- Cc: Ian Jackson Cc: Wei Liu --- tools/hotplug/FreeBSD/rc.d/xendriverdomain.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/hotplug/FreeBSD/rc.d/xendriverdomain.in b/tools/hotplug/FreeBSD/rc.d/xendriverdomain.in index 8ece7c3..3917de2 100644 --- a/tools/hotplug/FreeBSD/rc.d/xendriverdomain.in +++ b/tools/hotplug/FreeBSD/rc.d/xendriverdomain.in @@ -29,7 +29,7 @@ xendriverdomain_startcmd() { printf "Starting xenservices: xl devd." - ${sbindir}/xl devd --pidfile=$XLDEVD_PIDFILE ${XLDEVD_ARGS} + PATH="${bindir}:${sbindir}:$PATH" ${sbindir}/xl devd --pidfile ${XLDEVD_PIDFILE} ${XLDEVD_ARGS} printf "\n" } -- 2.10.1 (Apple Git-78) From owner-freebsd-xen@freebsd.org Tue Dec 20 04:16:37 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBE16C874F2 for ; Tue, 20 Dec 2016 04:16:37 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EDDB1455 for ; Tue, 20 Dec 2016 04:16:37 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x22e.google.com with SMTP id h30so170186198iod.2 for ; Mon, 19 Dec 2016 20:16:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=oTwim4y2ZtDvd1jgUTs5MLprcKSuj0zkx3ZnTv7qwVQ=; b=BVcxjffKSWV1N/5tVX1J+KvN+bJsGE0BD0UDnT27Yp9+VOMAk6sL6JZ2/gINwwGjLi KCnC0G108NOvwWmCfmSmddq+/jVU6KHxWuONKQwT48KicheN6qcGA4IMw6v7fbLKnC4A xuh1HomSQd/oeCaaEo10tyTEXir5CC6wc8X7wQlqxlrRqjC8hMfCPxFpBjVtL6o77stL +qugf+dbONmokVdcbGy5liD3dCMt4ZBzieweI5v2rmmnziytUh4F4hrtPQm7erheMnrV O+1/AdiwWVb/WIVA6Lb9cfMSrVYcMJmb+aIpjT14kePxLPqNNd/RMz2hZ8d/Ok8rjFY8 cEpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=oTwim4y2ZtDvd1jgUTs5MLprcKSuj0zkx3ZnTv7qwVQ=; b=Wup57TMxy5zbxo8A8aZIsd/1yUxPsxnHxJCNpAbL44Zlcd8C6enffvwgM3Ks18xAh8 7+5y4Uwe1c7LhmCstxNBM6/02Afh6niCRsd+jJUliJjRXNWxSrzS4io5fCwMahlg7H2T WmvT2zXV1xZFj+NJLWbL63XuwakIYl6C6mzdZX1TnwVyZvdBZGyWcKPHwBxHjLRdJGFt MpcUFripLmZeeKlQqzr96vUpq0vhdpyfu5aCbISqRwBe2Nx+gBd5PFVYC6Pw/hcFT+WZ OwGCbf0w+cWZ42JQnBUvSfYlEobzReGpCXbxYmvKGeNGRIULq6f9enK8KhpPZEia600o IFgA== X-Gm-Message-State: AIkVDXJGVr+8JGCo0V2/qKQRtgZARmUPKTU0s3zw3Oa4a13h42dllUCW2MbUJF7z93SyUQ== X-Received: by 10.107.137.165 with SMTP id t37mr17675015ioi.233.1482207396568; Mon, 19 Dec 2016 20:16:36 -0800 (PST) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [50.66.15.125]) by smtp.gmail.com with ESMTPSA id i62sm8310070itb.12.2016.12.19.20.16.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Dec 2016 20:16:35 -0800 (PST) Subject: Re: 11-RELEASE acting as vbd backend To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> <584ADC91.8050909@gmail.com> <20161212103257.tg46q2s3j5riixp5@dhcp-3-221.uk.xensource.com> <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> <20161219122727.pd4dzo2hb2m6ld7d@MacBook-Pro-de-Roger.local> Cc: freebsd-xen@freebsd.org From: Nathan Friess Message-ID: <9d8eee5c-1e37-869a-95c2-b340881d9875@gmail.com> Date: Mon, 19 Dec 2016 21:16:32 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161219122727.pd4dzo2hb2m6ld7d@MacBook-Pro-de-Roger.local> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 04:16:37 -0000 On 12/19/2016 5:27 AM, Roger Pau Monné wrote: >> Secondly, when /etc/xen/scripts/block runs, it doesn't have /usr/local/bin >> or sbin in its PATH, so it cannot execute xenstore-read or write. That was >> also easy enough to fix. > > Hm, I guess I haven't seen this because I was running xl devd manually from the > command line. AFAICT the patch I'm attaching at the bottom of this email should > solve it. > I should have mentioned earlier that also in the rc script there is a typo in the stop function. rc_pid is set to the pid of xl devd but the next lines refer to rc_pids. Not that I ever want to stop xl devd anyway. :) >> So, now the block script runs but after that nothing seems to happen. This >> is where I started to trace the code in libxl. I think what needs to happen >> is that xl devd (or the block script?) needs to set >> local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and >> sector-size. The .../state variable is also still set to 3 after the block >> script runs. > > Backend in state 3 means it's ready and waiting for the frontend to connect. > That's quite weird because I've just tested a FreeBSD driver domain from a > Linux Dom0 and it seems to work fine, other guests can properly see and attach > the devices served by the FreeBSD driver domain. > > Do you have driver_domain=1 set in your FreeBSD xl config file? I did not have that before but I added it now. >> It looks like the Linux hotplug scripts don't set those variables directly >> either, so I'm not sure where that should be happening. (Aside from that, I >> did see that Linux sets .../physical-device to a major:minor and some >> feature- variables that FreeBSD does not set.) The xldevd.log doesn't show >> anything interesting and running with -vvv only shows some cleanup being >> done after the script has finished. > > This should be set by blkback itself IIRC. Yes, now I see the missing connection. What is happening is that blkback is setting the state to XenbusStateInitWait (2) in xbb_attach(). Once the block script runs the state is set to XenbusStateInitialised in xbb_attach_disk(). Now it waits for the frontend to update some variables. In the Linux kernel (4.4 that I'm using, and 4.9 from what I see), drivers/block/xen-blkfront.c, the blkback_changed() function waits for state XenbusStateInitWait in the backend after which it will set the variables that the backend is waiting for. It ignores state XenbusStateInitialised entirely. I think what is happening in my case is that FreeBSD is too fast and is setting the state twice before Linux notices, so the frontend isn't continuing on as it should. If I manually set the state of the backend backwards to 2 then the frontend connects up and the block device seems to work. Now both the frontend and backend transition to the connected state as they should. I don't know what the protocol should be, but it seems that the two ends are not agreeing on those states. Maybe on your systems the timing is different so Linux sees the InitWait state before FreeBSD moves to Initalised and everything works? (Aside from that, I wonder why Linux's watch on the state isn't firing once for each update?) Nathan From owner-freebsd-xen@freebsd.org Wed Dec 21 23:30:44 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8E95C8BA1D for ; Wed, 21 Dec 2016 23:30:44 +0000 (UTC) (envelope-from 0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@amazonses.com) Received: from a8-237.smtp-out.amazonses.com (a8-237.smtp-out.amazonses.com [54.240.8.237]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A05A9199E for ; Wed, 21 Dec 2016 23:30:44 +0000 (UTC) (envelope-from 0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1482363036; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=mI09f6av3ZoejDxMqN0dZu0RbEh68sownzRn0eJcyhY=; b=WIt1jIjCYcAM5pzzveEdpt2zpjkj9l2seUAMErWnL9aeDP83udanyPFvNHYBTuWz IOq/+xcKYYelQ+pMrQCs0Gua0Mp/njea1JvoGCIuiHYnaOAbYEAiAlSUKDU4UBMAfTx R4+fL06qbqLjKTbEuWMB2LOpnKQAZPGHTCzakqe4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1482363036; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=mI09f6av3ZoejDxMqN0dZu0RbEh68sownzRn0eJcyhY=; b=yVuLpxwJGoQ8nbKGPKMW5aqrU2d1onvNdChlLeTLRrrwglamFSh6z16M9fAletOy GazdLGNr1S1EXTHDP1MPN4eZ1+G1zdwAtkwJ6RXzUMDDNj8oo6xasYZq6K0hYT3/6Gr qXvlZ2g1WSTPMXNZkmW5HUMU9BWv57LoH8Xpzt44= Subject: Re: svn commit: r310177 - in head/sys: amd64/conf i386/conf To: FreeBSD Lists References: <201612162110.uBGLAbG3091898@repo.freebsd.org> Cc: John Baldwin From: Colin Percival Message-ID: <0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@email.amazonses.com> Date: Wed, 21 Dec 2016 23:30:36 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 In-Reply-To: <201612162110.uBGLAbG3091898@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2016.12.21-54.240.8.237 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2016 23:30:44 -0000 This commit breaks the Xen boot: > panic: NULL pcpu device_t > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff82233a20 > vpanic() at vpanic+0x186/frame 0xffffffff82233aa0 > kassert_panic() at kassert_panic+0x126/frame 0xffffffff82233b10 > xen_setup_cpus() at xen_setup_cpus+0x5b/frame 0xffffffff82233b50 > mi_startup() at mi_startup+0x118/frame 0xffffffff82233b70 > btext() at btext+0x2c I'm assuming this means xenpvcpu_attach isn't running early enough; does anyone have time to track this down and fix it? On 12/16/16 13:10, John Baldwin wrote: > Author: jhb > Date: Fri Dec 16 21:10:37 2016 > New Revision: 310177 > URL: https://svnweb.freebsd.org/changeset/base/310177 > > Log: > Enable EARLY_AP_STARTUP on amd64 and i386 kernels by default. > > PR: 199321, 203682 > MFC after: 2 months > Sponsored by: Netflix > > Modified: > head/sys/amd64/conf/GENERIC > head/sys/amd64/conf/MINIMAL > head/sys/i386/conf/GENERIC > > Modified: head/sys/amd64/conf/GENERIC > ============================================================================== > --- head/sys/amd64/conf/GENERIC Fri Dec 16 20:49:50 2016 (r310176) > +++ head/sys/amd64/conf/GENERIC Fri Dec 16 21:10:37 2016 (r310177) > @@ -98,6 +98,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # Make an SMP-capable kernel by default > options SMP # Symmetric MultiProcessor Kernel > options DEVICE_NUMA # I/O Device Affinity > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > Modified: head/sys/amd64/conf/MINIMAL > ============================================================================== > --- head/sys/amd64/conf/MINIMAL Fri Dec 16 20:49:50 2016 (r310176) > +++ head/sys/amd64/conf/MINIMAL Fri Dec 16 21:10:37 2016 (r310177) > @@ -94,6 +94,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # Make an SMP-capable kernel by default > options SMP # Symmetric MultiProcessor Kernel > options DEVICE_NUMA # I/O Device Affinity > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > Modified: head/sys/i386/conf/GENERIC > ============================================================================== > --- head/sys/i386/conf/GENERIC Fri Dec 16 20:49:50 2016 (r310176) > +++ head/sys/i386/conf/GENERIC Fri Dec 16 21:10:37 2016 (r310177) > @@ -96,6 +96,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # To make an SMP kernel, the next two lines are needed > options SMP # Symmetric MultiProcessor Kernel > device apic # I/O APIC > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > > -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@freebsd.org Thu Dec 22 10:14:18 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 833AEC8A776 for ; Thu, 22 Dec 2016 10:14:18 +0000 (UTC) (envelope-from prvs=1579c2c41=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.ctxuk.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C21B818A4; Thu, 22 Dec 2016 10:14:17 +0000 (UTC) (envelope-from prvs=1579c2c41=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,387,1477958400"; d="scan'208";a="37291649" Date: Thu, 22 Dec 2016 10:13:44 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Colin Percival CC: FreeBSD Lists , John Baldwin Subject: Re: svn commit: r310177 - in head/sys: amd64/conf i386/conf Message-ID: <20161222101344.swlw37j2of7iiwm5@dhcp-3-221.uk.xensource.com> References: <201612162110.uBGLAbG3091898@repo.freebsd.org> <0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@email.amazonses.com> User-Agent: NeoMutt/20161126 (1.7.1) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2016 10:14:18 -0000 On Wed, Dec 21, 2016 at 11:30:36PM +0000, Colin Percival wrote: > This commit breaks the Xen boot: > > panic: NULL pcpu device_t > > > cpuid = 0 > > > KDB: stack backtrace: > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff82233a20 > > > vpanic() at vpanic+0x186/frame 0xffffffff82233aa0 > > > kassert_panic() at kassert_panic+0x126/frame 0xffffffff82233b10 > > > xen_setup_cpus() at xen_setup_cpus+0x5b/frame 0xffffffff82233b50 > > > mi_startup() at mi_startup+0x118/frame 0xffffffff82233b70 > > > btext() at btext+0x2c > > > I'm assuming this means xenpvcpu_attach isn't running early enough; does > anyone have time to track this down and fix it? xen_intr_alloc_and_bind_ipi requires a non-NULL device_t, and with the changes to the AP startup code now xen_cpu_ipi_init gets called before the dummy pcpu devices are attached. I will try to find some time this afternoon/tomorrow in order to rework the ipi bind function so that it no longer requires a device_t (it seems kind of pointless anyway, since IPIs by definition always belong to the CPU package). Roger. From owner-freebsd-xen@freebsd.org Thu Dec 22 12:22:23 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DD93C8BAA3 for ; Thu, 22 Dec 2016 12:22:23 +0000 (UTC) (envelope-from prvs=1579c2c41=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A2031111; Thu, 22 Dec 2016 12:22:21 +0000 (UTC) (envelope-from prvs=1579c2c41=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,388,1477958400"; d="scan'208";a="37297674" Date: Thu, 22 Dec 2016 12:22:13 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Colin Percival , FreeBSD Lists , John Baldwin , Sean Bruno Subject: Re: svn commit: r310177 - in head/sys: amd64/conf i386/conf Message-ID: <20161222122213.l6gq6mbgw3kz2hun@dhcp-3-221.uk.xensource.com> References: <201612162110.uBGLAbG3091898@repo.freebsd.org> <0100015923b8e2f4-689d4cb8-61d4-4634-8165-5a2c7bf537dd-000000@email.amazonses.com> <20161222101344.swlw37j2of7iiwm5@dhcp-3-221.uk.xensource.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20161222101344.swlw37j2of7iiwm5@dhcp-3-221.uk.xensource.com> User-Agent: NeoMutt/20161126 (1.7.1) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2016 12:22:23 -0000 On Thu, Dec 22, 2016 at 10:13:44AM +0000, Roger Pau Monné wrote: > On Wed, Dec 21, 2016 at 11:30:36PM +0000, Colin Percival wrote: > > This commit breaks the Xen boot: > > > panic: NULL pcpu device_t > > > > > cpuid = 0 > > > > > KDB: stack backtrace: > > > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff82233a20 > > > > > vpanic() at vpanic+0x186/frame 0xffffffff82233aa0 > > > > > kassert_panic() at kassert_panic+0x126/frame 0xffffffff82233b10 > > > > > xen_setup_cpus() at xen_setup_cpus+0x5b/frame 0xffffffff82233b50 > > > > > mi_startup() at mi_startup+0x118/frame 0xffffffff82233b70 > > > > > btext() at btext+0x2c > > > > > > I'm assuming this means xenpvcpu_attach isn't running early enough; does > > anyone have time to track this down and fix it? > > xen_intr_alloc_and_bind_ipi requires a non-NULL device_t, and with the changes > to the AP startup code now xen_cpu_ipi_init gets called before the dummy pcpu > devices are attached. I will try to find some time this afternoon/tomorrow in > order to rework the ipi bind function so that it no longer requires a device_t > (it seems kind of pointless anyway, since IPIs by definition always belong to > the CPU package). I've been able to successfully boot FreeBSD in Dom0 mode using the following patch, I think it should also work for a PVH/HVM DomU, could someone please give it a spin? Thanks. ---8<--- diff --git a/sys/dev/xen/evtchn/evtchn_dev.c b/sys/dev/xen/evtchn/evtchn_dev.c index 5d54d6d..df81251 100644 --- a/sys/dev/xen/evtchn/evtchn_dev.c +++ b/sys/dev/xen/evtchn/evtchn_dev.c @@ -373,9 +373,9 @@ evtchn_bind_user_port(struct per_user_data *u, struct user_evtchn *evtchn) mtx_lock(&u->bind_mutex); RB_INSERT(evtchn_tree, &u->evtchns, evtchn); mtx_unlock(&u->bind_mutex); - error = xen_intr_add_handler(evtchn_dev, evtchn_filter, - evtchn_interrupt, evtchn, INTR_TYPE_MISC | INTR_MPSAFE, - evtchn->handle); + error = xen_intr_add_handler(device_get_nameunit(evtchn_dev), + evtchn_filter, evtchn_interrupt, evtchn, + INTR_TYPE_MISC | INTR_MPSAFE, evtchn->handle); if (error != 0) { xen_intr_unbind(&evtchn->handle); mtx_lock(&u->bind_mutex); diff --git a/sys/x86/xen/xen_apic.c b/sys/x86/xen/xen_apic.c index 45c3c18..a7fa255 100644 --- a/sys/x86/xen/xen_apic.c +++ b/sys/x86/xen/xen_apic.c @@ -500,12 +500,9 @@ xen_cpu_ipi_init(int cpu) { xen_intr_handle_t *ipi_handle; const struct xen_ipi_handler *ipi; - device_t dev; int idx, rc; ipi_handle = DPCPU_ID_GET(cpu, ipi_handle); - dev = pcpu_find(cpu)->pc_device; - KASSERT((dev != NULL), ("NULL pcpu device_t")); for (ipi = xen_ipis, idx = 0; idx < nitems(xen_ipis); ipi++, idx++) { @@ -514,7 +511,7 @@ xen_cpu_ipi_init(int cpu) continue; } - rc = xen_intr_alloc_and_bind_ipi(dev, cpu, ipi->filter, + rc = xen_intr_alloc_and_bind_ipi(cpu, ipi->filter, INTR_TYPE_TTY, &ipi_handle[idx]); if (rc != 0) panic("Unable to allocate a XEN IPI port"); diff --git a/sys/x86/xen/xen_intr.c b/sys/x86/xen/xen_intr.c index 9c78306..3f6d190 100644 --- a/sys/x86/xen/xen_intr.c +++ b/sys/x86/xen/xen_intr.c @@ -392,7 +392,7 @@ xen_intr_release_isrc(struct xenisrc *isrc) */ static int xen_intr_bind_isrc(struct xenisrc **isrcp, evtchn_port_t local_port, - enum evtchn_type type, device_t intr_owner, driver_filter_t filter, + enum evtchn_type type, const char *intr_owner, driver_filter_t filter, driver_intr_t handler, void *arg, enum intr_type flags, xen_intr_handle_t *port_handlep) { @@ -401,8 +401,8 @@ xen_intr_bind_isrc(struct xenisrc **isrcp, evtchn_port_t local_port, *isrcp = NULL; if (port_handlep == NULL) { - device_printf(intr_owner, - "xen_intr_bind_isrc: Bad event handle\n"); + printf("%s: xen_intr_bind_isrc: Bad event handle\n", + intr_owner); return (EINVAL); } @@ -1175,8 +1175,9 @@ xen_intr_bind_local_port(device_t dev, evtchn_port_t local_port, struct xenisrc *isrc; int error; - error = xen_intr_bind_isrc(&isrc, local_port, EVTCHN_TYPE_PORT, dev, - filter, handler, arg, flags, port_handlep); + error = xen_intr_bind_isrc(&isrc, local_port, EVTCHN_TYPE_PORT, + device_get_nameunit(dev), filter, handler, arg, flags, + port_handlep); if (error != 0) return (error); @@ -1210,8 +1211,8 @@ xen_intr_alloc_and_bind_local_port(device_t dev, u_int remote_domain, } error = xen_intr_bind_isrc(&isrc, alloc_unbound.port, EVTCHN_TYPE_PORT, - dev, filter, handler, arg, flags, - port_handlep); + device_get_nameunit(dev), filter, handler, arg, flags, + port_handlep); if (error != 0) { evtchn_close_t close = { .port = alloc_unbound.port }; if (HYPERVISOR_event_channel_op(EVTCHNOP_close, &close)) @@ -1245,8 +1246,8 @@ xen_intr_bind_remote_port(device_t dev, u_int remote_domain, } error = xen_intr_bind_isrc(&isrc, bind_interdomain.local_port, - EVTCHN_TYPE_PORT, dev, filter, handler, - arg, flags, port_handlep); + EVTCHN_TYPE_PORT, device_get_nameunit(dev), filter, handler, arg, + flags, port_handlep); if (error) { evtchn_close_t close = { .port = bind_interdomain.local_port }; if (HYPERVISOR_event_channel_op(EVTCHNOP_close, &close)) @@ -1285,8 +1286,9 @@ xen_intr_bind_virq(device_t dev, u_int virq, u_int cpu, return (-error); } - error = xen_intr_bind_isrc(&isrc, bind_virq.port, EVTCHN_TYPE_VIRQ, dev, - filter, handler, arg, flags, port_handlep); + error = xen_intr_bind_isrc(&isrc, bind_virq.port, EVTCHN_TYPE_VIRQ, + device_get_nameunit(dev), filter, handler, arg, flags, + port_handlep); #ifdef SMP if (error == 0) @@ -1325,14 +1327,15 @@ xen_intr_bind_virq(device_t dev, u_int virq, u_int cpu, } int -xen_intr_alloc_and_bind_ipi(device_t dev, u_int cpu, - driver_filter_t filter, enum intr_type flags, - xen_intr_handle_t *port_handlep) +xen_intr_alloc_and_bind_ipi(u_int cpu, driver_filter_t filter, + enum intr_type flags, xen_intr_handle_t *port_handlep) { #ifdef SMP int vcpu_id = pcpu_find(cpu)->pc_vcpu_id; struct xenisrc *isrc; struct evtchn_bind_ipi bind_ipi = { .vcpu = vcpu_id }; + /* Same size as the one used by intr_handler->ih_name. */ + char name[MAXCOMLEN + 1]; int error; /* Ensure the target CPU is ready to handle evtchn interrupts. */ @@ -1348,12 +1351,10 @@ xen_intr_alloc_and_bind_ipi(device_t dev, u_int cpu, return (-error); } - error = xen_intr_bind_isrc(&isrc, bind_ipi.port, EVTCHN_TYPE_IPI, - dev, filter, NULL, NULL, flags, - port_handlep); - if (error == 0) - error = intr_event_bind(isrc->xi_intsrc.is_event, cpu); + snprintf(name, sizeof(name), "cpu%u", cpu); + error = xen_intr_bind_isrc(&isrc, bind_ipi.port, EVTCHN_TYPE_IPI, + name, filter, NULL, NULL, flags, port_handlep); if (error != 0) { evtchn_close_t close = { .port = bind_ipi.port }; @@ -1549,7 +1550,7 @@ xen_intr_port(xen_intr_handle_t handle) } int -xen_intr_add_handler(device_t dev, driver_filter_t filter, +xen_intr_add_handler(const char *name, driver_filter_t filter, driver_intr_t handler, void *arg, enum intr_type flags, xen_intr_handle_t handle) { @@ -1560,12 +1561,12 @@ xen_intr_add_handler(device_t dev, driver_filter_t filter, if (isrc == NULL || isrc->xi_cookie != NULL) return (EINVAL); - error = intr_add_handler(device_get_nameunit(dev), isrc->xi_vector, - filter, handler, arg, flags|INTR_EXCL, &isrc->xi_cookie); + error = intr_add_handler(name, isrc->xi_vector,filter, handler, arg, + flags|INTR_EXCL, &isrc->xi_cookie); if (error != 0) { - device_printf(dev, - "xen_intr_add_handler: intr_add_handler failed: %d\n", - error); + printf( + "%s: xen_intr_add_handler: intr_add_handler failed: %d\n", + name, error); } return (error); diff --git a/sys/xen/xen_intr.h b/sys/xen/xen_intr.h index fc419d6..eb3436d 100644 --- a/sys/xen/xen_intr.h +++ b/sys/xen/xen_intr.h @@ -143,7 +143,6 @@ int xen_intr_bind_virq(device_t dev, u_int virq, u_int cpu, * interupts and, if successful, associate the port with the specified * interrupt handler. * - * \param dev The device making this bind request. * \param cpu The cpu receiving the IPI. * \param filter The interrupt filter servicing this IPI. * \param irqflags Interrupt handler flags. See sys/bus.h. @@ -152,7 +151,7 @@ int xen_intr_bind_virq(device_t dev, u_int virq, u_int cpu, * * \returns 0 on success, otherwise an errno. */ -int xen_intr_alloc_and_bind_ipi(device_t dev, u_int cpu, +int xen_intr_alloc_and_bind_ipi(u_int cpu, driver_filter_t filter, enum intr_type irqflags, xen_intr_handle_t *handlep); @@ -259,7 +258,7 @@ int xen_release_msi(int vector); * * \returns 0 on success, otherwise an errno. */ -int xen_intr_add_handler(device_t dev, driver_filter_t filter, +int xen_intr_add_handler(const char *name, driver_filter_t filter, driver_intr_t handler, void *arg, enum intr_type flags, xen_intr_handle_t handle);