Date: Mon, 19 Dec 2016 21:16:32 -0700 From: Nathan Friess <nathan.friess@gmail.com> To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com> Cc: freebsd-xen@freebsd.org Subject: Re: 11-RELEASE acting as vbd backend Message-ID: <9d8eee5c-1e37-869a-95c2-b340881d9875@gmail.com> In-Reply-To: <20161219122727.pd4dzo2hb2m6ld7d@MacBook-Pro-de-Roger.local> References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> <584ADC91.8050909@gmail.com> <20161212103257.tg46q2s3j5riixp5@dhcp-3-221.uk.xensource.com> <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> <cda70ecb-6add-76ac-aee8-f9d7a1f2c015@gmail.com> <20161219122727.pd4dzo2hb2m6ld7d@MacBook-Pro-de-Roger.local>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/19/2016 5:27 AM, Roger Pau Monné wrote: >> Secondly, when /etc/xen/scripts/block runs, it doesn't have /usr/local/bin >> or sbin in its PATH, so it cannot execute xenstore-read or write. That was >> also easy enough to fix. > > Hm, I guess I haven't seen this because I was running xl devd manually from the > command line. AFAICT the patch I'm attaching at the bottom of this email should > solve it. > I should have mentioned earlier that also in the rc script there is a typo in the stop function. rc_pid is set to the pid of xl devd but the next lines refer to rc_pids. Not that I ever want to stop xl devd anyway. :) >> So, now the block script runs but after that nothing seems to happen. This >> is where I started to trace the code in libxl. I think what needs to happen >> is that xl devd (or the block script?) needs to set >> local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and >> sector-size. The .../state variable is also still set to 3 after the block >> script runs. > > Backend in state 3 means it's ready and waiting for the frontend to connect. > That's quite weird because I've just tested a FreeBSD driver domain from a > Linux Dom0 and it seems to work fine, other guests can properly see and attach > the devices served by the FreeBSD driver domain. > > Do you have driver_domain=1 set in your FreeBSD xl config file? I did not have that before but I added it now. >> It looks like the Linux hotplug scripts don't set those variables directly >> either, so I'm not sure where that should be happening. (Aside from that, I >> did see that Linux sets .../physical-device to a major:minor and some >> feature- variables that FreeBSD does not set.) The xldevd.log doesn't show >> anything interesting and running with -vvv only shows some cleanup being >> done after the script has finished. > > This should be set by blkback itself IIRC. Yes, now I see the missing connection. What is happening is that blkback is setting the state to XenbusStateInitWait (2) in xbb_attach(). Once the block script runs the state is set to XenbusStateInitialised in xbb_attach_disk(). Now it waits for the frontend to update some variables. In the Linux kernel (4.4 that I'm using, and 4.9 from what I see), drivers/block/xen-blkfront.c, the blkback_changed() function waits for state XenbusStateInitWait in the backend after which it will set the variables that the backend is waiting for. It ignores state XenbusStateInitialised entirely. I think what is happening in my case is that FreeBSD is too fast and is setting the state twice before Linux notices, so the frontend isn't continuing on as it should. If I manually set the state of the backend backwards to 2 then the frontend connects up and the block device seems to work. Now both the frontend and backend transition to the connected state as they should. I don't know what the protocol should be, but it seems that the two ends are not agreeing on those states. Maybe on your systems the timing is different so Linux sees the InitWait state before FreeBSD moves to Initalised and everything works? (Aside from that, I wonder why Linux's watch on the state isn't firing once for each update?) Nathan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9d8eee5c-1e37-869a-95c2-b340881d9875>