From owner-freebsd-xen@freebsd.org Sun Dec 18 19:36:36 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AA90C86334 for ; Sun, 18 Dec 2016 19:36:36 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D344F1333 for ; Sun, 18 Dec 2016 19:36:35 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-io0-x22d.google.com with SMTP id 136so137906185iou.3 for ; Sun, 18 Dec 2016 11:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=ZuQRuJbl1em0urs0pasYo81PwwKUdMCkzqQ5UyvdfAk=; b=twmGTkMCLppotOXVKLOaCfgFeduR/GjBTBxHOS29u7zL2LuQ9x8BuRRA+GtnlWx5Fd YCmWELeIPG61gTAOXS+Xp4+RNsxfw0e+mRddWp1KvBWl4FvHDuINUvEnC8CBHU2Xqs2u GVReowHurkyR6y0DAfArn100eb6ciPm6PpVTE7jYW+9mhJ/hYugVvNsFO62hbBB1/Szw j5ytDvM/bFObl3krdW6Pl9sQ2Vt/u3c0aHZraL74VH85wv2WS94K4qcIeUpimZjugIk9 b0S7+kvUbTJig+75AlH2Eg+Xi0WDyOv96M+VRwxEA9VoiX4gEnJFzCMSQ8SfqiC/E18I yP9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=ZuQRuJbl1em0urs0pasYo81PwwKUdMCkzqQ5UyvdfAk=; b=XcfZlz+4ITH05haIarimlMje4iGpvDPBbo2qR4ZJZlA4wztXVap4TsYrTZ5vPFrhja +HMndAv1K0HyT6rpVtSE5K3pR2JvO/ztRl7mLin765p4PcNy857s7oG6i65ipSRozYnW eoiudmLCQpQGeCpAKqutqysq9UWq7LYKkMZSBVdEbCe4W0hezoZS8n8g7gYzbV/ryBbK l2MHNruw02vcIKR7KS0XuoTGvIl9w4U4nluAn9LASfExcZTT5BUMqNch8XCJqoplgbsN 9WGrKp6hYqXgru4Cugyc/RNDbXo8aDvKWWlcJuIMiBKEgy2GiGhUc44qAmgDgHrRTeF+ +RLA== X-Gm-Message-State: AIkVDXLJBdyuqS28HzLPMpoY3buyjBCNtO53kOtjLSX38s5hmzmdnRObV4Wb7s3rc6yXlg== X-Received: by 10.107.46.227 with SMTP id u96mr13928454iou.58.1482089795115; Sun, 18 Dec 2016 11:36:35 -0800 (PST) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [50.66.15.125]) by smtp.gmail.com with ESMTPSA id j143sm1940785ita.1.2016.12.18.11.36.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Dec 2016 11:36:34 -0800 (PST) Subject: Re: 11-RELEASE acting as vbd backend To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , freebsd-xen@freebsd.org References: <247e5b6c-2713-27cf-f8fa-61e55c9e2025@gmail.com> <20161206100414.pi7ep2zbkduhuol7@dhcp-3-221.uk.xensource.com> <39e43ae4-6388-c698-c3c2-43cbc1f7b93c@gmail.com> <20161209144630.dfga5mozh72veo4g@dhcp-3-221.uk.xensource.com> <584ADC91.8050909@gmail.com> <20161212103257.tg46q2s3j5riixp5@dhcp-3-221.uk.xensource.com> <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> From: Nathan Friess Message-ID: Date: Sun, 18 Dec 2016 12:36:32 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161215173432.3qjzilzot7lnddhm@dhcp-3-221.uk.xensource.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2016 19:36:36 -0000 On 12/15/2016 10:34 AM, Roger Pau Monné wrote: > OK, I finally have this ready, you will need the following patches for FreeBSD, > which I pushed to my git repo on top of current HEAD: > > git://xenbits.xen.org/people/royger/freebsd.git add_watches_xenstore > > You can either compile a kernel from this branch or pickup the 4 top patches > and apply them to your local tree (I don't think you are going to find any > conflicts even if the tree is stable/11 or 11.0-RELEASE). > > You should also update your xen-tools package to the latest version, I've just > updated it to contain the xenstore device path fix: > > https://svnweb.freebsd.org/ports?view=revision&revision=428628 > > After this is done, it should just be a matter of launching xendriverdomain at > system boot and it should work out of the box. Let me know how this goes. Hi Roger, Thank you! With your patches I've made some progress but not quite there yet. I've been trying to trace through xl devd but I haven't been able to pinpoint the underlying issue. Here is where I'm at. I applied the 4 patches from your add_watches_xenstore branch to the 11.0-RELEASE kernel and built xen-tools from svn. The xendriverdomain rc script now starts xl devd successfully. However, when the other domU starts and the backend should be set up in FreeBSD, xl devd attempts to run /etc/xen/scripts/block, which is in /usr/local instead. My guess is that dom0 is generating that path and because dom0 is Linux the paths don't match. (I'm not specifying the script in the domU config directly.) Adding a symlink in /etc to /usr/local/etc/xen gets around this for now. Secondly, when /etc/xen/scripts/block runs, it doesn't have /usr/local/bin or sbin in its PATH, so it cannot execute xenstore-read or write. That was also easy enough to fix. So, now the block script runs but after that nothing seems to happen. This is where I started to trace the code in libxl. I think what needs to happen is that xl devd (or the block script?) needs to set local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and sector-size. The .../state variable is also still set to 3 after the block script runs. It looks like the Linux hotplug scripts don't set those variables directly either, so I'm not sure where that should be happening. (Aside from that, I did see that Linux sets .../physical-device to a major:minor and some feature- variables that FreeBSD does not set.) The xldevd.log doesn't show anything interesting and running with -vvv only shows some cleanup being done after the script has finished. Just to test, I set the above variables in xenstore and then set state to 4 manually, and that did kick the domU into trying to set up the vbd frontend but the kernel crashed in blkfront_gather_backend_features, which is the first thing that happens after reading the sectors, sector-size, and info variables. I was copying directly from what 10.3 was doing for the sectors and sector-size so they should be the same so I'm not sure what I did wrong. Sorry for the long email but hopefully that gives you some idea of where to look next. If nothing seems obvious, then if you could point me in the direction of what should be happening in xl after the block script runs I can try to trace things further. Nathan