From owner-freebsd-xen@FreeBSD.ORG Wed Jan 15 16:24:27 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A813D9FD for ; Wed, 15 Jan 2014 16:24:27 +0000 (UTC) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 351B315DA for ; Wed, 15 Jan 2014 16:24:26 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id w62so2029596wes.13 for ; Wed, 15 Jan 2014 08:24:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=oFqgP2bYNTmnHFB2wz8dF2DyDmx5LNBdt4iOM+2CgZ0=; b=X2mVUg0C/nGNXsPW5aHKEIQGNk+iiZ6lmdlZjrvTfRkHGRWl8Fj1BDnDwxqITuhndX 3evYp/Uqc9cMfAX9dg+tn1af6PzDP5QEdzIpVAx4x/znRtNqBlIEbivKylUSeV2owxcq +zSycDJDcs11Okixsg3pvNoHAa8Nf6C9Gd96+bJ7PUPjVuUd7WH9mmFsSmStXqJYT76N pKVuU1x8CtJluLxFvrq8fkHPZX9OZ9uDNR+nkiRGMKgS+Cuqha2e4NVZjQcQzu75Lc1m 4IPoLg9DzYrq7fjyQl1S2DLQp/jIz89S0+wdDYG3fnY9Nfow+nL7lP2lworlm1DZikkm qFow== X-Gm-Message-State: ALoCoQn9wCl+d3N/wk8jUIcgoq4QbCkCU1MpBIWRB/oQFXVgcpeWhARduMzNbRLVbOIcRzSsXLJW X-Received: by 10.194.63.134 with SMTP id g6mr3273698wjs.46.1389803057418; Wed, 15 Jan 2014 08:24:17 -0800 (PST) Received: from [10.80.2.139] ([185.25.64.249]) by mx.google.com with ESMTPSA id y8sm3685838wje.12.2014.01.15.08.24.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 15 Jan 2014 08:24:16 -0800 (PST) Message-ID: <52D6B62A.9000208@linaro.org> Date: Wed, 15 Jan 2014 16:24:10 +0000 From: Julien Grall User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131104 Icedove/17.0.10 MIME-Version: 1.0 To: Warner Losh Subject: Re: [RFC] Add support for Xen ARM guest on FreeBSD References: <1389733267-20822-1-git-send-email-julien.grall@linaro.org> <24851B79-7EC7-4E3A-94DB-4B9B86FDFFFC@bsdimp.com> In-Reply-To: <24851B79-7EC7-4E3A-94DB-4B9B86FDFFFC@bsdimp.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, xen-devel@lists.xen.org, freebsd-xen@freebsd.org, freebsd-arm@FreeBSD.org, nwhitehorn@freebsd.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jan 2014 16:24:27 -0000 On 01/15/2014 01:26 AM, Warner Losh wrote: > > On Jan 14, 2014, at 2:01 PM, Julien Grall wrote: >> This new support brings 2 open questions (for both Xen and FreeBSD community). >> When a new guest is created, the toolstack will generated a device tree which >> will contains: >> - The amount of memory >> - The description of the platform (gic, timer, hypervisor node) >> - PSCI node for SMP bringup >> >> Until now, Xen on ARM supported only Linux-based OS. When the support of >> device tree was added in Xen for guest, we chose to use Linux device tree >> bindings (gic, timer,...). It seems that FreeBSD choose a different way to >> implement device tree: >> - strictly respect ePAR (for interrupt-parent property) >> - gic bindings different (only 1 interrupt cell) >> >> I would like to come with a common device tree specification (bindings, ...) >> across every operating system. I know the Linux community is working on having >> a device tree bindings out of the kernel tree. Does FreeBSD community plan to >> work with Linux community for this purpose? > > We generally try to follow the common definitions for the FDT stuff. > There are a few cases where we either lack the feature set of Linux, or > where the Linux folks are moving quickly and changing the underlying definitions > where we wait for the standards to mature before we implement. In some cases, where > maturity hasn't happened, or where the bindings are overly Linux centric (which in > theory they aren't supposed to be, but sometimes wind up that way). where we've not > implemented things. As I understand main bindings (gic, timer) are set in stone and won't change. Ian Campbell has a repository with all the ARM bindings here: http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git; a=tree;f=Bindings/arm To compare the difference between the DT provided by Xen, and the one correctly parsed by FreeBSD - Xen: http://xenbits.xen.org/people/julieng/xenvm-4.2.dts - FreeBSD: http://xenbits.xen.org/people/julieng/xenvm-bsd.dts >From Xen side: - Every device should move under a simple-bus. I think it's harmless for Linux side. - How about hypervisor node? IHMO this node should also live under the simple-bus. >From FreeBSD side: - GIC and Timer bindings needs to be handled correctly (see below the problem for the GIC) - Less stricter about interrupt-parent property. Eg, look at the grant-parent if there is no property interrupt-parent - If the hypervisor doesn't live under simple-bus, the interrupt/memory retrieving should be moved from simple-bus to nexus? Before the revision r260282 (I have added Nathan on cc), I was able to use the Linux GIC bindings (see http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git;a=blob;f=Bindings/arm/gic.txt) without any issue. It seems the fdt bus, now consider the number of interrupt cells is always 1. >> As specified early, the device tree can vary accross Xen version and user input >> (eg the memory). I have noticed few places (mainly the timer) where the IRQs >> number are harcoded. > > These cases should be viewed as bugs in FreeBSD, I believe. One of the goals > that has wide support, at leas tin theory, is that we can boot with an unaltered > Linux FDT. This goal is some time in the future. The timer interrupt used by FreeBSD doesn't match the one used by Xen (secure vs non-secure interrupt). This should be easily resolved by retrieving the interrupt from the device tree. Is there a particular reason to use the secure interrupt for the timer? > >> The second question is related to memory attribute for page table. The early >> page table setup by FreeBSD are using Write-Through memory attribute which >> result to a likely random (not every time the same place) crash before the >> real page table are initialized. >> Replacing Write-Through by Write-Back made FreeBSD boot correctly. Even today, >> I have no idea if it's a requirement from Xen or a bug (either in Xen or >> FreeBSD). > > There were some problems with pages being setup improperly for the mutexes > to operate properly. Have you confirmed this on a recent version of FreeBSD? I have checkout the freeBSD branch last week. I'm not sure to understand the relation with mutexes. The symptoms I have is mostly data/prefetch abort receive to FreeBSD. Could it be related? >> The code is taking its first faltering steps, therefore the TODO is quite big: >> - Try the code on x86 architecture. I did lots of rework for the event >> channels code and didn't even try to compile >> - Clean up event channel code >> - Clean up xen control code >> - Add support for Device Tree loading via Linux Boot ABI >> - Fixing crashing on userspace. Could be related to the patch >> series "arm SMP on Cortex-A15" >> - Add guest SMP support >> - DOM0 support? >> >> Any help, comments, questions are welcomed. > > I think this is great! We'd love to work with you to make this happen. > > Warner > >> Sincerely yours, >> >> ============= Instruction to test FreeBSD on Xen on ARM =========== > [trimmed] > -- Julien Grall