From owner-freebsd-xen@FreeBSD.ORG Mon Sep 13 11:07:08 2010 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 601A71065693 for ; Mon, 13 Sep 2010 11:07:08 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4D7718FC08 for ; Mon, 13 Sep 2010 11:07:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8DB78gO002075 for ; Mon, 13 Sep 2010 11:07:08 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8DB7707002073 for freebsd-xen@FreeBSD.org; Mon, 13 Sep 2010 11:07:07 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 13 Sep 2010 11:07:07 GMT Message-Id: <201009131107.o8DB7707002073@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 11:07:08 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/149853 xen [panic] [xen] HYPERVISOR_update_va_mapping o kern/148262 xen [xen] [panic] HYPERVISOR_update_va_mapping o kern/147248 xen [xen] 8.0-RELEASE-p3 i386 Xen DomU filesystem "bufwait o kern/145664 xen [xen] mutex vm page queue mutex not owned o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after o kern/141328 xen [xen] [panic] gstat exit causes kernel panic from unma o kern/140313 xen [xen] [panic] FreeBSD8 RC2 as PV domU crashes during c o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani 14 problems total. From owner-freebsd-xen@FreeBSD.ORG Thu Sep 16 13:59:10 2010 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F41B1065675 for ; Thu, 16 Sep 2010 13:59:10 +0000 (UTC) (envelope-from nospampam@hotmail.fr) Received: from snt0-omc4-s15.snt0.hotmail.com (snt0-omc4-s15.snt0.hotmail.com [65.55.90.218]) by mx1.freebsd.org (Postfix) with ESMTP id 1418D8FC2A for ; Thu, 16 Sep 2010 13:59:09 +0000 (UTC) Received: from SNT121-W30 ([65.55.90.200]) by snt0-omc4-s15.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 16 Sep 2010 06:47:08 -0700 Message-ID: X-Originating-IP: [82.230.32.136] From: Zladivliba Voskuy To: Date: Thu, 16 Sep 2010 15:47:09 +0200 Importance: Normal In-Reply-To: References: , , , , , , , MIME-Version: 1.0 X-OriginalArrivalTime: 16 Sep 2010 13:47:08.0789 (UTC) FILETIME=[A811BA50:01CB55A5] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: FreeBSD on EC2 maybe real now ? X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 13:59:10 -0000 Hey guys ! It seems there's a little hope to make progress with FreeBSD on EC2=2C as A= mazon posted this announcement : http://aws.typepad.com/aws/2010/07/use-you= r-own-kernel-with-amazon-ec2.html=20 I couldn't read the documentation about how to do this but they say "You co= uld (if you are sufficiently adept) use this facility to launch an operatin= g system that we don't support directly (e.g. FreeBSD). If you manage to do= this=2C please feel free to let me know". Does anyone knows how to do this actually ? Or is this really possible ?=20 I've been waiting for an FreeBSD ec2 image for years now ! I know there's a= lot of persons waiting also around so if anyone knows how to make this hap= pen=2C please post somehting !!!! Thanks ! Z. = From owner-freebsd-xen@FreeBSD.ORG Thu Sep 16 15:47:15 2010 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2801C1065672 for ; Thu, 16 Sep 2010 15:47:15 +0000 (UTC) (envelope-from gs@omegacube.fr) Received: from slow3-v.mail.gandi.net (slow3-v.mail.gandi.net [217.70.178.89]) by mx1.freebsd.org (Postfix) with ESMTP id AE3C28FC08 for ; Thu, 16 Sep 2010 15:47:14 +0000 (UTC) X-WhiteListed: mail was accepted with no delay X-WhiteListed: mail was accepted with no delay Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by slow3-v.mail.gandi.net (Postfix) with ESMTP id 4B2C638E26 for ; Thu, 16 Sep 2010 17:26:39 +0200 (CEST) X-Originating-IP: 217.70.178.41 Received: from mfilter1-d.gandi.net (mfilter1-d.gandi.net [217.70.178.41]) by relay1-d.mail.gandi.net (Postfix) with ESMTP id 3A886255306 for ; Thu, 16 Sep 2010 17:26:38 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at gandi.net Received: from relay1-d.mail.gandi.net ([217.70.183.193]) by mfilter1-d.gandi.net (mfilter1-d.gandi.net [217.70.178.41]) (amavisd-new, port 10024) with ESMTP id UITGRvBJA0VA for ; Thu, 16 Sep 2010 17:26:35 +0200 (CEST) X-Originating-IP: 209.85.215.182 Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) (Authenticated sender: gs@omegacube.fr) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id DDA992552FA for ; Thu, 16 Sep 2010 17:26:32 +0200 (CEST) Received: by eyx24 with SMTP id 24so780550eyx.13 for ; Thu, 16 Sep 2010 08:26:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.213.113.209 with SMTP id b17mr5563503ebq.39.1284650792015; Thu, 16 Sep 2010 08:26:32 -0700 (PDT) Received: by 10.220.200.65 with HTTP; Thu, 16 Sep 2010 08:26:31 -0700 (PDT) In-Reply-To: References: Date: Thu, 16 Sep 2010 17:26:31 +0200 Message-ID: From: Guillaume Seigneuret To: Zladivliba Voskuy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-xen@freebsd.org Subject: Re: FreeBSD on EC2 maybe real now ? X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 15:47:15 -0000 something :p Cordialement, *Guillaume Seigneuret* *Network and System Security Architect Web : http://www.omegacube.fr Address :* H=F4tel Technologique - BP 100 Technop=F4le de Ch=E2teau Gombert 13382 Marseille Cedex 13 2010/9/16 Zladivliba Voskuy > > Hey guys ! > > It seems there's a little hope to make progress with FreeBSD on EC2, as > Amazon posted this announcement : > http://aws.typepad.com/aws/2010/07/use-your-own-kernel-with-amazon-ec2.ht= ml > > I couldn't read the documentation about how to do this but they say "You > could (if you are sufficiently adept) use this facility to launch an > operating system that we don't support directly (e.g. FreeBSD). If you > manage to do this, please feel free to let me know". > > Does anyone knows how to do this actually ? Or is this really possible ? > > I've been waiting for an FreeBSD ec2 image for years now ! I know there's= a > lot of persons waiting also around so if anyone knows how to make this > happen, please post somehting !!!! > > Thanks ! > Z. > > > > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" > From owner-freebsd-xen@FreeBSD.ORG Thu Sep 16 23:23:24 2010 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C05310656C5 for ; Thu, 16 Sep 2010 23:23:24 +0000 (UTC) (envelope-from john.c.prather@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id F03078FC13 for ; Thu, 16 Sep 2010 23:23:23 +0000 (UTC) Received: by wwi17 with SMTP id 17so273217wwi.31 for ; Thu, 16 Sep 2010 16:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=wim5PHtfn0r2U70uB6bM+gOalNA1bqSpDXUdG0i2Lps=; b=cjFF/XNZjXCnIHv3dhzbO99xOu+BpebKewnVQtDMXpwJ4LAw8Ff3aPLEDS0oFd4P4y prWSmmf0yhqD++wQnIYjgtMXxyMWUkO0unke4thkE5xzbPZ64cOOH33RYD645996pUuo 0maD0K50sXjDinjQawtuQ/sbbP9hVL22Dxo3s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=BVlGYaEDD45DBNLaRfVdIiVpJroBR5ezkUpoXryAmaA4WB+dxwCknVYua+bmT7Xt+b IOuepVZn0Wdpst6lFno8FH6d8bqVIuBs8rx24oSB7peLzqSWrFvuqMeIHz5myZqjgInS LIigatH++gS1D4J9+kPgBN4XCXJJ67I/z8c9E= MIME-Version: 1.0 Received: by 10.216.4.19 with SMTP id 19mr3313929wei.110.1284678002094; Thu, 16 Sep 2010 16:00:02 -0700 (PDT) Received: by 10.216.56.81 with HTTP; Thu, 16 Sep 2010 16:00:02 -0700 (PDT) In-Reply-To: References: Date: Thu, 16 Sep 2010 16:00:02 -0700 Message-ID: From: John Prather To: Zladivliba Voskuy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-xen@freebsd.org Subject: Re: FreeBSD on EC2 maybe real now ? X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 23:23:24 -0000 I'm not familiar with EC2 myself yet, but I did play with FreeBSD booted out of pvgrub on linode's xen system. It was something of a pain, and required their finnix linux setup (some rescue linux system that they offer which did not consume my own paid-for disk volumes) to write the OS disk image to a volume, create the /boot fs and put the kernel into it, etc. (linode uses paravirt which resulted in unstable FreeBSD, so I never did get a Linode FreeBSD system setup that was acceptable) Depending on the ability to run one of their own provided systems on a very tiny portion of available volume space in order to put the FreeBSD pieces into place, it might be similar. I used this as my guide: http://www.linode.com/wiki/index.php/NetBSD_howto Which I had to make heavy modifications to because FreeBSD !=3D NetBSD (for instance, netbsd apparently compiles under linux) Using my experience applying that info to FreeBSD, I threw together some documentation about all the failures I had getting FreeBSD to be stable on paravirt. http://wiki.interwebfun.org/index.php?title=3DFreeBSD_XEN_Guests I imagine one of the more relevant portions of this page would be "Building XEN Kernels" and "Creating Disk Image". The former shows how I built XEN kernels on my local desktop for this. The latter details how I created the disk image for the OS volume. Apparently I did not detail how I setup the boot fs with the kernel image(s), so refer to the NetBSD_howto page for copying the kernels over and booting them. Keep in mind that these are suggestions for a starting point when trying to setup FreeBSD on a remote system with pvgrub, and almost everything will need to be considered and adjusted for amazon's EC2 environment. I had FreeBSD 7.2 with xen patches happily booting from pvgrub on Linode's xen system, but 7.3, 8.0, 9.0 were all terribly unstable, and most would not even boot in SMP. I'm under the impression that HVM is far more reliable than ParaVirt for FreeBSD, however I don't know what EC2 provides. I will probably play around with this a bit myself in the coming weeks. If i run into some success, I will post an update with more useful information. -john On Thu, Sep 16, 2010 at 6:47 AM, Zladivliba Voskuy w= rote: > > Hey guys ! > > It seems there's a little hope to make progress with FreeBSD on EC2, as A= mazon posted this announcement : http://aws.typepad.com/aws/2010/07/use-you= r-own-kernel-with-amazon-ec2.html > > I couldn't read the documentation about how to do this but they say "You = could (if you are sufficiently adept) use this facility to launch an operat= ing system that we don't support directly (e.g. FreeBSD). If you manage to = do this, please feel free to let me know". > > Does anyone knows how to do this actually ? Or is this really possible ? > > I've been waiting for an FreeBSD ec2 image for years now ! I know there's= a lot of persons waiting also around so if anyone knows how to make this h= appen, please post somehting !!!! > > Thanks ! > Z. > > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0_______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" > From owner-freebsd-xen@FreeBSD.ORG Fri Sep 17 02:23:08 2010 Return-Path: Delivered-To: xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 243C51065670 for ; Fri, 17 Sep 2010 02:23:08 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id E36B68FC0A for ; Fri, 17 Sep 2010 02:23:07 +0000 (UTC) Received: from [192.168.4.132] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id o8H1ivvZ020661 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 16 Sep 2010 19:44:57 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Message-ID: <4C92C815.7070508@scsiguy.com> Date: Thu, 16 Sep 2010 19:44:53 -0600 From: "Justin T. Gibbs" Organization: SCSIGuy.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: current@FreeBSD.org, xen@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: CFT - Xen infrastructure and block I/O improvements X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@scsiguy.com List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 02:23:08 -0000 At Spectra Logic, we are using FreeBSD amd64 under Xen to serve storage to other Xen domains. Over the past 9 months, we've made several changes to FreeBSD's Xen support. These include: o Support for backend devices (e.g. blkback) o Extensions to the Xen para-virtualized block API to allow for larger and more outstanding I/Os. o A completely rewritten block back driver with support for fronting I/O to both raw devices and files. o General cleanup and documentation of the XenBus and XenStore support code. o Robustness and performance updates for the block front driver. o Fixes to the netfront driver. Some of these changes have already been pushed back into FreeBSD, but the bulk of them need additional testing, especially under i386 PV, before they can be committed. If you work in the Xen area, I'd appreciate your review and/or testing of these changes. http://people.freebsd.org/~gibbs/FreeBSD-head-xen-diffs_2010_09_16.txt Thanks, Justin From owner-freebsd-xen@FreeBSD.ORG Fri Sep 17 06:21:11 2010 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BDF51065672 for ; Fri, 17 Sep 2010 06:21:11 +0000 (UTC) (envelope-from nospampam@hotmail.fr) Received: from snt0-omc1-s49.snt0.hotmail.com (snt0-omc1-s49.snt0.hotmail.com [65.54.61.86]) by mx1.freebsd.org (Postfix) with ESMTP id 70C708FC1F for ; Fri, 17 Sep 2010 06:21:11 +0000 (UTC) Received: from SNT121-W29 ([65.55.90.8]) by snt0-omc1-s49.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 16 Sep 2010 23:21:11 -0700 Message-ID: X-Originating-IP: [82.230.32.136] From: Zladivliba Voskuy To: Date: Fri, 17 Sep 2010 08:21:10 +0200 Importance: Normal In-Reply-To: References: , , , , , MIME-Version: 1.0 X-OriginalArrivalTime: 17 Sep 2010 06:21:11.0122 (UTC) FILETIME=[85AB8320:01CB5630] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-xen@freebsd.org Subject: RE: FreeBSD on EC2 maybe real now ? X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 06:21:11 -0000 Thanks a lot for your post. This is really usefull for all of us how are dr= eaming to put FreeBSD on EC2. I'll get it a try this weekend to see if I ca= n do something. Ok=2C let's keep us posted through this list !=20 Great ! Z. > Date: Thu=2C 16 Sep 2010 16:00:02 -0700 > Subject: Re: FreeBSD on EC2 maybe real now ? > From: john.c.prather@gmail.com > To: nospampam@hotmail.fr > CC: freebsd-xen@freebsd.org >=20 > I'm not familiar with EC2 myself yet=2C but I did play with FreeBSD > booted out of pvgrub on linode's xen system. >=20 > It was something of a pain=2C and required their finnix linux setup > (some rescue linux system that they offer which did not consume my own > paid-for disk volumes) to write the OS disk image to a volume=2C create > the /boot fs and put the kernel into it=2C etc. (linode uses paravirt > which resulted in unstable FreeBSD=2C so I never did get a Linode > FreeBSD system setup that was acceptable) >=20 > Depending on the ability to run one of their own provided systems on a > very tiny portion of available volume space in order to put the > FreeBSD pieces into place=2C it might be similar. >=20 > I used this as my guide: >=20 > http://www.linode.com/wiki/index.php/NetBSD_howto >=20 > Which I had to make heavy modifications to because FreeBSD !=3D NetBSD > (for instance=2C netbsd apparently compiles under linux) >=20 > Using my experience applying that info to FreeBSD=2C I threw together > some documentation about all the failures I had getting FreeBSD to be > stable on paravirt. >=20 > http://wiki.interwebfun.org/index.php?title=3DFreeBSD_XEN_Guests >=20 > I imagine one of the more relevant portions of this page would be > "Building XEN Kernels" and "Creating Disk Image". >=20 > The former shows how I built XEN kernels on my local desktop for this. > The latter details how I created the disk image for the OS volume. > Apparently I did not detail how I setup the boot fs with the kernel > image(s)=2C so refer to the NetBSD_howto page for copying the kernels > over and booting them. >=20 > Keep in mind that these are suggestions for a starting point when > trying to setup FreeBSD on a remote system with pvgrub=2C and almost > everything will need to be considered and adjusted for amazon's EC2 > environment. I had FreeBSD 7.2 with xen patches happily booting from > pvgrub on Linode's xen system=2C but 7.3=2C 8.0=2C 9.0 were all terribly > unstable=2C and most would not even boot in SMP. I'm under the > impression that HVM is far more reliable than ParaVirt for FreeBSD=2C > however I don't know what EC2 provides. >=20 > I will probably play around with this a bit myself in the coming > weeks. If i run into some success=2C I will post an update with more > useful information. >=20 >=20 > -john >=20 >=20 >=20 >=20 >=20 > On Thu=2C Sep 16=2C 2010 at 6:47 AM=2C Zladivliba Voskuy wrote: > > > > Hey guys ! > > > > It seems there's a little hope to make progress with FreeBSD on EC2=2C = as Amazon posted this announcement : http://aws.typepad.com/aws/2010/07/use= -your-own-kernel-with-amazon-ec2.html > > > > I couldn't read the documentation about how to do this but they say "Yo= u could (if you are sufficiently adept) use this facility to launch an oper= ating system that we don't support directly (e.g. FreeBSD). If you manage t= o do this=2C please feel free to let me know". > > > > Does anyone knows how to do this actually ? Or is this really possible = ? > > > > I've been waiting for an FreeBSD ec2 image for years now ! I know there= 's a lot of persons waiting also around so if anyone knows how to make this= happen=2C please post somehting !!!! > > > > Thanks ! > > Z. > > > > > > ______________________________= _________________ > > freebsd-xen@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > > To unsubscribe=2C send any mail to "freebsd-xen-unsubscribe@freebsd.org= " > > = From owner-freebsd-xen@FreeBSD.ORG Fri Sep 17 07:19:54 2010 Return-Path: Delivered-To: xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 364DD106564A for ; Fri, 17 Sep 2010 07:19:54 +0000 (UTC) (envelope-from rpaulo@freebsd.org) Received: from karen.lavabit.com (karen.lavabit.com [72.249.41.33]) by mx1.freebsd.org (Postfix) with ESMTP id 106CD8FC13 for ; Fri, 17 Sep 2010 07:19:53 +0000 (UTC) Received: from d.earth.lavabit.com (d.earth.lavabit.com [192.168.111.13]) by karen.lavabit.com (Postfix) with ESMTP id EFE5A34E595; Fri, 17 Sep 2010 01:55:26 -0500 (CDT) Received: from 10.0.10.3 (221.163.108.93.rev.vodafone.pt [93.108.163.221]) by lavabit.com with ESMTP id 4LUMFMT7CVIL; Fri, 17 Sep 2010 01:55:26 -0500 Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Rui Paulo In-Reply-To: <4C92C815.7070508@scsiguy.com> Date: Fri, 17 Sep 2010 07:55:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1801A881-E1C8-4B41-AD14-940686A37A27@FreeBSD.org> References: <4C92C815.7070508@scsiguy.com> To: gibbs@scsiguy.com X-Mailer: Apple Mail (2.1081) Cc: current@FreeBSD.org, xen@FreeBSD.org Subject: Re: CFT - Xen infrastructure and block I/O improvements X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 07:19:54 -0000 On 17 Sep 2010, at 02:44, Justin T. Gibbs wrote: > At Spectra Logic, we are using FreeBSD amd64 under Xen to serve = storage > to other Xen domains. Over the past 9 months, we've made several = changes > to FreeBSD's Xen support. These include: >=20 > o Support for backend devices (e.g. blkback) > o Extensions to the Xen para-virtualized block API to allow for larger > and more outstanding I/Os. > o A completely rewritten block back driver with support for fronting > I/O to both raw devices and files. > o General cleanup and documentation of the XenBus and XenStore support = code. > o Robustness and performance updates for the block front driver. > o Fixes to the netfront driver. >=20 > Some of these changes have already been pushed back into FreeBSD, but = the > bulk of them need additional testing, especially under i386 PV, before > they can be committed. If you work in the Xen area, I'd appreciate = your > review and/or testing of these changes. >=20 > http://people.freebsd.org/~gibbs/FreeBSD-head-xen-diffs_2010_09_16.txt Justin, this is quite a big diff (16k lines). I wonder if you can create = separate diffs (xenstore, blkback, xenbus, etc.) for easier review and = commenting. Regards, -- Rui Paulo From owner-freebsd-xen@FreeBSD.ORG Fri Sep 17 07:27:51 2010 Return-Path: Delivered-To: xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3002106564A; Fri, 17 Sep 2010 07:27:51 +0000 (UTC) (envelope-from rpaulo@freebsd.org) Received: from karen.lavabit.com (karen.lavabit.com [72.249.41.33]) by mx1.freebsd.org (Postfix) with ESMTP id 7BBE58FC15; Fri, 17 Sep 2010 07:27:51 +0000 (UTC) Received: from d.earth.lavabit.com (d.earth.lavabit.com [192.168.111.13]) by karen.lavabit.com (Postfix) with ESMTP id E253834E5B0; Fri, 17 Sep 2010 02:27:50 -0500 (CDT) Received: from 10.0.10.3 (221.163.108.93.rev.vodafone.pt [93.108.163.221]) by lavabit.com with ESMTP id XDNNPDE2I5V2; Fri, 17 Sep 2010 02:27:50 -0500 Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Rui Paulo In-Reply-To: <1801A881-E1C8-4B41-AD14-940686A37A27@FreeBSD.org> Date: Fri, 17 Sep 2010 08:27:47 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <57D2C852-4A1E-41AC-8074-BBBA8E0E7DCF@FreeBSD.org> References: <4C92C815.7070508@scsiguy.com> <1801A881-E1C8-4B41-AD14-940686A37A27@FreeBSD.org> To: gibbs@scsiguy.com X-Mailer: Apple Mail (2.1081) Cc: "current@freebsd.org Current" , xen@FreeBSD.org Subject: Re: CFT - Xen infrastructure and block I/O improvements X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 07:27:51 -0000 On 17 Sep 2010, at 07:55, Rui Paulo wrote: > On 17 Sep 2010, at 02:44, Justin T. Gibbs wrote: >=20 >> At Spectra Logic, we are using FreeBSD amd64 under Xen to serve = storage >> to other Xen domains. Over the past 9 months, we've made several = changes >> to FreeBSD's Xen support. These include: >>=20 >> o Support for backend devices (e.g. blkback) >> o Extensions to the Xen para-virtualized block API to allow for = larger >> and more outstanding I/Os. >> o A completely rewritten block back driver with support for fronting >> I/O to both raw devices and files. >> o General cleanup and documentation of the XenBus and XenStore = support code. >> o Robustness and performance updates for the block front driver. >> o Fixes to the netfront driver. >>=20 >> Some of these changes have already been pushed back into FreeBSD, but = the >> bulk of them need additional testing, especially under i386 PV, = before >> they can be committed. If you work in the Xen area, I'd appreciate = your >> review and/or testing of these changes. >>=20 >> = http://people.freebsd.org/~gibbs/FreeBSD-head-xen-diffs_2010_09_16.txt >=20 > Justin, this is quite a big diff (16k lines). I wonder if you can = create separate diffs (xenstore, blkback, xenbus, etc.) for easier = review and commenting. '... and comment'. Regards, -- Rui Paulo From owner-freebsd-xen@FreeBSD.ORG Fri Sep 17 15:36:29 2010 Return-Path: Delivered-To: xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E6BC1065675; Fri, 17 Sep 2010 15:36:29 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id C7A568FC0A; Fri, 17 Sep 2010 15:36:28 +0000 (UTC) Received: from [192.168.0.8] (tumnus.scsiguy.com [192.168.0.8]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id o8HFaQCA026818 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 17 Sep 2010 09:36:27 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Message-ID: <4C938AEE.2080006@scsiguy.com> Date: Fri, 17 Sep 2010 09:36:14 -0600 From: "Justin T. Gibbs" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Rui Paulo References: <4C92C815.7070508@scsiguy.com> <1801A881-E1C8-4B41-AD14-940686A37A27@FreeBSD.org> <57D2C852-4A1E-41AC-8074-BBBA8E0E7DCF@FreeBSD.org> In-Reply-To: <57D2C852-4A1E-41AC-8074-BBBA8E0E7DCF@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "current@freebsd.org Current" , xen@freebsd.org Subject: Re: CFT - Xen infrastructure and block I/O improvements X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 15:36:29 -0000 On 9/17/2010 1:27 AM, Rui Paulo wrote: > On 17 Sep 2010, at 07:55, Rui Paulo wrote: > >> On 17 Sep 2010, at 02:44, Justin T. Gibbs wrote: >> >>> At Spectra Logic, we are using FreeBSD amd64 under Xen to serve storage >>> to other Xen domains. Over the past 9 months, we've made several changes >>> to FreeBSD's Xen support. These include: ... >> Justin, this is quite a big diff (16k lines). I wonder if you can create >> separate diffs (xenstore, blkback, xenbus, etc.) for easier review and >> commenting. > > '... and comment'. The bulk of the patch is due to code reorganization. Unfortunately SVN doesn't reflect the copies properly in a diff, but at least the diff will apply properly even in a non-SVN backed source tree. This original patch should be the best for testers. The following SVN operations were made to the source tree to clean up it's organization and to import functionality from the vendor (1 file): # Does not apply to FreeBSD's NewBus method for dealing with XenBus devices svn delete sys/xen/xenbus/init.txt # Linux version of backend XenBus service routines. Never ported to FreeBSD. # OBE: See xenbusb.c, xenbusb_if.m, xenbusb_front.c xenbusb_back.c svn delete sys/xen/xenbus/xenbus_probe_backend.c # Split XenStore into its own tree. XenBus is a software layer built on top # of XenStore. The old arrangement and the naming of some structures and # functions blurred these lines making it difficult to discern what services # are provided by which layer and what times these services are available # (e.g. during system startup and shutdown). mkdir sys/xen/xenstore svn add sys/xen/xenstore svn move sys/xen/xenbus/xenbus_dev.c sys/xen/xenstore/xenstore_dev.c svn copy sys/xen/xenbus/xenbusvar.h sys/xen/xenstore/xenstorevar.h svn move sys/xen/xenbus/xenbus_xs.c sys/xen/xenstore/xenstore.c # Split up XenBus code into methods available for use by client # drivers (xenbus.c) and code used by the XenBus "bus code" to # enumerate, attach, detach, and service bus drivers. svn move sys/xen/xenbus/xenbus_client.c sys/xen/xenbus/xenbus.c svn move sys/xen/xenbus/xenbus_probe.c sys/xen/xenbus/xenbusb.c svn copy sys/xen/xenbus/xenbusb.c sys/xen/xenbus/xenbusb.h # Merged with xenstore.c svn delete sys/xen/xenbus/xenbus_comms.c svn delete sys/xen/xenbus/xenbus_comms.h # Merged with new XenBus control driver mkdir sys/dev/xen/control svn add sys/dev/xen/control svn move sys/xen/reboot.c sys/dev/xen/control/control.c # New file from Xen vendor with macros and structures used by # a block back driver to service requires from a VM running a # different ABI (e.g. amd64 back with i386 front). svn add sys/xen/blkif.h These alone account for 6k lines of svn diff. A diff against a tree with these operations already made may make more sense to a reviewer. You can download this diff from here: http://people.FreeBSD.org/~gibbs/FreeBSD-head-xen_post-svn-ops_2010-09-17_diffs.txt It isn't much shorter since the additional context has amplified the changes. The bulk is largely caused by refactoring, and comments. It will probably be easier to just review the files in sys/xen/xenstore and sys/xen/xenbus in their entirety, but the comments bellow (boiled down from our SCM system), when coupled with the diffs, should give you enough information to understand the intentions behind the changes. -- Justin sys/conf/files: Adjust kernel build spec for new XenBus/XenStore layout and added Xen functionality. sys/dev/xen/balloon/balloon.c: sys/dev/xen/netfront/netfront.c: sys/dev/xen/blkfront/blkfront.c: sys/xen/xenbus/... sys/xen/xenstore/... o Rename XenStore APIs and structures from xenbus_* to xs_*. o Adjust to use of M_XENBUS and M_XENSTORE malloc types for allocation of objects returned by these APIs. o Adjust for changes in the bus interface for Xen drivers. sys/dev/xen/blkback/blkback.c: Rewrite the Block Back driver to attach properly via newbus, operate correctly in both PV and HVM mode regardless of domain (e.g. can be in a DOM other than 0), and to deal with the latest metadata available in XenStore for block devices. Allow users to specify a file as a backend to blkback, in addition to character devices. Use the namei lookup to figure out whether it's a device node or a file, and use the appropriate interface. One current issue with the file interface is that we're effectively limited to having a single command at a time outstanding. To get around this, we may need to try using the vnode pager more directly, or perhaps coming up with a direct interface into ZFS. (i.e. something similar to zvols, but without the performance issues.) This will impact reads more than writes, because writes are cached but with random reads you have to go all the way down to the disk, so you suffer the full latency of the stack. sys/dev/xen/blkback/blkback.c: sys/xen/interface/io/blkif.h: sys/xen/blkif.h: sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: Extend the Xen blkif API: Negotiable request size and number of requests. This change extends the information recorded in the XenStore allowing block front/back devices to negotiate for optimal I/O parameters. This has been achieved without sacrificing backward compatibility with drivers that are unaware of these protocol enhancements. The extensions center around the connection protocol which now includes these additions: o The back-end device publishes its maximum supported values for, request I/O size, the number of page segments that can be associated with a request, the maximum number of requests that can be concurrently active, and the maximum number of pages that can be in the shared request ring. These values are published before the back-end enters the XenbusStateInitWait state. o The front-end waits for the back-end to enter either the InitWait or Initialize state. At this point, the front end limits it's own capabilities to the lesser of the values it finds published by the backend, it's own maximums, or, should any back-end data be missing in the store, the values supported by the original protocol. It then initializes it's internal data structures including allocation of the shared ring, publishes its maximum capabilities to the XenStore and transitions to the Initialized state. o The back-end waits for the front-end to enter the Initalized state. At this point, the back end limits it's own capabilities to the lesser of the values it finds published by the frontend, it's own maximums, or, should any front-end data be missing in the store, the values supported by the original protocol. It then initializes it's internal data structures, attaches to the shared ring and transitions to the Connected state. o The front-end waits for the back-end to enter the Connnected state, transitions itself to the connected state, and can commence I/O. Although an updated front-end driver must be aware of the back-end's InitWait state, the back-end has been coded such that it can tolerate a front-end that skips this step and transitions directly to the Initialized state without waiting for the back-end. sys/xen/interface/io/blkif.h: o Increase BLKIF_MAX_SEGMENTS_PER_REQUEST to 255. This is the maximum number possible without changing the blkif request header structure (nr_segs is a uint8_t). o Add two new constants: BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK, and BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK. These respectively indicate the number of segments that can fit in the first ring-buffer entry of a request, and for each subsequent (sg element only) ring-buffer entry associated with the "header" ring-buffer entry of the request. o Add the blkif_request_segment_t typedef for segment elements. o Add the BLKRING_GET_SG_REQUEST() macro which wraps the RING_GET_REQUEST() macro and returns a properly cast pointer to an array of blkif_request_segment_ts. o Add the BLKIF_SEGS_TO_BLOCKS() macro which calculates the number of ring entries that will be consumed by a blkif request with the given number of segments. sys/xen/blkif.h: o Update for changes in interface/io/blkif.h macros. o Update the BLKIF_MAX_RING_REQUESTS() macro to take the ring size as an argument to allow this calculation on multi-page rings. o Add a companion macro to BLKIF_MAX_RING_REQUESTS(), BLKIF_RING_PAGES(). This macro determines the number of ring pages required in order to support a ring with the supplied number of request blocks. sys/dev/xen/blkback/blkback.c: sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: o Negotiate with the other-end with the following limits: Reqeust Size: MAXPHYS Max Segments: (MAXPHYS/PAGE_SIZE) + 1 Max Requests: 256 Max Ring Pages: Sufficient to support Max Requests with Max Segments. o Dynamically allocate request pools and segemnts-per-request. o Update ring allocation/attachment code to support a multi-page shared ring. o Update routines that access the shared ring to handle multi-block requests. sys/dev/xen/blkfront/blkfront.c: o Track blkfront allocations in a blkfront driver specific malloc pool. o Strip out XenStore transaction retry logic in the connection code. Transactions only need to be used when the update to multiple XenStore nodes must be atomic. That is not the case here. o Fully disable blkif_resume() until it can be fixed properly (it didn't work before this change). o Destroy bus-dma objects during device instance tear-down. sys/dev/xen/blkback/blkback.c: Advertise support for and implement the BLKIF_OP_WRITE_BARRIER and BLKIF_OP_FLUSH_DISKCACHE blkif opcodes using BIO_FLUSH and the BIO_ORDERED attribute of bios. sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: Fix various bugs in blkfront. blkfront.c: gnttab_alloc_grant_references() returns 0 for success and non-zero for failure. The check for < 0 is a leftover Linuxism. When we negotiate with blkback and have to reduce some of our capabilities, print out the original and reduced capability before changing the local capability. So the user now gets the correct information. Fix blkif_restart_queue_callback() formatting. Make sure we hold the mutex in that function before calling xb_startio(). Fix a couple of KASSERT()s. block.h: Fix a check in the xb_remove_* macro to be a little more specific. sys/dev/xen/control/control.c: Add a XenBus front driver for handling shutdown, reboot, suspend, and resume events published in the XenStore. Move all PV suspend/reboot support from reboot.c into this driver. sys/xen/gnttab.h: sys/xen/gnttab.c: Define GNTTAB_LIST_END as GRANT_REF_INVALID. Get rid of our private definition of GNTTAB_LIST_END. sys/dev/xen/netfront/netfront.c: Use GRANT_REF_INVALID instead of driver private definitions of the same constant. sys/xen/gnttab.h: sys/xen/gnttab.c: Add the gnttab_end_foreign_access_references() API. This API allows a client to batch the release of an array of grant references, instead of coding a private for loop. The implementation takes advantage of this batching to reduce lock overhead to one acquisition and release per-batch instead of per-freed grant reference. While here, reduce the duration the gnttab_list_lock is held during gnttab_free_grant_references() operations. The search to find the tail of the incoming free list does not rely on global state and so can be performed without holding the lock. sys/dev/xen/xenpci/evtchn.c: sys/dev/xen/evtchn/evtchn.c: sys/xen/xen_intr.h: o Implement the bind_interdomain_evtchn_to_irqhandler API for HVM mode. This allows an HVM domain to serve back end devices to other domains. This API is already implemented for PV mode. o Synchronize the API between HVM and PV. sys/dev/xen/xenpci/xenpci.c: o Scan the full region of CPUID space in which the Xen VMM interface may be implemented. On systems using SuSE as a Dom0 where the Viridian API is also exported, the VMM interface is above the region we used to search. o Pass through bus_alloc_resource() calls so that XenBus drivers attaching on an HVM system can allocate unused physical address space from the nexus. The block back driver makes use of this facility. sys/i386/xen/xen_machdep.c: o Use the correct type for accessing the statically mapped xenstore metadata. sys/xen/interface/hvm/params.h: sys/xen/xenstore/xenstore.c: Move hvm_get_parameter() to the correct global header file instead of as a private method to the XenStore. sys/xen/interface/io/protocols.h: Sync with vendor. sys/xeninterface/io/ring.h: Add macro for calculating the number of ring pages needed for an N deep ring. To avoid duplication within the macros, create and use the new __RING_HEADER_SIZE() macro. This macro calculates the size of the ring book keeping struct (producer/consumer indexes, etc.) that resides at the head of the ring. Add the __RING_PAGES() macro which calculates the number of shared ring pages required to support a ring with the given number of requests. These APIs are used to support the multi-page ring version of the Xen block API. sys/xeninterface/io/xenbus.h: Add Comments. sys/xen/xenbus/... Refactor the FreeBSD XenBus support code to allow for both front and backend device attachments. Make use of new config_intr_hook capabilities to allow front and back devices to be probed/attached in parallel. Fix bugs in probe/attach state machine that could cause the system to hang when confronted with a failure either in the local domain or in a remote domain to which one of our driver instances is attaching. Publish all required state to the XenStore on device detach and failure. The majority of the missing functionality was for serving as a back end since the typical "hot-plug" scripts in Dom0 don't handle the case of cleaning up for a "service domain" that is not itself. Add doxygen style comments to the majority of the code. Cleanup types, formatting, etc. sys/xen/xenbus/xenbusb.c: Common code used by both front and back XenBus busses. sys/xen/xenbus/xenbusb_if.m: Method definitions for a XenBus bus. sys/xen/xenbus/xenbusb_front.c: sys/xen/xenbus/xenbusb_back.c: XenBus bus specialization for front and back devices. From owner-freebsd-xen@FreeBSD.ORG Sat Sep 18 08:10:15 2010 Return-Path: Delivered-To: xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7C01106566B; Sat, 18 Sep 2010 08:10:15 +0000 (UTC) (envelope-from rpaulo@freebsd.org) Received: from karen.lavabit.com (karen.lavabit.com [72.249.41.33]) by mx1.freebsd.org (Postfix) with ESMTP id 6A5A58FC0A; Sat, 18 Sep 2010 08:10:15 +0000 (UTC) Received: from b.earth.lavabit.com (b.earth.lavabit.com [192.168.111.11]) by karen.lavabit.com (Postfix) with ESMTP id A090311B886; Sat, 18 Sep 2010 03:10:14 -0500 (CDT) Received: from rui-macbook.lan (83.144.140.56) by lavabit.com with ESMTP id 0EHP40G978KT; Sat, 18 Sep 2010 03:10:14 -0500 References: <4C92C815.7070508@scsiguy.com> <1801A881-E1C8-4B41-AD14-940686A37A27@FreeBSD.org> <57D2C852-4A1E-41AC-8074-BBBA8E0E7DCF@FreeBSD.org> <4C938AEE.2080006@scsiguy.com> In-Reply-To: <4C938AEE.2080006@scsiguy.com> Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii Message-Id: <0DB67CB1-B2E3-487D-A7DB-E11D6778C62D@freebsd.org> Content-Transfer-Encoding: quoted-printable From: Rui Paulo Date: Sat, 18 Sep 2010 09:10:11 +0100 To: Justin T. Gibbs X-Mailer: Apple Mail (2.1081) Cc: "current@freebsd.org Current" , xen@freebsd.org Subject: Re: CFT - Xen infrastructure and block I/O improvements X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Sep 2010 08:10:15 -0000 On 17 Sep 2010, at 16:36, Justin T. Gibbs wrote: > On 9/17/2010 1:27 AM, Rui Paulo wrote: >> On 17 Sep 2010, at 07:55, Rui Paulo wrote: >>=20 >>> On 17 Sep 2010, at 02:44, Justin T. Gibbs wrote: >>>=20 >>>> At Spectra Logic, we are using FreeBSD amd64 under Xen to serve = storage >>>> to other Xen domains. Over the past 9 months, we've made several = changes >>>> to FreeBSD's Xen support. These include: >=20 > ... >=20 >>> Justin, this is quite a big diff (16k lines). I wonder if you can = create >>> separate diffs (xenstore, blkback, xenbus, etc.) for easier review = and >>> commenting. >>=20 >> '... and comment'. >=20 > The bulk of the patch is due to code reorganization. Unfortunately = SVN > doesn't reflect the copies properly in a diff, but at least the diff = will > apply properly even in a non-SVN backed source tree. This original = patch > should be the best for testers. >=20 > The following SVN operations were made to the source tree to clean up = it's > organization and to import functionality from the vendor (1 file): >=20 > # Does not apply to FreeBSD's NewBus method for dealing with XenBus = devices > svn delete sys/xen/xenbus/init.txt >=20 > # Linux version of backend XenBus service routines. Never ported to = FreeBSD. > # OBE: See xenbusb.c, xenbusb_if.m, xenbusb_front.c xenbusb_back.c > svn delete sys/xen/xenbus/xenbus_probe_backend.c >=20 > # Split XenStore into its own tree. XenBus is a software layer built = on top > # of XenStore. The old arrangement and the naming of some structures = and > # functions blurred these lines making it difficult to discern what = services > # are provided by which layer and what times these services are = available > # (e.g. during system startup and shutdown). > mkdir sys/xen/xenstore > svn add sys/xen/xenstore > svn move sys/xen/xenbus/xenbus_dev.c sys/xen/xenstore/xenstore_dev.c > svn copy sys/xen/xenbus/xenbusvar.h sys/xen/xenstore/xenstorevar.h > svn move sys/xen/xenbus/xenbus_xs.c sys/xen/xenstore/xenstore.c >=20 > # Split up XenBus code into methods available for use by client > # drivers (xenbus.c) and code used by the XenBus "bus code" to > # enumerate, attach, detach, and service bus drivers. > svn move sys/xen/xenbus/xenbus_client.c sys/xen/xenbus/xenbus.c > svn move sys/xen/xenbus/xenbus_probe.c sys/xen/xenbus/xenbusb.c > svn copy sys/xen/xenbus/xenbusb.c sys/xen/xenbus/xenbusb.h >=20 > # Merged with xenstore.c > svn delete sys/xen/xenbus/xenbus_comms.c > svn delete sys/xen/xenbus/xenbus_comms.h >=20 > # Merged with new XenBus control driver > mkdir sys/dev/xen/control > svn add sys/dev/xen/control > svn move sys/xen/reboot.c sys/dev/xen/control/control.c >=20 > # New file from Xen vendor with macros and structures used by > # a block back driver to service requires from a VM running a > # different ABI (e.g. amd64 back with i386 front). > svn add sys/xen/blkif.h >=20 > These alone account for 6k lines of svn diff. >=20 > A diff against a tree with these operations already made may make more = sense > to a reviewer. You can download this diff from here: >=20 > = http://people.FreeBSD.org/~gibbs/FreeBSD-head-xen_post-svn-ops_2010-09-17_= diffs.txt >=20 > It isn't much shorter since the additional context has amplified the = changes. > The bulk is largely caused by refactoring, and comments. It will > probably be easier to just review the files in sys/xen/xenstore and > sys/xen/xenbus in their entirety, but the comments bellow (boiled down = from > our SCM system), when coupled with the diffs, should give you enough = information > to understand the intentions behind the changes. Oh, IC. It's still pretty large and I don't have the necessary time, but = it appears that this is good to go (am afraid the past people who have = worked on Xen are now gone, no?). -- Rui Paulo