Date: Tue, 05 Jul 2011 20:04:20 -0600 From: "Justin T. Gibbs" <gibbs@FreeBSD.org> To: Colin Percival <cperciva@FreeBSD.org> Cc: "freebsd-xen@freebsd.org" <freebsd-xen@FreeBSD.org> Subject: Re: breakage in blkfront with ring_pages > 1 Message-ID: <4E13C2A4.5040202@FreeBSD.org> In-Reply-To: <4E13B6E9.9070202@freebsd.org> References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 7/5/11 7:14 PM, Colin Percival wrote: > On 06/10/11 13:30, Justin T. Gibbs wrote: > > On 6/9/11 9:26 PM, Colin Percival wrote: > >> Has anyone seen anything like this? Is it possible that there's a bug > >> in how our blkfront negotiates the request ring? Does anyone have > >> ring_pages > 1 in use? > > > > The only backend driver I know of that can support more than one ring page > > is FreeBSD. So, the problem likely is that blkfront is getting the > > negotiation wrong and only 1 page is in use. > > Turns out that the linux backend in question really does support multiple > pages. Can you provide a source or spec reference for this driver? There was a proposal (in 2009?) to add multiple page support to the XenSource provided blkfront/back, but it didn't get committed to the Xen repository. It also doesn't use the same XenStore variables that you list. Is this something Amazon developed in house? > There's also an inconsistency about how multiple rings are negotiated: > We set: > * ring-pages to the number of pages blkfront wants to use > * ring-ref to the reference for the first page > * ring-refXX to the references for other pages > while linux sets: > * num-ring-pages to the number of pages blkfront wants to use > * ring-refXX to the page references. You forgot to mention that the FreeBSD driver allows request size to be negotiated and explicitly supports "chained" requests to break the 44k request size limit imposed by the original interface. The 2009 proposal did not allow larger requests which is why I didn't follow it when I implemented the FreeBSD extension. Allowing larger requests both improves performance and removes the need to do costly split/coalesce operations on both sides of the blkif interface. > It seems to be impossible to be compatible with both, since Linux interprets > having a value set for ring-ref to indicate that the single-ring protocol is > being used and doesn't check anything else. It's not "Linux". There is no multi-page support in either the kernel.org hosted drivers or those in the Linux tree hosted in the official Xen repository. > Is there any official source which defines the blkback/front protocol? Only the source code in the Xen repository. Both the FreeBSD extensions that I did and those for the Linux system you are using are likely backwards compatible to the original interface, but unfortunately are not compatible with each other. > Maybe > the right option is to have a loader tunable dev.xn.linuxback to control which > version of the protocol is used? What a mess. -- Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4E13C2A4.5040202>