From owner-freebsd-xen@FreeBSD.ORG Mon Jul 4 11:07:17 2011 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A739106566C for ; Mon, 4 Jul 2011 11:07:17 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CC5168FC0A for ; Mon, 4 Jul 2011 11:07:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p64B7Gaf040640 for ; Mon, 4 Jul 2011 11:07:16 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p64B7Gu9040636 for freebsd-xen@FreeBSD.org; Mon, 4 Jul 2011 11:07:16 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 4 Jul 2011 11:07:16 GMT Message-Id: <201107041107.p64B7Gu9040636@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2011 11:07:17 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/155468 xen [xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen [xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen [xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen [xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen [xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen [xen] xn0 network interface and PF - Massive performan o kern/154302 xen [xen] [panic] [patch] xn0: Error 2 parsing device/vif/ o kern/153789 xen [xen] [regression] FreeBSD 8.2-RC1 crashes under Solar o kern/153674 xen [xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen [xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen [xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen [xen] XEN pmap code abuses vm page queue lock o kern/153150 xen [xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen [xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xen ufs filesystem corruption on XEN DomU system f kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen [xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen [xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 24 problems total. From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 01:14:23 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 613231065676 for ; Wed, 6 Jul 2011 01:14:23 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id 23AE2151FED for ; Wed, 6 Jul 2011 01:14:18 +0000 (UTC) Received: (qmail 3879 invoked from network); 6 Jul 2011 01:14:17 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 6 Jul 2011 01:14:17 -0000 Message-ID: <4E13B6E9.9070202@freebsd.org> Date: Tue, 05 Jul 2011 18:14:17 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: gibbs@freebsd.org References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> In-Reply-To: <4DF27EFF.5060005@FreeBSD.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-xen@freebsd.org" Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 01:14:23 -0000 On 06/10/11 13:30, Justin T. Gibbs wrote: > On 6/9/11 9:26 PM, Colin Percival wrote: >> Has anyone seen anything like this? Is it possible that there's a bug >> in how our blkfront negotiates the request ring? Does anyone have >> ring_pages > 1 in use? > > The only backend driver I know of that can support more than one ring page > is FreeBSD. So, the problem likely is that blkfront is getting the > negotiation wrong and only 1 page is in use. Turns out that the linux backend in question really does support multiple pages. There's also an inconsistency about how multiple rings are negotiated: We set: * ring-pages to the number of pages blkfront wants to use * ring-ref to the reference for the first page * ring-refXX to the references for other pages while linux sets: * num-ring-pages to the number of pages blkfront wants to use * ring-refXX to the page references. It seems to be impossible to be compatible with both, since Linux interprets having a value set for ring-ref to indicate that the single-ring protocol is being used and doesn't check anything else. Is there any official source which defines the blkback/front protocol? Maybe the right option is to have a loader tunable dev.xn.linuxback to control which version of the protocol is used? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 02:04:21 2011 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9F3F1065670; Wed, 6 Jul 2011 02:04:21 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 8A25E8FC08; Wed, 6 Jul 2011 02:04:21 +0000 (UTC) Received: from macbook.scsiguy.com (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p6625iDr073073 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 5 Jul 2011 20:05:44 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) Message-ID: <4E13C2A4.5040202@FreeBSD.org> Date: Tue, 05 Jul 2011 20:04:20 -0600 From: "Justin T. Gibbs" Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 MIME-Version: 1.0 To: Colin Percival References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org> In-Reply-To: <4E13B6E9.9070202@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Tue, 05 Jul 2011 20:05:44 -0600 (MDT) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-xen@freebsd.org" Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@FreeBSD.org List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 02:04:21 -0000 On 7/5/11 7:14 PM, Colin Percival wrote: > On 06/10/11 13:30, Justin T. Gibbs wrote: > > On 6/9/11 9:26 PM, Colin Percival wrote: > >> Has anyone seen anything like this? Is it possible that there's a bug > >> in how our blkfront negotiates the request ring? Does anyone have > >> ring_pages > 1 in use? > > > > The only backend driver I know of that can support more than one ring page > > is FreeBSD. So, the problem likely is that blkfront is getting the > > negotiation wrong and only 1 page is in use. > > Turns out that the linux backend in question really does support multiple > pages. Can you provide a source or spec reference for this driver? There was a proposal (in 2009?) to add multiple page support to the XenSource provided blkfront/back, but it didn't get committed to the Xen repository. It also doesn't use the same XenStore variables that you list. Is this something Amazon developed in house? > There's also an inconsistency about how multiple rings are negotiated: > We set: > * ring-pages to the number of pages blkfront wants to use > * ring-ref to the reference for the first page > * ring-refXX to the references for other pages > while linux sets: > * num-ring-pages to the number of pages blkfront wants to use > * ring-refXX to the page references. You forgot to mention that the FreeBSD driver allows request size to be negotiated and explicitly supports "chained" requests to break the 44k request size limit imposed by the original interface. The 2009 proposal did not allow larger requests which is why I didn't follow it when I implemented the FreeBSD extension. Allowing larger requests both improves performance and removes the need to do costly split/coalesce operations on both sides of the blkif interface. > It seems to be impossible to be compatible with both, since Linux interprets > having a value set for ring-ref to indicate that the single-ring protocol is > being used and doesn't check anything else. It's not "Linux". There is no multi-page support in either the kernel.org hosted drivers or those in the Linux tree hosted in the official Xen repository. > Is there any official source which defines the blkback/front protocol? Only the source code in the Xen repository. Both the FreeBSD extensions that I did and those for the Linux system you are using are likely backwards compatible to the original interface, but unfortunately are not compatible with each other. > Maybe > the right option is to have a loader tunable dev.xn.linuxback to control which > version of the protocol is used? What a mess. -- Justin From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 02:42:25 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id EA519106564A for ; Wed, 6 Jul 2011 02:42:24 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id B286F152958 for ; Wed, 6 Jul 2011 02:42:24 +0000 (UTC) Received: (qmail 4707 invoked from network); 6 Jul 2011 02:42:23 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 6 Jul 2011 02:42:23 -0000 Message-ID: <4E13CB8F.8000009@freebsd.org> Date: Tue, 05 Jul 2011 19:42:23 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: gibbs@freebsd.org, "freebsd-xen@freebsd.org" References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org> <4E13C2A4.5040202@FreeBSD.org> In-Reply-To: <4E13C2A4.5040202@FreeBSD.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 02:42:25 -0000 [oops, let's try sending this again with reply-all instead of reply...] On 07/05/11 19:04, Justin T. Gibbs wrote: > On 7/5/11 7:14 PM, Colin Percival wrote: >> On 06/10/11 13:30, Justin T. Gibbs wrote: >>> On 6/9/11 9:26 PM, Colin Percival wrote: >>>> Has anyone seen anything like this? Is it possible that there's a bug >>>> in how our blkfront negotiates the request ring? Does anyone have >>>> ring_pages > 1 in use? >>> >>> The only backend driver I know of that can support more than one ring page >>> is FreeBSD. So, the problem likely is that blkfront is getting the >>> negotiation wrong and only 1 page is in use. >> >> Turns out that the linux backend in question really does support multiple >> pages. > > Can you provide a source or spec reference for this driver? Ah, here it is, dating from 2008: http://xen.1045712.n5.nabble.com/attachment/2527534/0/big_ring.patch Sorry, I had several windows open and thought I was looking at kernel.org tree commits rather than a separate patch. > There was a > proposal (in 2009?) to add multiple page support to the XenSource > provided blkfront/back, but it didn't get committed to the Xen repository. > It also doesn't use the same XenStore variables that you list. Is this > something Amazon developed in house? I don't know what hand if any Amazon had in writing this, but it looks like they've convinced at least Ubuntu and RedHat to use this version. >> There's also an inconsistency about how multiple rings are negotiated: >> We set: >> * ring-pages to the number of pages blkfront wants to use >> * ring-ref to the reference for the first page >> * ring-refXX to the references for other pages >> while linux sets: >> * num-ring-pages to the number of pages blkfront wants to use >> * ring-refXX to the page references. > > You forgot to mention that the FreeBSD driver allows request size to > be negotiated and explicitly supports "chained" requests to break the > 44k request size limit imposed by the original interface. I was trying to highlight the compatibility issues, not list all the ways that FreeBSD is superior to Linux. ;-) >> It seems to be impossible to be compatible with both, since Linux > interprets >> having a value set for ring-ref to indicate that the single-ring > protocol is >> being used and doesn't check anything else. > > It's not "Linux". There is no multi-page support in either the kernel.org > hosted drivers or those in the Linux tree hosted in the official Xen > repository. Agreed. But it's multiple Linux distributions (and EC2). >> Is there any official source which defines the blkback/front protocol? > > Only the source code in the Xen repository. Both the FreeBSD extensions > that I did and those for the Linux system you are using are likely backwards > compatible to the original interface, but unfortunately are not compatible > with each other. Yes, both versions are definitely backwards compatible. >> Maybe >> the right option is to have a loader tunable dev.xn.linuxback to > control which >> version of the protocol is used? > > What a mess. Yep. Mess or not, shall I go ahead with having a loader tunable control this, or can you think of a better solution? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 04:14:10 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 52A5D106566B for ; Wed, 6 Jul 2011 04:14:10 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id EFD8714E214 for ; Wed, 6 Jul 2011 04:14:09 +0000 (UTC) Received: (qmail 6161 invoked from network); 6 Jul 2011 04:14:09 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 6 Jul 2011 04:14:09 -0000 Message-ID: <4E13E111.9070005@freebsd.org> Date: Tue, 05 Jul 2011 21:14:09 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: gibbs@freebsd.org, "freebsd-xen@freebsd.org" References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org> <4E13C2A4.5040202@FreeBSD.org> <4E13CB8F.8000009@freebsd.org> In-Reply-To: <4E13CB8F.8000009@freebsd.org> X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------030903070809000902080104" Cc: Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 04:14:10 -0000 This is a multi-part message in MIME format. --------------030903070809000902080104 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 07/05/11 19:42, Colin Percival wrote: > On 07/05/11 19:04, Justin T. Gibbs wrote: >> On 7/5/11 7:14 PM, Colin Percival wrote: >>> Maybe the right option is to have a loader tunable dev.xn.linuxback to >>> control which version of the protocol is used? >> >> What a mess. > > Yep. Mess or not, shall I go ahead with having a loader tunable control this, > or can you think of a better solution? Does anyone object to the attached patch? It keeps the differing behaviour to a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST NOT set it with a linux blkback -- but otherwise errs in the direction of setting more variables than are needed, to maximize the possibility of a future blkback being compatible with both blkback_is_linux=0 and blkback_is_linux=1. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid --------------030903070809000902080104 Content-Type: text/x-diff; name="blkfront.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="blkfront.c.patch" Index: sys/dev/xen/blkfront/blkfront.c =================================================================== --- sys/dev/xen/blkfront/blkfront.c (revision 223815) +++ sys/dev/xen/blkfront/blkfront.c (working copy) @@ -616,7 +616,16 @@ if (setup_blkring(sc) != 0) return; + /* Different backends use different names for this variable. */ error = xs_printf(XST_NIL, node_path, + "num-ring-pages","%u", sc->ring_pages); + if (error) { + xenbus_dev_fatal(sc->xb_dev, error, + "writing %s/num-ring-pages", + node_path); + return; + } + error = xs_printf(XST_NIL, node_path, "ring-pages","%u", sc->ring_pages); if (error) { xenbus_dev_fatal(sc->xb_dev, error, @@ -673,6 +682,9 @@ xenbus_set_state(sc->xb_dev, XenbusStateInitialised); } +static int blkback_is_linux = 0; +TUNABLE_INT("dev.xbd.blkback_is_linux", &blkback_is_linux); + static int setup_blkring(struct xb_softc *sc) { @@ -702,14 +714,16 @@ return (error); } } - error = xs_printf(XST_NIL, xenbus_get_node(sc->xb_dev), - "ring-ref","%u", sc->ring_ref[0]); - if (error) { - xenbus_dev_fatal(sc->xb_dev, error, "writing %s/ring-ref", - xenbus_get_node(sc->xb_dev)); - return (error); + if (!blkback_is_linux || sc->ring_pages == 1) { + error = xs_printf(XST_NIL, xenbus_get_node(sc->xb_dev), + "ring-ref","%u", sc->ring_ref[0]); + if (error) { + xenbus_dev_fatal(sc->xb_dev, error, "writing %s/ring-ref", + xenbus_get_node(sc->xb_dev)); + return (error); + } } - for (i = 1; i < sc->ring_pages; i++) { + for (i = 0; i < sc->ring_pages; i++) { char ring_ref_name[]= "ring_refXX"; snprintf(ring_ref_name, sizeof(ring_ref_name), "ring-ref%u", i); --------------030903070809000902080104-- From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 19:02:37 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 9CE2F1065673 for ; Wed, 6 Jul 2011 19:02:37 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id 165B214E3D6 for ; Wed, 6 Jul 2011 19:02:36 +0000 (UTC) Received: (qmail 16082 invoked from network); 6 Jul 2011 19:02:33 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 6 Jul 2011 19:02:33 -0000 Message-ID: <4E14B149.6030307@freebsd.org> Date: Wed, 06 Jul 2011 12:02:33 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: freebsd-net@freebsd.org, "freebsd-xen@freebsd.org" X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------060605090108080700050202" Cc: Subject: how to get "max # of mbufs in a packet" from xn to the tcp stack? X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 19:02:37 -0000 This is a multi-part message in MIME format. --------------060605090108080700050202 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi all, I've attached a patch which fixes the "nfrags > 18, netback won't be able to handle it" problem with xen netfront when TSO is enabled. It's not finished, though: + int max_mbuf_chain_len = 16; /* XXX Set this based on interface? */ I'm not sure what the right way is to feed a value from the interface up into tcp_output; can someone advise? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid --------------060605090108080700050202 Content-Type: text/x-diff; name="tcp_mbuf_chain_limit.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tcp_mbuf_chain_limit.patch" Index: kern/uipc_mbuf.c =================================================================== --- kern/uipc_mbuf.c (revision 223824) +++ kern/uipc_mbuf.c (working copy) @@ -525,12 +525,14 @@ * only their reference counts are incremented. */ struct mbuf * -m_copym(struct mbuf *m, int off0, int len, int wait) +m_copy_nbufs(struct mbuf *m, int off0, int len, int wait, long * outlen, + int nbufmax) { struct mbuf *n, **np; int off = off0; struct mbuf *top; int copyhdr = 0; + int len_orig = len; KASSERT(off >= 0, ("m_copym, negative off %d", off)); KASSERT(len >= 0, ("m_copym, negative len %d", len)); @@ -546,7 +548,7 @@ } np = ⊤ top = 0; - while (len > 0) { + while (len > 0 && nbufmax-- > 0) { if (m == NULL) { KASSERT(len == M_COPYALL, ("m_copym, length > size of mbuf chain")); @@ -584,6 +586,9 @@ if (top == NULL) mbstat.m_mcfail++; /* XXX: No consistency. */ + if (outlen) + *outlen = len_orig - len; + return (top); nospace: m_freem(top); @@ -591,6 +596,13 @@ return (NULL); } +struct mbuf * +m_copym(struct mbuf *m, int off0, int len, int wait) +{ + + return (m_copy_nbufs(m, off0, len, wait, NULL, INT_MAX)); +} + /* * Returns mbuf chain with new head for the prepending case. * Copies from mbuf (chain) n from off for len to mbuf (chain) m Index: netinet/tcp_output.c =================================================================== --- netinet/tcp_output.c (revision 223824) +++ netinet/tcp_output.c (working copy) @@ -183,6 +183,7 @@ int sack_rxmit, sack_bytes_rxmt; struct sackhole *p; int tso; + int max_mbuf_chain_len = 16; /* XXX Set this based on interface? */ struct tcpopt to; #if 0 int maxburst = TCP_MAXBURST; @@ -806,16 +807,6 @@ struct mbuf *mb; u_int moff; - if ((tp->t_flags & TF_FORCEDATA) && len == 1) - TCPSTAT_INC(tcps_sndprobe); - else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) { - tp->t_sndrexmitpack++; - TCPSTAT_INC(tcps_sndrexmitpack); - TCPSTAT_ADD(tcps_sndrexmitbyte, len); - } else { - TCPSTAT_INC(tcps_sndpack); - TCPSTAT_ADD(tcps_sndbyte, len); - } MGETHDR(m, M_DONTWAIT, MT_DATA); if (m == NULL) { SOCKBUF_UNLOCK(&so->so_snd); @@ -847,7 +838,8 @@ mtod(m, caddr_t) + hdrlen); m->m_len += len; } else { - m->m_next = m_copy(mb, moff, (int)len); + m->m_next = m_copy_nbufs(mb, moff, len, M_DONTWAIT, + &len, max_mbuf_chain_len); if (m->m_next == NULL) { SOCKBUF_UNLOCK(&so->so_snd); (void) m_free(m); @@ -856,6 +848,18 @@ } } + /* Update stats here as m_copy_nbufs may have adjusted len. */ + if ((tp->t_flags & TF_FORCEDATA) && len == 1) + TCPSTAT_INC(tcps_sndprobe); + else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) { + tp->t_sndrexmitpack++; + TCPSTAT_INC(tcps_sndrexmitpack); + TCPSTAT_ADD(tcps_sndrexmitbyte, len); + } else { + TCPSTAT_INC(tcps_sndpack); + TCPSTAT_ADD(tcps_sndbyte, len); + } + /* * If we're sending everything we've got, set PUSH. * (This will keep happy those implementations which only Index: sys/mbuf.h =================================================================== --- sys/mbuf.h (revision 223824) +++ sys/mbuf.h (working copy) @@ -849,6 +849,7 @@ int, int, int, int); struct mbuf *m_copypacket(struct mbuf *, int); void m_copy_pkthdr(struct mbuf *, struct mbuf *); +struct mbuf *m_copy_nbufs(struct mbuf *, int, int, int, long *, int); struct mbuf *m_copyup(struct mbuf *n, int len, int dstoff); struct mbuf *m_defrag(struct mbuf *, int); void m_demote(struct mbuf *, int); --------------060605090108080700050202-- From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 22:55:20 2011 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58BA3106566B; Wed, 6 Jul 2011 22:55:20 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 3060B8FC18; Wed, 6 Jul 2011 22:55:20 +0000 (UTC) Received: from Justins-MacBook-Pro.local (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p66Muhlp079759 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 6 Jul 2011 16:56:43 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) Message-ID: <4E14E7D2.9040708@FreeBSD.org> Date: Wed, 06 Jul 2011 16:55:14 -0600 From: "Justin T. Gibbs" Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 MIME-Version: 1.0 To: Colin Percival References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org> <4E13C2A4.5040202@FreeBSD.org> <4E13CB8F.8000009@freebsd.org> <4E13E111.9070005@freebsd.org> In-Reply-To: <4E13E111.9070005@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Wed, 06 Jul 2011 16:56:43 -0600 (MDT) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-xen@freebsd.org" Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@FreeBSD.org List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 22:55:20 -0000 On 7/5/11 10:14 PM, Colin Percival wrote: > On 07/05/11 19:42, Colin Percival wrote: > > On 07/05/11 19:04, Justin T. Gibbs wrote: > >> On 7/5/11 7:14 PM, Colin Percival wrote: > >>> Maybe the right option is to have a loader tunable dev.xn.linuxback to > >>> control which version of the protocol is used? > >> > >> What a mess. > > > > Yep. Mess or not, shall I go ahead with having a loader tunable control this, > > or can you think of a better solution? > > Does anyone object to the attached patch? It keeps the differing behaviour to > a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST NOT set > it with a linux blkback -- but otherwise errs in the direction of setting more > variables than are needed, to maximize the possibility of a future blkback > being compatible with both blkback_is_linux=0 and blkback_is_linux=1. It would be better to just change the FreeBSD blkback driver to be compatible with the RedHat convention. I'm still unclear on why the current FreeBSD blkfront driver believes that it can use more than one page in your configuration given that the RedHat blkfront doesn't advertise this capability in a way that the FreeBSD blkfront understands (max-ring-pages isn'te set by blkback). Did you do something to force blkfront to use more than one page? -- Justin From owner-freebsd-xen@FreeBSD.ORG Wed Jul 6 23:55:37 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id AF2BD1065672 for ; Wed, 6 Jul 2011 23:55:37 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id 59A88151A84 for ; Wed, 6 Jul 2011 23:55:37 +0000 (UTC) Received: (qmail 19037 invoked from network); 6 Jul 2011 23:55:36 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 6 Jul 2011 23:55:36 -0000 Message-ID: <4E14F5F8.7080108@freebsd.org> Date: Wed, 06 Jul 2011 16:55:36 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: gibbs@freebsd.org References: <4DF18EE5.5090704@freebsd.org> <4DF27EFF.5060005@FreeBSD.org> <4E13B6E9.9070202@freebsd.org> <4E13C2A4.5040202@FreeBSD.org> <4E13CB8F.8000009@freebsd.org> <4E13E111.9070005@freebsd.org> <4E14E7D2.9040708@FreeBSD.org> In-Reply-To: <4E14E7D2.9040708@FreeBSD.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-xen@freebsd.org" Subject: Re: breakage in blkfront with ring_pages > 1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 23:55:37 -0000 On 07/06/11 15:55, Justin T. Gibbs wrote: > On 7/5/11 10:14 PM, Colin Percival wrote: >> On 07/05/11 19:42, Colin Percival wrote: >>> Yep. Mess or not, shall I go ahead with having a loader tunable > control this, >>> or can you think of a better solution? >> >> Does anyone object to the attached patch? It keeps the differing > behaviour to >> a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST > NOT set >> it with a linux blkback -- but otherwise errs in the direction of > setting more >> variables than are needed, to maximize the possibility of a future blkback >> being compatible with both blkback_is_linux=0 and blkback_is_linux=1. > > It would be better to just change the FreeBSD blkback driver to be > compatible with the RedHat convention. Fine with me, but that will of course break compatibility between pre- and post- patch versions of FreeBSD. Aside from you, how many people use the FreeBSD blkback driver? > I'm still unclear on why the current FreeBSD blkfront driver believes > that it can use more than one page in your configuration given that the > RedHat blkfront doesn't advertise this capability in a way that the FreeBSD > blkfront understands (max-ring-pages isn'te set by blkback). Did you do > something to force blkfront to use more than one page? I'm seeing max-ring-pages set to 4. I don't know what tree EC2 is using on their Dom0 -- I've heard rumours that there's a lot of RedHat going on behind the scenes, but given Amazon's size it's entirely possible that they got a blkback which hasn't been released publicly. I'm sent an email asking where their blkback came from; I'll report back if/when I know something. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@FreeBSD.ORG Fri Jul 8 01:59:29 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id B2241106564A for ; Fri, 8 Jul 2011 01:59:29 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id 3D81914DD8E for ; Fri, 8 Jul 2011 01:59:29 +0000 (UTC) Received: (qmail 15917 invoked from network); 8 Jul 2011 01:59:28 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 8 Jul 2011 01:59:28 -0000 Message-ID: <4E166480.8070703@freebsd.org> Date: Thu, 07 Jul 2011 18:59:28 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101220 Thunderbird/3.0.11 MIME-Version: 1.0 To: "freebsd-xen@freebsd.org" X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: FreeBSD now available on more EC2 instance types X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 01:59:29 -0000 FreeBSD can now run on all 64-bit EC2 instance types: http://www.daemonology.net/blog/2011-07-08-FreeBSD-on-EC2-via-defenestration.html -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@FreeBSD.ORG Fri Jul 8 05:04:49 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 087C7106566B for ; Fri, 8 Jul 2011 05:04:49 +0000 (UTC) (envelope-from wind@planetcreations.com) Received: from smtp.net4india.com (smtp.net4india.com [118.67.236.11]) by mx1.freebsd.org (Postfix) with ESMTP id 764C58FC16 for ; Fri, 8 Jul 2011 05:04:48 +0000 (UTC) Received: from [14.97.201.137] (helo=CORE108) by smtp.net4india.com with esmtp (Exim 4.72) (envelope-from ) id 1QesMg-00083d-W1 for freebsd-xen@freebsd.org; Thu, 07 Jul 2011 22:57:51 +0530 From: "Khushi P." To: freebsd-xen@freebsd.org MIME-Version: 1.0 Date: Thu, 7 Jul 2011 22:57:48 +0530 Message-ID: <1464101732585194@core108> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: General Awarness - Unknown Facts X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mail@khushiwebworld.com List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 05:04:49 -0000 Hi, Various unknown facts are listed on the website http://khushiwebworld.= com . The unknown facts are really good an= d I have found it very useful, so thought to share the same with you. = They are categorized as 1. Unknown facts 2. Health Facts 3. Ayurveda facts 4. And Many more . After visiting the website http://khushiwebworld.com you will have awareness towards various unknown facts which= is very useful in day to day life. Regards . Khushi **********************************************************************= *** Khushi World Disclaimer Policy This mail has ben sent to freebsd-xen@freebsd.org dated 07/07/2011= In case if you have received this email accidently and If you prefer n= ot to receive future emails of this type please UNSUBSCRIBE . If this link does not works reply with the subject "Remo= ve" . =20 **********************************************************************= ******