From owner-freebsd-net Fri Jul 5 8:37:52 2002 Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD7C437B400 for ; Fri, 5 Jul 2002 08:37:47 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2ABC643E3B for ; Fri, 5 Jul 2002 08:37:47 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id LAA24350; Fri, 5 Jul 2002 11:37:45 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g65FbF029122; Fri, 5 Jul 2002 11:37:15 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15653.48427.102047.117270@grasshopper.cs.duke.edu> Date: Fri, 5 Jul 2002 11:37:15 -0400 (EDT) To: Bosko Milekic Cc: "Kenneth D. Merry" , net@FreeBSD.ORG Subject: Re: virtually contig jumbo mbufs (was Re: new zero copy sockets snapshot) In-Reply-To: <20020705112101.A566@unixdaemons.com> References: <20020620114511.A22413@unixdaemons.com> <15634.534.696063.241224@grasshopper.cs.duke.edu> <20020620134723.A22954@unixdaemons.com> <15652.46870.463359.853754@grasshopper.cs.duke.edu> <20020705002056.A5365@unixdaemons.com> <15653.35919.24295.698563@grasshopper.cs.duke.edu> <20020705093435.A25047@unixdaemons.com> <15653.43437.658461.49860@grasshopper.cs.duke.edu> <20020705104247.A518@unixdaemons.com> <15653.45342.299963.692216@grasshopper.cs.duke.edu> <20020705112101.A566@unixdaemons.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Bosko Milekic writes: > > On Fri, Jul 05, 2002 at 10:45:50AM -0400, Andrew Gallatin wrote: > > > > Bosko Milekic writes: > > > > > > On Fri, Jul 05, 2002 at 10:14:05AM -0400, Andrew Gallatin wrote: > > > > I think this would be fine, But we'd need to know more about the > > > > hardware limitations of the popular GiGE boards out there. We know > > > > Tigon-II can handle 4 scatters, but are there any that can handle 3 > > > > but not four? > > > > > > Why would you need 4? I can absolutely guarantee that a jumbo buf > > > will not go over 3 pages (on i386, and 2 pages on alpha). > > > > > > > Perhaps I misread your earlier message. I thought you wanted to be > > free to align them on something other than a page boundary, so as to > > not waste so much space. In that case you'd need 4 scatters on i386, > > right? > > All I meant was that I wouldn't guarantee that the buffer _begins_ at > a page boundary. However, it still should only span at most 3 pages > on i386. Let's say for the sake of argument that we won't need more > than 9212 bytes for a jumbo buffer. Let's make the buffer 4 bytes > bigger so that we'll have space for a ref. counter for it too (how > convenient), so we'll allocate buffers of 9212 + 4 = 9216 bytes. It's > time to allocate from the map: I'll try to grab 9 pages for a total > of 4 jumbo buffers each spanning at most 3 pages (this is i386). I'll > start the first buffer at the beginning of the first page and it'll > end ((9216 - 8192) = 1024) bytes into the third page. I'll then start > the second jumbo buffer there and finish it 2048 bytes into 5th page. > I would then start the 3rd jumbo buffer there and finish it 3072 bytes > into the 7th page. The last jumbo buf would start there and finish at > the end of the 9th page. If my calculations are correct, that means > that each jumbo buffer would span exactly 3 pages, but only the first > one in the grouping would begin at a page boundary. There is also > minimum wastage here. Briefly: > > Assuming that size_of_buf >= 2 * PAGE_SIZE, this is the above: > > PAGE_SIZE % (size_of_buf - 2 * PAGE_SIZE) == 0 > > We pick size_of_buf as small as possible but enough to accomodate the > MTU and, and if possible, the ref. counter. I think 9216 should be a > good number. At the same time, we try to make: > > bufs_per_bucket = PAGE_SIZE (integer DIV) (size_of_buf - 2 * PAGE_SIZE) > > bufs_per_bucket fairly reasonable (i.e., not too big). In the above > scenario, size_of_buf = 9216 and bufs_per_bucket = 4. > > Admittedly, the numbers above were cooked up, but I think that they're > fairly reasonable. That looks great. I hadn't realized that things would divide out so cleanly! I think you should go for it after talking to wpaul & pdeuskar (and anybody else who maintains GiGE drivers these days). Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message