From owner-freebsd-hackers@FreeBSD.ORG  Wed Mar  3 09:59:52 2004
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id A3A1516A4CE
	for <freebsd-hackers@freebsd.org>;
	Wed,  3 Mar 2004 09:59:52 -0800 (PST)
Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 647F743D2F
	for <freebsd-hackers@freebsd.org>;
	Wed,  3 Mar 2004 09:59:52 -0800 (PST)
	(envelope-from gallatin@cs.duke.edu)
Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu
	[152.3.145.30])
	by duke.cs.duke.edu (8.12.10/8.12.10) with ESMTP id i23HxpF6004099
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Wed, 3 Mar 2004 12:59:52 -0500 (EST)
Received: (from gallatin@localhost)
	by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id i23Hxkko080587;
	Wed, 3 Mar 2004 12:59:46 -0500 (EST)
	(envelope-from gallatin)
From: Andrew Gallatin <gallatin@cs.duke.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <16454.7442.840699.631259@grasshopper.cs.duke.edu>
Date: Wed, 3 Mar 2004 12:59:46 -0500 (EST)
To: Luigi Rizzo <rizzo@icir.org>
In-Reply-To: <20040303094524.A26257@xorpc.icir.org>
References: <FE045D4D9F7AED4CBFF1B3B813C85337045D832C@mail.sandvine.com>
	<16453.62383.59435.72390@grasshopper.cs.duke.edu>
	<20040303094524.A26257@xorpc.icir.org>
X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid
cc: freebsd-hackers@freebsd.org
Subject: Re: em0, polling performance, P4 2.8ghz FSB 800mhz
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Mar 2004 17:59:52 -0000


Luigi Rizzo writes:
 > On Wed, Mar 03, 2004 at 10:03:11AM -0500, Andrew Gallatin wrote:
<...>
 > > I'm trying to design a new ethernet API for a firmware-based nic,
 > > and I'm trying to convince a colleague that having separate
 > > receive rings for small and large frames is a really good thing.
 > 
 > i am actually not very convinced either, unless you are telling me
 > that there is a way to preserve ordering. Or you'd be in trouble
 > when, on your busy link, there is a mismatch between user-level and
 > link-level block sizes.
 > 
 > So, what is your design like, you want to pass the NIC buffers of
 > 2-3 different sizes and let the NIC choose from the most appropriate
 > pool depending on the incoming frame size, but still return
 > received frames in a single ring in arrival order ?

Yes, exactly.  This way you get to pass the stack small (<MHLEN)
frames in mbufs, rather than clusters without doing something like
copying them in the driver's rx interrupt handler.  You can allocate
tons of mbufs so that you can absorb the occasional burst (or spike in
host latency) without being as bad of pig as you'd be if you allocated
a huge number of clusters ;)

You also get to set yourself up for zero-copy receive by splitting
the headers into mbufs, and the payloads into jumbo clusters
that can get page-flipped.  But that's a lot trickier and not
really in the scope of the initial implementation.

Drew