From owner-freebsd-current@FreeBSD.ORG Fri Sep 29 22:06:04 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C742316A40F; Fri, 29 Sep 2006 22:06:04 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id B7F2843D55; Fri, 29 Sep 2006 22:06:00 +0000 (GMT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.13.6/8.13.6) with ESMTP id k8TM60LP023391 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Fri, 29 Sep 2006 18:06:00 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id k8TM5t7A061952; Fri, 29 Sep 2006 18:05:55 -0400 (EDT) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17693.39106.950631.742167@grasshopper.cs.duke.edu> Date: Fri, 29 Sep 2006 18:05:54 -0400 (EDT) To: Andre Oppermann In-Reply-To: <451D9440.6060105@cisco.com> References: <451C4850.5030302@freebsd.org> <451D884F.1030807@cisco.com> <20060929213722.GR80527@funkthat.com> <451D9440.6060105@cisco.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: Much improved sosend_*() functions X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 22:06:04 -0000 Andre, I meant to ask: Did you try 16KB jumbos? Did they perform any better than page-sized jumbos? Also, if we're going to change how mbufs work, let's add something like Linux's skb_frag_t frags[MAX_SKB_FRAGS]; In FreeBSD parlence, this embeds something like an array of sf_bufs pointers in mbuf. The big difference to a chain of M_EXT mbufs is that you need to allocate only one mbuf wrapper, rather than one for each item in the list. Also, the reference is kept in the page (or sf_buf) itself, and the data offset is kept in the skbbuf (or mbuf). This allows us to do cool things like allocate a single page, and use both halves of it for 2 separate 1500 byte frames. This allows us to achieve *amazing* results in combination with LRO, because it allows us to do, on average, many fewer allocations per byte. Especially in combination with Linux's "high order" page allocations. Using order-2 allocations and LRO, I've actually seen 10GbE line rate receives on a wimpy 2.0GHz Athlon64. Drew