From owner-freebsd-current@FreeBSD.ORG Fri Sep 29 22:29:48 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 484C216A412 for ; Fri, 29 Sep 2006 22:29:48 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C22C43D69 for ; Fri, 29 Sep 2006 22:29:47 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 98113 invoked from network); 29 Sep 2006 22:30:57 -0000 Received: from dotat.atdotat.at (HELO [62.48.0.47]) ([62.48.0.47]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 29 Sep 2006 22:30:57 -0000 Message-ID: <451D9E59.9050000@freebsd.org> Date: Sat, 30 Sep 2006 00:29:45 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b) Gecko/20050217 MIME-Version: 1.0 To: Andrew Gallatin References: <451C4850.5030302@freebsd.org> <451D884F.1030807@cisco.com> <20060929213722.GR80527@funkthat.com> <451D9440.6060105@cisco.com> <17693.39106.950631.742167@grasshopper.cs.duke.edu> In-Reply-To: <17693.39106.950631.742167@grasshopper.cs.duke.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: Much improved sosend_*() functions X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 22:29:48 -0000 Andrew Gallatin wrote: > Andre, > > I meant to ask: Did you try 16KB jumbos? Did they perform > any better than page-sized jumbos? No, I didn't try 16K jumbos. The problem with anything larger than page size is that it may look contigous in kernel memory but isn't in physical memory. Thus you need the same number of descriptors for the network card as with page sized (4K) clusters. > Also, if we're going to change how mbufs work, let's add something > like Linux's skb_frag_t frags[MAX_SKB_FRAGS]; In FreeBSD parlence, > this embeds something like an array of sf_bufs pointers in mbuf. The > big difference to a chain of M_EXT mbufs is that you need to allocate > only one mbuf wrapper, rather than one for each item in the list. > Also, the reference is kept in the page (or sf_buf) itself, and the > data offset is kept in the skbbuf (or mbuf). We are not going to change how mbufs work. > This allows us to do cool things like allocate a single page, and use > both halves of it for 2 separate 1500 byte frames. This allows us to > achieve *amazing* results in combination with LRO, because it allows > us to do, on average, many fewer allocations per byte. Especially in > combination with Linux's "high order" page allocations. Using order-2 > allocations and LRO, I've actually seen 10GbE line rate receives on a > wimpy 2.0GHz Athlon64. I have just started tackling the receive path. Lets see what comes out of it first before we jump to conclusions. -- Andre