From owner-freebsd-current@FreeBSD.ORG Fri Sep 29 23:12:46 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6214C16A407; Fri, 29 Sep 2006 23:12:46 +0000 (UTC) (envelope-from rrs@cisco.com) Received: from sj-iport-6.cisco.com (sj-iport-6.cisco.com [171.71.176.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DF1E43D8A; Fri, 29 Sep 2006 23:12:36 +0000 (GMT) (envelope-from rrs@cisco.com) Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-6.cisco.com with ESMTP; 29 Sep 2006 16:12:33 -0700 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-4.cisco.com (8.12.11.20060308/8.12.11) with ESMTP id k8TNCXCA029019; Fri, 29 Sep 2006 16:12:33 -0700 Received: from xbh-sjc-221.amer.cisco.com (xbh-sjc-221.cisco.com [128.107.191.63]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id k8TNCW1E003174; Fri, 29 Sep 2006 16:12:32 -0700 (PDT) Received: from xfe-sjc-211.amer.cisco.com ([171.70.151.174]) by xbh-sjc-221.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 29 Sep 2006 16:12:32 -0700 Received: from [127.0.0.1] ([171.68.225.134]) by xfe-sjc-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 29 Sep 2006 16:12:32 -0700 Message-ID: <451DA83C.4050808@cisco.com> Date: Fri, 29 Sep 2006 19:11:56 -0400 From: Randall Stewart User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050920 X-Accept-Language: en-us, en MIME-Version: 1.0 To: John-Mark Gurney References: <451C4850.5030302@freebsd.org> <451D884F.1030807@cisco.com> <20060929213722.GR80527@funkthat.com> <451D973C.8070004@freebsd.org> <20060929231007.GS80527@funkthat.com> In-Reply-To: <20060929231007.GS80527@funkthat.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 29 Sep 2006 23:12:32.0346 (UTC) FILETIME=[BDEFBBA0:01C6E41C] DKIM-Signature: a=rsa-sha1; q=dns; l=2158; t=1159571553; x=1160435553; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=rrs@cisco.com; z=From:Randall=20Stewart=20 |Subject:Re=3A=20Much=20improved=20sosend_*()=20functions; X=v=3Dcisco.com=3B=20h=3DVUE9hWftOvPULvUJ9EhsWe/RDnk=3D; b=SW2mOXpPB6lMuG3nFobwJGfZ1GztAodblcNhw4PziNJ/rSBtM6mpmmCLwG/aZYCRcZ8V3n0Z OP7F12zIZrfjREa8RRRHyWwpQTBJxLqeZXpEOsXAs99RrtVUeXnpqKPa; Authentication-Results: sj-dkim-4.cisco.com; header.From=rrs@cisco.com; dkim=pass ( sig from cisco.com verified; ); Cc: freebsd-net@freebsd.org, Mike Silbersack , freebsd-current@freebsd.org, Andre Oppermann , gallatin@cs.duke.edu Subject: Re: Much improved sosend_*() functions X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 23:12:46 -0000 John-Mark Gurney wrote: > Andre Oppermann wrote this message on Fri, Sep 29, 2006 at 23:59 +0200: > >>>w/ 512 byte mbuf and a 2k cluster just to store just 1514 bytes of data, >>>that's only 60% effeciency wrt to memory usage... so, we currently >>>waste 40% of memory allocated to mbufs+clusters... Even reducing >>>mbufs back to 128 or 256 would be a big help, though IPSEC I believe >>>would have issues... >> >>mbufs are 256 bytes. > > > Hmmm.. I keep getting this confused... maybe because there was discussion > about increasing this a few years back... or maybe because NOTES has > it as 512.. :) > > >>>Hmmm.. If we switched clusters to 1536 bytes in size, we'd be able to >>>fit 8 in 12k (though I guess for 8k page boxes we'd do 16 in 24k)... The >>>only issue w/ that would be that a few of the clusters would possibly >>>split page boundaries... How much this would effect performance would >>>be an interesting question to answer... >> >>Splitting page boundaries is not an option as it may not be physically >>contigous. > > > unless we do something strange like allocate them contigously... though > that introduces another set of issues.... > > >>Just don't overengineer the stuff. Mbufs are only used temporarily and >>a bit theoretical waste is not much a problem (so far at least). > > > Well, I beg to differ... most gige cards grab mbuf+cluster for every > single ring buffer they have.. which is usually 512... so every gige > interface for the most part consumes 1meg of memory that is not > reusable... because if we run out of mbuf+clusters to replace in the > receive ring, we will not tap into the 1meg of mbuf+clusters available > to us... so, if you have a quad gige, that's 4megs wasted, plus w/ the > fact that we could only use ~65% of that memory, that's a lot of memory > wasted... > > Yeh, everyone says you have gigs of memory, but do we really want to > be known as the wasteful OS? > Let me try to find some cycles (somewhere) and play with this :-) R -- Randall Stewart NSSTG - Cisco Systems Inc. 803-345-0369 815-342-5222 (cell)