From owner-freebsd-net@FreeBSD.ORG Fri Sep 29 23:10:22 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D86816A415; Fri, 29 Sep 2006 23:10:22 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.FreeBSD.org (Postfix) with ESMTP id EA5CA43D70; Fri, 29 Sep 2006 23:10:11 +0000 (GMT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (1getp7ac4kr9olq0@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id k8TNA8Nr037297; Fri, 29 Sep 2006 16:10:08 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id k8TNA7i3037296; Fri, 29 Sep 2006 16:10:07 -0700 (PDT) (envelope-from jmg) Date: Fri, 29 Sep 2006 16:10:07 -0700 From: John-Mark Gurney To: Andre Oppermann Message-ID: <20060929231007.GS80527@funkthat.com> Mail-Followup-To: Andre Oppermann , freebsd-net@freebsd.org, Randall Stewart , freebsd-current@freebsd.org, Mike Silbersack , gallatin@cs.duke.edu References: <451C4850.5030302@freebsd.org> <451D884F.1030807@cisco.com> <20060929213722.GR80527@funkthat.com> <451D973C.8070004@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <451D973C.8070004@freebsd.org> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Cc: freebsd-net@freebsd.org, Randall Stewart , freebsd-current@freebsd.org, gallatin@cs.duke.edu Subject: Re: Much improved sosend_*() functions X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 23:10:22 -0000 Andre Oppermann wrote this message on Fri, Sep 29, 2006 at 23:59 +0200: > >w/ 512 byte mbuf and a 2k cluster just to store just 1514 bytes of data, > >that's only 60% effeciency wrt to memory usage... so, we currently > >waste 40% of memory allocated to mbufs+clusters... Even reducing > >mbufs back to 128 or 256 would be a big help, though IPSEC I believe > >would have issues... > > mbufs are 256 bytes. Hmmm.. I keep getting this confused... maybe because there was discussion about increasing this a few years back... or maybe because NOTES has it as 512.. :) > >Hmmm.. If we switched clusters to 1536 bytes in size, we'd be able to > >fit 8 in 12k (though I guess for 8k page boxes we'd do 16 in 24k)... The > >only issue w/ that would be that a few of the clusters would possibly > >split page boundaries... How much this would effect performance would > >be an interesting question to answer... > > Splitting page boundaries is not an option as it may not be physically > contigous. unless we do something strange like allocate them contigously... though that introduces another set of issues.... > Just don't overengineer the stuff. Mbufs are only used temporarily and > a bit theoretical waste is not much a problem (so far at least). Well, I beg to differ... most gige cards grab mbuf+cluster for every single ring buffer they have.. which is usually 512... so every gige interface for the most part consumes 1meg of memory that is not reusable... because if we run out of mbuf+clusters to replace in the receive ring, we will not tap into the 1meg of mbuf+clusters available to us... so, if you have a quad gige, that's 4megs wasted, plus w/ the fact that we could only use ~65% of that memory, that's a lot of memory wasted... Yeh, everyone says you have gigs of memory, but do we really want to be known as the wasteful OS? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."