From owner-freebsd-current Sun Nov 24 13:24:10 2002 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BCF0837B401; Sun, 24 Nov 2002 13:24:08 -0800 (PST) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id B40C743E4A; Sun, 24 Nov 2002 13:24:04 -0800 (PST) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id QAA08736; Sun, 24 Nov 2002 16:24:03 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id gAOLNX473981; Sun, 24 Nov 2002 16:23:33 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15841.17237.826666.653505@grasshopper.cs.duke.edu> Date: Sun, 24 Nov 2002 16:23:33 -0500 (EST) To: Julian Elischer Cc: Robert Watson , Luigi Rizzo , current@freebsd.org Subject: Re: mbuf header bloat ? In-Reply-To: References: <15840.8629.324788.887872@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > > > On Sat, 23 Nov 2002, Andrew Gallatin wrote: > > > > > As you eloquently state, there are a number of tradeoffs involved. On > > a 64-bit platform, 99% of users are paying 40 bytes/pkt for something > > that they will never use. On x86, 99.99% of users are paying 20 > > bytes/pkt for a feature they will never use. At least a signifigant > > fraction of nics make use of csum offloading (xl, ti, bge, em, myri). > > > the downside to the TAG stuff is that you need to allocate a separate > tag storage, and that is a malloc.. which has certain characteristics > vs the mbuf allocator. We have a special allocator for mbufs for a > reason. (I'm not sure how many of the original reasons still apply). > so it's worth looking at whether malloc is a suitable method of > allocating all that stuff before we take it out of the mbuf. > If we're going to nitpick the mbuf system, a much, much worse problem is that you cannot allocate an mbuf chain w/o holding Giant, which stems from the mbuf system eventually calling kmem_malloc(). This effectively prevents any network driver from being giant-free. When mbufs are low, mb_alloc() calls mb_pop_cont(). This, in turn, calls kmem_malloc() which requires Giant... The mbuf system calls malloc in other ways too. The first time you use a cluster, m_ext.ext_ref_cnt is malloc()'ed, and malloc is called when the mbuf map is expanded... I assume malloc will eventually call kmem_malloc(), leading to the same locking problems. I know that both tru64 and aix just malloc their mbufs. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message