From owner-freebsd-current Mon Nov 25 10:28:18 2002 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E152E37B401; Mon, 25 Nov 2002 10:28:16 -0800 (PST) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 56C5A43E88; Mon, 25 Nov 2002 10:28:12 -0800 (PST) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id NAA06277; Mon, 25 Nov 2002 13:28:09 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id gAPIRdV75432; Mon, 25 Nov 2002 13:27:39 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15842.27547.385354.151541@grasshopper.cs.duke.edu> Date: Mon, 25 Nov 2002 13:27:39 -0500 (EST) To: Bosko Milekic Cc: Julian Elischer , Robert Watson , Luigi Rizzo , current@freebsd.org Subject: Re: mbuf header bloat ? In-Reply-To: <20021125130005.A75177@unixdaemons.com> References: <15840.8629.324788.887872@grasshopper.cs.duke.edu> <15841.17237.826666.653505@grasshopper.cs.duke.edu> <20021125130005.A75177@unixdaemons.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bosko Milekic writes: > > On Sun, Nov 24, 2002 at 04:23:33PM -0500, Andrew Gallatin wrote: > > If we're going to nitpick the mbuf system, a much, much worse problem > > is that you cannot allocate an mbuf chain w/o holding Giant, which > > stems from the mbuf system eventually calling kmem_malloc(). This > > effectively prevents any network driver from being giant-free. When > > mbufs are low, mb_alloc() calls mb_pop_cont(). This, in turn, calls > > kmem_malloc() which requires Giant... > > This is not entirely true. You can allocate an mbuf chain without > holding Giant if the caches are well populated - and they should be > in the common/general case. You can in fact modify the allocator to > just not do a kmem_malloc() if called with M_DONTWAIT, but I don't > think you'd want to do this at this point. Unfortunately, the common case is not good enough when figuring out locking for network drivers and the uncommon case forces all network drivers under Giant. I was thinking about doing what you describe, but my misconception about ref counters being malloced was holding me back. We'll also need a kproc that can wake up every now and then to expand the pool if allocations at interrupt time failed. Or do you already have a mechanism for that? > > The mbuf system calls malloc in other ways too. The first time you > > use a cluster, m_ext.ext_ref_cnt is malloc()'ed, and malloc is called > > when the mbuf map is expanded... I assume malloc will eventually > > call kmem_malloc(), leading to the same locking problems. > > Actually, that has been changed a while ago. The ref_cnt for clusters > is no longer malloc()'d. A contiguous space is used from which > cluster ref counters are "allocated" (just like in the old/original > design). This modification was made a while ago as an optimisation. Cool. The following code confused me into making the above statement. Is it out of date? #define _mext_init_ref(m, ref) do { \ (m)->m_ext.ref_cnt = ((ref) == NULL) ? \ malloc(sizeof(u_int), M_MBUF, M_NOWAIT) : (u_int *)(ref); \ if ((m)->m_ext.ref_cnt != NULL) { \ *((m)->m_ext.ref_cnt) = 0; \ MEXT_ADD_REF((m)); \ } \ } while (0) Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message