From owner-freebsd-net@FreeBSD.ORG Wed Jan 29 22:27:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35F7DBE6; Wed, 29 Jan 2014 22:27:19 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8CE81D3A; Wed, 29 Jan 2014 22:27:18 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s0TMRE5r006197 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Jan 2014 14:27:14 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s0TMREGH006196; Wed, 29 Jan 2014 14:27:14 -0800 (PST) (envelope-from jmg) Date: Wed, 29 Jan 2014 14:27:14 -0800 From: John-Mark Gurney To: Adrian Chadd Subject: Re: Big physically contiguous mbuf clusters Message-ID: <20140129222714.GK93141@funkthat.com> Mail-Followup-To: Adrian Chadd , Garrett Wollman , FreeBSD Net References: <21225.20047.947384.390241@khavrinen.csail.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 29 Jan 2014 14:27:14 -0800 (PST) Cc: Garrett Wollman , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2014 22:27:19 -0000 Adrian Chadd wrote this message on Wed, Jan 29, 2014 at 14:21 -0800: > On 29 January 2014 10:54, Garrett Wollman wrote: > > Resolved: that mbuf clusters longer than one page ought not be > > supported. There is too much physical-memory fragmentation for them > > to be of use on a moderately active server. 9k mbufs are especially > > bad, since in the fragmented case they waste 3k per allocation. > > I've been wondering whether it'd be feasible to teach the physical > memory allocator about >page sized allocations and to create zones of > slightly more physically contiguous memory. > > For servers with lots of memory we could then keep these around and > only dip into them for temporary allocations (eg not VM pages that may > be held for some unknown amount of time.) > > Question is - can we enforce that kind of behaviour? It shouldn't be too hard to do... Since everything pretty much goes through uma we can adopt a scheme similar to what Solaris does (read Magazines and Vmem: Extending the Slab Allocator to Many CPUs and Arbitrary Resources)... Instead of dealing w/ page size allocations, everything is larger, say 16KB, and broken down from there... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."