From owner-freebsd-net@FreeBSD.ORG Tue Feb 11 23:30:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 790CDEF7; Tue, 11 Feb 2014 23:30:05 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DE74A1E50; Tue, 11 Feb 2014 23:30:04 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id n7so14429163qcx.30 for ; Tue, 11 Feb 2014 15:30:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9yVXwaVVqPeN4Fha315zgo3wSTUcvkRhbGan/xgcr9s=; b=k1Gocyu4EHqqeqA0Rst2W+KQYAhMnKqIT6lxfn6ihzRN6wjivEJ71G1f2xbp6iv7LX ezBmkfazgfTS33ZIwbEV7Qr11c5lPgUjzrcd56R07Bz59HJxOuR/PP7EueKnxd5NH+nS tDEYfbC2h7Xjx6H9nlIWTQOsj1DexxlGiMPVWUtPHgBygTTxoKBwKRqGtW0w5rge/+ul SROTJtwzjNwxm13T8wZ78GLW/o1gBX2cHEYbbPD4Z7qxiJW/0mvcvDE2ASal1Pmj2kzh I1SC25hrJr5bXtndFShKXf8D267J5nz9QQF84lI4IwcCcG8wiudK7T1lDa/oRBQEfb7X B3wg== MIME-Version: 1.0 X-Received: by 10.140.42.54 with SMTP id b51mr50150877qga.87.1392161404108; Tue, 11 Feb 2014 15:30:04 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Tue, 11 Feb 2014 15:30:04 -0800 (PST) In-Reply-To: <201402111348.52135.jhb@freebsd.org> References: <21216.22944.314697.179039@hergotha.csail.mit.edu> <21216.36928.132606.318491@hergotha.csail.mit.edu> <201402111348.52135.jhb@freebsd.org> Date: Tue, 11 Feb 2014 15:30:04 -0800 X-Google-Sender-Auth: -9V2MhqYjWqnZ2N1ad-tNqCJ4XA Message-ID: Subject: Re: Use of contiguous physical memory in cxgbe driver From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , Garrett Wollman , Navdeep Parhar , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 23:30:05 -0000 On 11 February 2014 10:48, John Baldwin wrote: > On Thursday, January 23, 2014 2:32:51 am Adrian Chadd wrote: >> It's about time we taught the physmem allocator to be more conducive >> to physically contiguous allocations. >> >> A server with gigabytes of memory should be able to keep a couple tens >> of megabytes of 64k sized allocation chunks around for exactly this. > > Alan Cox already taught the physmem allocator to do this for superpages. > However, this change was part of the superpages changes, so you can't count > experiences from machines older than about 7.2 when evaluating the > effectiveness of the new allocator. the problem is that we don't have pressure to _not_ fragment the physical memory from the allocator, so all of the memory ends up getting very fragmented versy quickly. the physmem superpage stuff stops being viable after a short while of say, "pound lots of packets around from vm objects" workload, because suddenly we end up chewing through all of physical memory with pages extremely quickly. -a