From owner-freebsd-net@FreeBSD.ORG  Thu Feb 13 21:05:16 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id D5EA7681;
 Thu, 13 Feb 2014 21:05:16 +0000 (UTC)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 7206313CD;
 Thu, 13 Feb 2014 21:05:16 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s1DL5Enc049484;
 Thu, 13 Feb 2014 16:05:14 -0500 (EST)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s1DL5ErT049481;
 Thu, 13 Feb 2014 16:05:14 -0500 (EST) (envelope-from wollman)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <21245.13194.145088.287817@hergotha.csail.mit.edu>
Date: Thu, 13 Feb 2014 16:05:14 -0500
From: Garrett Wollman <wollman@bimajority.org>
To: John Baldwin <jhb@freebsd.org>
Subject: Re: Use of contiguous physical memory in cxgbe driver
In-Reply-To: <6154054.5G06XIekg6@ralph.baldwin.cx>
References: <21216.22944.314697.179039@hergotha.csail.mit.edu>
 <201402121446.19278.jhb@freebsd.org>
 <CAJ-VmokDb4dkhzO6NnYrFKKV2g-bau8H47h2g=wOkvGcqYzRow@mail.gmail.com>
 <6154054.5G06XIekg6@ralph.baldwin.cx>
X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 13 Feb 2014 16:05:14 -0500 (EST)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: FreeBSD Net <freebsd-net@freebsd.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Feb 2014 21:05:16 -0000

<<On Thu, 13 Feb 2014 15:16:54 -0500, John Baldwin <jhb@freebsd.org> said:

> It seeks to use the smallest size possible however.  It is true that
> we don't attempt to move a busy page elsewhere to free up memory
> (e.g. if you had a 2MB free chunk with one busy 4k page in the
> middle), but we can't really do that safely.  Given the existence of
> the direct map, we can't relocate a page and be sure that we have
> also relocated all possible pointers to it.

I doubt that this can be usefully auto-configured.  As an
administrator, I would be perfectly happy with a situation where I can
tell the system to set aside some amount of memory at initialization
(say, 16 superpages per (virtual) CPU) and get better performance.
It's not a real imposition on a 128-GB server to set aside 1 GB of
memory for big contiguous allocations if the applications will benefit
from it.  But in the absence of a guaranteed allocation, network
drivers need to avoid requiring contiguous physical memory when they
don't need it.

-GAWollman