From owner-freebsd-net@FreeBSD.ORG  Thu Jan 23 03:45:07 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 96A3192B;
 Thu, 23 Jan 2014 03:45:07 +0000 (UTC)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 50DCB16CD;
 Thu, 23 Jan 2014 03:45:07 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s0N3j4Bb090696;
 Wed, 22 Jan 2014 22:45:04 -0500 (EST)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s0N3j4ZS090693;
 Wed, 22 Jan 2014 22:45:04 -0500 (EST) (envelope-from wollman)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <21216.36928.132606.318491@hergotha.csail.mit.edu>
Date: Wed, 22 Jan 2014 22:45:04 -0500
From: Garrett Wollman <wollman@freebsd.org>
To: Navdeep Parhar <np@FreeBSD.org>
Subject: Re: Use of contiguous physical memory in cxgbe driver
In-Reply-To: <52E06104.70600@FreeBSD.org>
References: <21216.22944.314697.179039@hergotha.csail.mit.edu>
 <52E06104.70600@FreeBSD.org>
X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 22 Jan 2014 22:45:04 -0500 (EST)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: net@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Jan 2014 03:45:07 -0000

<<On Wed, 22 Jan 2014 16:23:32 -0800, Navdeep Parhar <np@FreeBSD.org> said:

> On 01/22/14 15:52, Garrett Wollman wrote:
>> At this point everyone is well aware that requiring contiguous
>> physical page when the hardware can do scatter-gather is a very bad
>> idea.

> I wouldn't put it this way.  Using buffers with size > PAGE_SIZE has its
> advantages.

These advantages do not come close to balancing out the disadvantage
of "your server eventually falls off the network due to physmem
fragmentation, better hope you can reset it remotely because driving
in to work at 3 AM sucks."

If any free pages are available at all, the allocation of a 4k jumbo
mbuf will succeed.  A 9k jumbo mbuf requires three physically
contiguous pages, and it's very, very easy for physical memory to get
fragmented to the point where that is impossible.

-GAWollman