From owner-freebsd-net@FreeBSD.ORG  Mon Mar 11 16:05:19 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id E1AD5336;
 Mon, 11 Mar 2013 16:05:19 +0000 (UTC)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 by mx1.freebsd.org (Postfix) with ESMTP id 7E400C31;
 Mon, 11 Mar 2013 16:05:19 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2BG5Ixp073053;
 Mon, 11 Mar 2013 12:05:18 -0400 (EDT)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2BG5I6v073052;
 Mon, 11 Mar 2013 12:05:18 -0400 (EDT) (envelope-from wollman)
Date: Mon, 11 Mar 2013 12:05:18 -0400 (EDT)
From: Garrett Wollman <wollman@hergotha.csail.mit.edu>
Message-Id: <201303111605.r2BG5I6v073052@hergotha.csail.mit.edu>
To: andre@freebsd.org
Subject: Re: Limits on jumbo mbuf cluster allocation
In-Reply-To: <513DB550.5010004@freebsd.org>
References: <1154859394.3748712.1362959165419.JavaMail.root@erie.cs.uoguelph.ca>
Organization: none
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7
 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 11 Mar 2013 12:05:18 -0400 (EDT)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: freebsd-net@freebsd.org, rmacklem@uoguelph.ca
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Mar 2013 16:05:19 -0000

In article <513DB550.5010004@freebsd.org>, andre@freebsd.org writes:

>Garrett's problem is receive side specific and NFS can't do much about it.
>Unless, of course, NFS is holding on to received mbufs for a longer time.

Well, I have two problems: one is running out of mbufs (caused, we
think, by ixgbe requiring 9k clusters when it doesn't actually need
them), and one is livelock.  Allowing potentially hundreds of clients
to queue 2 MB of requests before TCP pushes back on them helps to
sustain the livelock once it gets started, and of course those packets
will be of the 9k jumbo variety, which makes the first problem worse
as well.

-GAWollman