From owner-freebsd-net@FreeBSD.ORG  Thu Jan 23 00:23:35 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 99945863;
 Thu, 23 Jan 2014 00:23:35 +0000 (UTC)
Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com
 [IPv6:2607:f8b0:400e:c03::235])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 66DC214A8;
 Thu, 23 Jan 2014 00:23:35 +0000 (UTC)
Received: by mail-pa0-f53.google.com with SMTP id lj1so1100173pab.40
 for <multiple recipients>; Wed, 22 Jan 2014 16:23:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:message-id:date:from:user-agent:mime-version:to:subject
 :references:in-reply-to:content-type:content-transfer-encoding;
 bh=oAxcYp4l+1rDT6BRvRlJKuxMGaAmx71wGUOFtLONTmw=;
 b=iVlAwqEPiRIlS9byGZekisaDcSJJZhWPxCSWzua7FNLWeNdCo+odzbTZUrkxjRNXzd
 tGbpk9X0zHvF6yGZz4eepPpAEJBS9E8g6I3kjaV5ScR/XJGCDpk5M9MgujS60NxYths6
 JcVdV8Z/LWf08R2eerEFrqEYBeTsiW1bxzzOclxHJoaXHDq3DGV9v3fg9jPz2l+RpneJ
 NVMSc4aI8t4MQSXeKLkA/OvCOqyT8QWX5qhBUAGZvn4ro4xjGv4O5QIjXkUVdXeP2efU
 167m4hnHiHh3doNNjJ8xJff+JTDsD0wFC9qbEB89zjwh2GE2kkESYqW3qiisLRHKaHwL
 dtBA==
X-Received: by 10.66.41.106 with SMTP id e10mr4723060pal.109.1390436614936;
 Wed, 22 Jan 2014 16:23:34 -0800 (PST)
Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58])
 by mx.google.com with ESMTPSA id gg10sm28632310pbc.46.2014.01.22.16.23.33
 for <multiple recipients>
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Wed, 22 Jan 2014 16:23:33 -0800 (PST)
Sender: Navdeep Parhar <nparhar@gmail.com>
Message-ID: <52E06104.70600@FreeBSD.org>
Date: Wed, 22 Jan 2014 16:23:32 -0800
From: Navdeep Parhar <np@FreeBSD.org>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:24.0) Gecko/20100101 Thunderbird/24.2.0
MIME-Version: 1.0
To: Garrett Wollman <wollman@freebsd.org>, net@freebsd.org
Subject: Re: Use of contiguous physical memory in cxgbe driver
References: <21216.22944.314697.179039@hergotha.csail.mit.edu>
In-Reply-To: <21216.22944.314697.179039@hergotha.csail.mit.edu>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Jan 2014 00:23:35 -0000

On 01/22/14 15:52, Garrett Wollman wrote:
> At this point everyone is well aware that requiring contiguous
> physical page when the hardware can do scatter-gather is a very bad
> idea.

I wouldn't put it this way.  Using buffers with size > PAGE_SIZE has its
advantages.

> I have a new server under test (running 9.2) which uses Chelsio
> rather than Intel 10G Ethernet controllers.  We fixed the Intel driver
> not to use more-than-page-sized jumbo mbufs.  Can anyone say with
> certainty whether the Chelsio hardware actually requires
> physically-contiguous allocations for jumbo frames?

No it doesn't.  It will do a scatter DMA if the next available rx
buffer's size is less than the size of the incoming frame.

> Has this already
> been fixed in a more recent driver?  (The hardware is identified
> specifically as a T420-CR.)

I'm not sure there's anything to fix here but I can add a knob that
would limit the driver to a maximum of PAGE_SIZE sized rx buffers.

Regards,
Navdeep