From owner-freebsd-net@FreeBSD.ORG  Wed Feb  5 05:27:18 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 8EAA5228;
 Wed,  5 Feb 2014 05:27:18 +0000 (UTC)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 476B71F60;
 Wed,  5 Feb 2014 05:27:18 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s155RGb4004806;
 Wed, 5 Feb 2014 00:27:16 -0500 (EST)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s155RGJE004803;
 Wed, 5 Feb 2014 00:27:16 -0500 (EST) (envelope-from wollman)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <21233.52147.912022.488615@hergotha.csail.mit.edu>
Date: Wed, 5 Feb 2014 00:27:15 -0500
From: Garrett Wollman <wollman@freebsd.org>
To: freebsd-net@freebsd.org
Subject: ixgbe/NFS m_defrag() instrumentation
X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 05 Feb 2014 00:27:16 -0500 (EST)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: rmacklem@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Feb 2014 05:27:18 -0000

I instrumented calls to m_defrag() in ixgbe.  As expected, it gets
called *a lot* when NFS is running with the default read size of 64k.
A simple benchmark (single-threaded sequential read of a 128 GB file
which I didn't even run to completion) tells the tale:

$ sysctl dev.ix.0.mbuf_defrag_attempted
dev.ix.0.mbuf_defrag_attempted: 1737994

(There's already a similar counter for m_defrag() failures, which made
it easy to add this counter.  Unfortunately, there is no analogous
instrumentation in cxgbe so I couldn't do likewise for that NIC.)

-GAWollman