From owner-freebsd-net@FreeBSD.ORG Thu Jan 30 03:22:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11C454E5 for ; Thu, 30 Jan 2014 03:22:30 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CB1321A9F for ; Thu, 30 Jan 2014 03:22:29 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAJfE6VKDaFve/2dsb2JhbABZhBuDAboagRl0giUBAQEEI1YbGAICDRkCWQYTiAWqeqBxF4EpjSI0B4JvgUkEiUmgfoNLHoFu X-IronPort-AV: E=Sophos;i="4.95,746,1384318800"; d="scan'208";a="91740667" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 29 Jan 2014 22:22:22 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D571DB40D2; Wed, 29 Jan 2014 22:22:22 -0500 (EST) Date: Wed, 29 Jan 2014 22:22:22 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <1315174039.18735121.1391052142869.JavaMail.root@uoguelph.ca> In-Reply-To: <21225.49699.916951.881502@hergotha.csail.mit.edu> Subject: Re: Terrible NFS performance under 9.2-RELEASE? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jan 2014 03:22:30 -0000 Garrett Wollman wrote: > < said: > > > However, I do suspect we'll be putting a refined version of the > > patch > > in head someday (maybe April, sooner would have to be committed by > > someone else). I suspect that Garrett's code for server read will > > work > > well and I'll cobble something to-gether for server readdir and > > client write. > > Once I can get this mps(4) issue ironed out, I should be in a > position > to get some real data on this. > If you can check the network device driver you use and if it looks like it has a scatter size of less than 36 (often a constant with "TXSEG" or "TX_SEG" in the name) and calls either m_defrag() or m_collapse(), adding a counter to see if those functions are being called, would be nice. If the m_collapse()/m_defrag() function is being called without the patch and not with the patch, the performance difference may be avoiding that call and not a more generic benefit. I just did a quick find/grep and it looks like a lot of drivers have *TXSEGS* set to around 32 and then call one of two functions for more than that. Since without a patch, 64K NFS reads/writes hand sosend() an mbuf list of 34 entries, it seems like this could be happening a lot. (I didn't look to see which ones set if_hw_tsomax to significantly less than 64K.) Thanks for working on this, rick. ps: you might want to combine your patch with mine, so readdir and client side writes use 4K clusters. > -GAWollman > >