From owner-freebsd-fs@FreeBSD.ORG Thu Mar 29 14:38:46 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2174D106564A for ; Thu, 29 Mar 2012 14:38:46 +0000 (UTC) (envelope-from sven@crashme.org) Received: from celaeno.tauri.mw.lg.virgo.supercluster.net (celaeno.tauri.mw.lg.virgo.supercluster.net [213.252.140.33]) by mx1.freebsd.org (Postfix) with ESMTP id CE17F8FC15 for ; Thu, 29 Mar 2012 14:38:45 +0000 (UTC) Received: from miram.persei.mw.lg.virgo.supercluster.net ([213.252.140.37] helo=[192.168.20.6]) by celaeno.tauri.mw.lg.virgo.supercluster.net with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1SDG4N-0000kJ-GQ; Thu, 29 Mar 2012 14:11:19 +0000 Message-ID: <4F746D8C.8010903@crashme.org> Date: Thu, 29 Mar 2012 16:11:24 +0200 From: Sven Brandenburg User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120310 Thunderbird/11.0 MIME-Version: 1.0 To: Rick Macklem References: <40825357.1703430.1332798477468.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <40825357.1703430.1332798477468.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) Cc: freebsd-fs@freebsd.org Subject: Re: NFSv3, ZFS, 10GE performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Mar 2012 14:38:46 -0000 On 03/26/2012 11:47 PM, Rick Macklem wrote: > MAX_BSIZE is 64kb. I'd like to try making that bigger, but haven't gotten > around to it yet. (If you wanted to try bumping MAX_BSIZE to 128Kb on both > client and server and seeing what happens, that might be interesting, since > my understanding is that ZFS uses a 128Kb block size.) I finally came around and tested it (with 256k and 1M) - there is good and bad news. The good news is the system does indeed boot (off of zfs at least, no idea on ufs) and it does increase performance. I am now seeing roughly 800MB/s off the bat which is quite nice. The bad news is that I had to use a Linux client because the FreeBSD client declined to work: mount_nfs: /mnt, : No buffer space available (Although I will freely admit that my knowledge of where to ajdust this value is rather limited: What I did was changing MAXBSIZE MAXPHYS to 1M in /usr/src/sys/sys/param.h, remaking world+kernel then reboot. I forgot MAXPHYS in my first try and this crashed the client machine as soon as I tried to mount something via nfs. Notably, the server seems to be working ok even with a mismatched MAXPHYS/MAXBSIZE). So far, the results are very promising. regards, Sven