From owner-freebsd-fs@FreeBSD.ORG Thu Apr 4 22:21:16 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4E10F135 for ; Thu, 4 Apr 2013 22:21:16 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1908BF0 for ; Thu, 4 Apr 2013 22:21:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAD38XVGDaFvO/2dsb2JhbABDDgiDJoMovWGBG3SCHwEBBAEjVgUWGAICDRkCWQaIIQYMrxeSSIEjjEGBAzQHgi2BEwOWboEgj22CTFsggS89 X-IronPort-AV: E=Sophos;i="4.87,411,1363147200"; d="scan'208";a="22509868" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 04 Apr 2013 18:21:14 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5EE03B4041; Thu, 4 Apr 2013 18:21:14 -0400 (EDT) Date: Thu, 4 Apr 2013 18:21:14 -0400 (EDT) From: Rick Macklem To: Graham Allan Message-ID: <338176325.529222.1365114074330.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <515D0287.2060704@physics.umn.edu> Subject: Re: zfs home directories best practice MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 22:21:16 -0000 Graham Allan wrote: > On 4/3/2013 6:56 PM, Rick Macklem wrote: > >> > > Well, there isn't any limit to the # of exported file systems afaik, > > but updating a large /etc/exports file takes quite a bit of time and > > when you use mountd (the default) for this, you can have problems. > > (You either have a period of time when no client can get response > > from the server or a period of time when I/O fails because the > > file system isn't re-exported yet.) > > > > If you choose this approach, you should look seriously at using > > nfse (on sourceforge) instead of mountd. > > That's an interesting-looking project though I'm beginning to think > that > unless there's some serious downside to the "one big filesystem", I > should just defer the per-user filesystems for the system after this > one. As you remind me below, I'll probably have other issues to chase > down besides that one (performance as well as making the jump to > NFSv4...) > > > You might also want to contact Garrett Wollman w.r.t. the NFS > > server patch(es) and setup he is using, since he has been > > working through performance issues (relatively successfully > > now, as I understand) for a fairly large NFS/ZFS server. > > You should be able to find a thread discussing this on > > freebsd-fs or freebsd-current. > > I found the thread "NFS server bottlenecks" on freebsd-hackers, which > has a lot of interesting reading, and then also "NFS DRC size" on > freebsd-fs. We might dig into some of that material (eg DRC-related > patches) though I probably need to spend more time on basics first > (kernel parameters, number of nfsd threads, etc). > Hopefully I can get to-gether with ivoras@ in May and come up with a patch for head/current. If you want to try something before then, this patch is roughly what Garrett is using: http://people.freebsd.org/~rmacklem/drc4.patch rick > Thanks for the pointers, > > Graham > --