From owner-freebsd-fs@FreeBSD.ORG Thu Apr 4 00:22:01 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9D487ED2 for ; Thu, 4 Apr 2013 00:22:01 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx1.freebsd.org (Postfix) with ESMTP id 5D6FEB1C for ; Thu, 4 Apr 2013 00:22:01 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id hv10so1919767vcb.40 for ; Wed, 03 Apr 2013 17:21:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=n3+4753YLb2fOGg8Gtfy+K0caXMk9TkPXSXSKhp/25s=; b=r3m6219h6Zt/K591nhq7nsvYNNF63JeyNrBXKljmiv4X8SfbdYRXBU2h1pxRMtc5we L+Aw0cxxmcsXw3Px82rEosFfBQakGnZcYFQ5nFiQfEEImWLg/aLYVCSS/045UullnwIx mvIjIDqK4nfQHBTFZFyRBLDkDfSBtn5LytXf1gWeu4CYz9mSQ1cG1ieUw07kjEVr8GcM ov7aqju1qUZrB4NMTVBdFnUW08jwfHTgUADRSTnuir0xxEd9X4NgRMuEPFKjExCJIQ5M C+ThS4tZyQjVehfWnzX8iLN++J/E4CzPm86yivrPt+Q81LxC3ES2ROyhfAaAS5bIYpg4 2eEQ== MIME-Version: 1.0 X-Received: by 10.52.27.17 with SMTP id p17mr2705824vdg.0.1365034914927; Wed, 03 Apr 2013 17:21:54 -0700 (PDT) Received: by 10.58.132.203 with HTTP; Wed, 3 Apr 2013 17:21:54 -0700 (PDT) In-Reply-To: <238802714.483457.1365033407086.JavaMail.root@erie.cs.uoguelph.ca> References: <515B84E8.2090202@physics.umn.edu> <238802714.483457.1365033407086.JavaMail.root@erie.cs.uoguelph.ca> Date: Wed, 3 Apr 2013 17:21:54 -0700 Message-ID: Subject: Re: zfs home directories best practice From: Mehmet Erol Sanliturk To: Rick Macklem Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 00:22:01 -0000 On Wed, Apr 3, 2013 at 4:56 PM, Rick Macklem wrote: > Graham Allan wrote: > > We're building a new NFS home directory server on FreeBSD with ZFS. > > The > > Solaris ZFS Best Practices docs say to create a separate filesystem > > for > > each user home directory. My instinct is to ask "Are you serious???". > > My > > gut feeling isn't entirely logical but the idea of getting 1000+ lines > > of output from a simple "df" just feels wrong... > > > > Can anyone comment about how well this approach actually works, > > specifically on FreeBSD? (we're running 9.1) Obviously it has some > > nice > > features, such as quota controls, snapshots directly available to > > users > > within their home, etc, but it leaves me concerned. I chatted with > > some > > neighbors who have a larger, Solaris-based shop, and they said that > > with > > 10,000 user home filesystems, their server could take an hour to boot > > (at least using the default startup scripts). They reverted to having > > one big shared filesystem for all, but would like to revisit the > > per-user approach with fewer users per server. > > > > Ours wouldn't be so large, but we could easily have around 1000 user > > filesystems. I haven't tested yet what effect that would have on boot > > time, though hope to test it over the next week. Perhaps it implies > > other resource usage besides the boot time issue (is there any limit > > to > > number of filesystems mounted or NFS-exported?). I wonder if anyone > > here > > has built a system along these lines and has experiences to share. > > > Well, there isn't any limit to the # of exported file systems afaik, > but updating a large /etc/exports file takes quite a bit of time and > when you use mountd (the default) for this, you can have problems. > (You either have a period of time when no client can get response > from the server or a period of time when I/O fails because the > file system isn't re-exported yet.) > > If you choose this approach, you should look seriously at using > nfse (on sourceforge) instead of mountd. > > You might also want to contact Garrett Wollman w.r.t. the NFS > server patch(es) and setup he is using, since he has been > working through performance issues (relatively successfully > now, as I understand) for a fairly large NFS/ZFS server. > You should be able to find a thread discussing this on > freebsd-fs or freebsd-current. > > rick > > > Thanks for any comments, > > > > Graham > > -- > > ------------------------------------------------------------------------- > > Graham Allan > > School of Physics and Astronomy - University of Minnesota > > ------------------------------------------------------------------------- > > _______________________________________________ > > > I am sorry that , my previous message sent early . After trying some more retry , it worked : http://sourceforge.net/projects/nfse/ ( NFSE for FreeBSD NFS server ) Thank you very much . Mehmet Erol Sanliturk