From owner-freebsd-fs@FreeBSD.ORG Thu Apr 4 02:31:19 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DA774D13 for ; Thu, 4 Apr 2013 02:31:19 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id 88972FE7 for ; Thu, 4 Apr 2013 02:31:19 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id r342KWvV005394; Wed, 3 Apr 2013 21:20:32 -0500 (CDT) Date: Wed, 3 Apr 2013 21:20:32 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Graham Allan Subject: Re: zfs home directories best practice In-Reply-To: <515B84E8.2090202@physics.umn.edu> Message-ID: References: <515B84E8.2090202@physics.umn.edu> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Wed, 03 Apr 2013 21:20:32 -0500 (CDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 02:31:19 -0000 On Tue, 2 Apr 2013, Graham Allan wrote: > We're building a new NFS home directory server on FreeBSD with ZFS. The > Solaris ZFS Best Practices docs say to create a separate filesystem for each > user home directory. My instinct is to ask "Are you serious???". My gut > feeling isn't entirely logical but the idea of getting 1000+ lines of output > from a simple "df" just feels wrong... > > Can anyone comment about how well this approach actually works, specifically > on FreeBSD? (we're running 9.1) Obviously it has some nice features, such as > quota controls, snapshots directly available to users within their home, etc, > but it leaves me concerned. I chatted with some neighbors who have a larger, > Solaris-based shop, and they said that with 10,000 user home filesystems, > their server could take an hour to boot (at least using the default startup > scripts). They reverted to having one big shared filesystem for all, but > would like to revisit the per-user approach with fewer users per server. As others have said, the NFS export is where the time gets expended. It is not necessary to have zfs do the exports for you. There is indeed value to each user having their own home directory. The 1000+ lines of output from 'df' is quite useful if it tells you exactly how much space each user has consumed so that you don't have to do something really evil like run 'du' in each directory to find the hogs. The ability to snapshot filesystems on a per-user basis is quite useful. Probably I shouldn't be answering since I have only used this at a small scale with a Solaris server (but with a FreeBSD client). Having a good NFS automounter on the clients is useful if you have a home directory per user. The AMD automounter which comes with FreeBSD is just barely competent for the task. It is able to automount user home directories on request but not enumerate them via 'ls /home/*' as Solaris and Apple OS X clients can. It will only list the currently automounted directories. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/