From owner-freebsd-fs@FreeBSD.ORG Wed Apr 3 01:52:03 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 826032FF for ; Wed, 3 Apr 2013 01:52:02 +0000 (UTC) (envelope-from allan@physics.umn.edu) Received: from mail.physics.umn.edu (smtp.spa.umn.edu [128.101.220.4]) by mx1.freebsd.org (Postfix) with ESMTP id 60019BDE for ; Wed, 3 Apr 2013 01:52:02 +0000 (UTC) Received: from c-174-53-189-64.hsd1.mn.comcast.net ([174.53.189.64] helo=[192.168.0.136]) by mail.physics.umn.edu with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1UNCRl-000Pwi-AV for freebsd-fs@freebsd.org; Tue, 02 Apr 2013 20:25:09 -0500 Message-ID: <515B84E8.2090202@physics.umn.edu> Date: Tue, 02 Apr 2013 20:24:56 -0500 From: Graham Allan User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mrmachenry.spa.umn.edu X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_20 autolearn=no version=3.3.2 Subject: zfs home directories best practice X-SA-Exim-Version: 4.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 01:52:03 -0000 We're building a new NFS home directory server on FreeBSD with ZFS. The Solaris ZFS Best Practices docs say to create a separate filesystem for each user home directory. My instinct is to ask "Are you serious???". My gut feeling isn't entirely logical but the idea of getting 1000+ lines of output from a simple "df" just feels wrong... Can anyone comment about how well this approach actually works, specifically on FreeBSD? (we're running 9.1) Obviously it has some nice features, such as quota controls, snapshots directly available to users within their home, etc, but it leaves me concerned. I chatted with some neighbors who have a larger, Solaris-based shop, and they said that with 10,000 user home filesystems, their server could take an hour to boot (at least using the default startup scripts). They reverted to having one big shared filesystem for all, but would like to revisit the per-user approach with fewer users per server. Ours wouldn't be so large, but we could easily have around 1000 user filesystems. I haven't tested yet what effect that would have on boot time, though hope to test it over the next week. Perhaps it implies other resource usage besides the boot time issue (is there any limit to number of filesystems mounted or NFS-exported?). I wonder if anyone here has built a system along these lines and has experiences to share. Thanks for any comments, Graham -- ------------------------------------------------------------------------- Graham Allan School of Physics and Astronomy - University of Minnesota -------------------------------------------------------------------------