From owner-freebsd-stable@FreeBSD.ORG Fri Dec 31 23:47:49 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B98781065670 for ; Fri, 31 Dec 2010 23:47:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 6195A8FC08 for ; Fri, 31 Dec 2010 23:47:49 +0000 (UTC) Received: from omta17.westchester.pa.mail.comcast.net ([76.96.62.89]) by qmta01.westchester.pa.mail.comcast.net with comcast id pzfk1f0021vXlb851znpPJ; Fri, 31 Dec 2010 23:47:49 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta17.westchester.pa.mail.comcast.net with comcast id pzno1f0082tehsa3dznojD; Fri, 31 Dec 2010 23:47:49 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 21FE59B427; Fri, 31 Dec 2010 15:47:47 -0800 (PST) Date: Fri, 31 Dec 2010 15:47:47 -0800 From: Jeremy Chadwick To: Peter Jeremy Message-ID: <20101231234747.GA8171@icarus.home.lan> References: <4D1AF388.3080107@infracaninophile.co.uk> <4D1B7431.7070808@infracaninophile.co.uk> <4D1BD8D0.5010402@langille.org> <4D1C4A2D.4020206@infracaninophile.co.uk> <4D1C7929.3040809@langille.org> <20101231233343.GB48579@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20101231233343.GB48579@server.vk2pj.dyndns.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org, Dan Langille Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Dec 2010 23:47:49 -0000 On Sat, Jan 01, 2011 at 10:33:43AM +1100, Peter Jeremy wrote: > On 2010-Dec-30 07:20:57 -0500, Dan Langille wrote: > >The reason I've not installed ZFS on root is because of the added > >complications. I run the OS on ufs (with gmirror) and my data is on > >ZFS. We must be hanging out with different groups. Most of the people > >I know don't have ZFS on root. > > My primary system at home is setup this way - primarily because at the > time I built it (Nov 2008), I felt ZFS was a bit immature and wanted > to have src and obj on UFS so I could do a rebuild if I lost access to > ZFS for some reason. My experience has been that the UFS root has > caused me far more headaches than the ZFS parts. I've since done some > reconfiguration and plan to switch to ZFS root soon. > > Based on my experiences at home, I converted my desktop at work to > pure ZFS. The only issues I've run into have been programs that > extensively use mmap(2) - which is a known issue with ZFS. Is your ZFS root filesystem associated with a pool that's mirrored or using raidzX? What about mismatched /boot content (ZFS vs. UFS)? What about booting into single-user mode? http://wiki.freebsd.org/ZFSOnRoot indirectly hints at these problems but doesn't outright admit them (yet should), so I'm curious to know how people have solved them. Remembering manual "one-offs" for a system configured this way is not acceptable (read: highly prone to error/mistake). Is it worth the risk? Most administrators don't have the tolerance for stuff like that in the middle of a system upgrade or what not; they should be able to follow exactly what's in the handbook, to a tee. There's a link to www.dan.me.uk at the bottom of the above Wiki page that outlines "the madness" that's required to configure the setup, all of which has to be done by hand. I don't know many administrators who are going to tolerate this when deploying numerous machines, especially when compounded by the complexities mentioned above. The mmap(2) and sendfile(2) complexities will bite an junior or mid-level SA in the butt too -- they won't know why software starts failing or behaving oddly (FreeBSD ftpd is a good example). It just so happens that Apache, out-of-the-box, comes with mmap and sendfile use disabled. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |