Date: Tue, 22 Jan 2013 09:40:24 -0600 From: Kevin Day <toasty@dragondata.com> To: Warren Block <wblock@wonkity.com> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org>, Scott Long <scottl@samsco.org>, wblock@freebsd.org Subject: Re: RFC: Suggesting ZFS "best practices" in FreeBSD Message-ID: <A196DCF7-B5C0-4EAB-960F-795D99A4B7F5@dragondata.com> In-Reply-To: <alpine.BSF.2.00.1301220804530.61512@wonkity.com> References: <314B600D-E8E6-4300-B60F-33D5FA5A39CF@sarenet.es> <alpine.BSF.2.00.1301220804530.61512@wonkity.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jan 22, 2013, at 9:12 AM, Warren Block <wblock@wonkity.com> wrote: >=20 > I would like to see guidelines for at least two common scenarios: >=20 > Multi-terabyte file server with multi-drive pool. [=85] > The first is easy with the defaults, but particular tuning could be = beneficial. And would be a good place to talk about NFS on ZFS, usage = of SSDs, and so on. I run ftpmirror.your.org, which is a 72 x 3TB drive ZFS server. It's a = very busy server. It currently houses the only off-site backup of all of = the Wikimedia projects(121TB), a full FreeBSD FTP mirror(1T), a full = CentOS mirror, all of FreeBSD-Archive(1.5TB), FreeBSD-CVS, etc. It's = usually running between 100 and 1500mbps of ethernet traffic in/out of = it. There are usually around 15 FTP connections, 20-50 HTTP connections, = 10 rsync connections and 1 or 2 CVS connections.=20 The only changes we've made that are ZFS specific are atime=3Doff and = sync=3Ddisabled. Nothing we do uses atimes so disabling that cuts down = on a ton of unnecessary writes. Disabling sync is okay here too - we're = just mirroring stuff that's available elsewhere, so there's no threat of = data loss. Other than some TCP tuning in sysctl.conf, this is running a = totally stock kernel with no special settings.=20 I've looked at using an SSD for meta-data only caching, but it appears = that we've got far more than 256GB of metadata here that's being = accessed regularly (nearly every file is being stat'ed when rsync runs) = so I'm guessing it's not going to be incredibly effective unless I buy a = seriously large SSD. If you have any specific questions I'm happy to answer though. -- Kevin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A196DCF7-B5C0-4EAB-960F-795D99A4B7F5>