Date: Wed, 12 Jun 2013 11:26:06 +0200 From: Attila Nagy <bra@fsn.hu> To: "Kenneth D. Merry" <ken@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: An order of magnitude higher IOPS needed with ZFS than UFS Message-ID: <51B83EAE.7060603@fsn.hu> In-Reply-To: <20130611232124.GA42577@nargothrond.kdm.org> References: <51B79023.5020109@fsn.hu> <253074981.119060.1370985609747.JavaMail.root@erie.cs.uoguelph.ca> <20130611232124.GA42577@nargothrond.kdm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, On 06/12/13 01:21, Kenneth D. Merry wrote: > On Tue, Jun 11, 2013 at 17:20:09 -0400, Rick Macklem wrote: >> >> ken@ recently committed a change to the new NFS server to add file >> handle affinity support to it. He reported that he had found that, >> without file handle affinity, that ZFS's sequential reading heuristic >> broke badly (or something like that, you can probably find the email >> thread or maybe he will chime in). > That is correct. The problem, if the I/O is sequential, is that simultaneous > requests for adjacent blocks in a file will get farmed out to different The IO is pretty much random, and the files aren't so big either (mean size around 400k). > threads in the NFS server. These can easily go down into ZFS out of order, > and make the ZFS prefetch code think that the file is not being read > sequentially. It blows away the zfetch stream, and you wind up with a lot > of I/O bandwidth getting used (with a lot of prefetching done and then > re-done), but not much performance. I've tried disabling prefetch, without any noticeable effects. > > Linux clients are more likely than FreeBSD and MacOS clients to queue a lot > of reads to the server. The clients are also FreeBSD (8.3 and 7.2 mostly). Running NFSv3 of course. > >> Anyhow, you could try switching the FreeBSD 9 system to use the old >> NFS server (assuming your clients are doing NFSv3 mounts) and see if >> that has a significant effect. (For FreeBSD9, the old server has file >> handle affinity, but the new server does not.) > If using the old NFS server helps, then the FHA code for the new server > will help as well. Perhaps more, because the default FHA tuning parameters > have changed somewhat and parallel writes are now possible. > > If you want to try out the FHA changes in stable/9, I just MFCed them, > change 251641. > Sure, I will try both 251641 and the old nfsd. Thanks,
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51B83EAE.7060603>