Date: Thu, 21 Feb 2013 20:47:17 -0600 (CST) From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> To: Kevin Day <toasty@dragondata.com> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: Improving ZFS performance for large directories Message-ID: <alpine.GSO.2.01.1302212046040.11141@freddy.simplesystems.org> In-Reply-To: <19E0C908-79F1-43F8-899C-6B60F998D4A5@dragondata.com> References: <19DB8F4A-6788-44F6-9A2C-E01DEA01BED9@dragondata.com> <CAJjvXiE%2B8OMu_yvdRAsWugH7W=fhFW7bicOLLyjEn8YrgvCwiw@mail.gmail.com> <F4420A8C-FB92-4771-B261-6C47A736CF7F@dragondata.com> <20130201192416.GA76461@server.rulingia.com> <19E0C908-79F1-43F8-899C-6B60F998D4A5@dragondata.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 19 Feb 2013, Kevin Day wrote: > Sorry for the late followup, I've been doing some testing with an L2ARC device. > >>> Doing it twice back-to-back makes a bit of difference but it's still slow either way. >> >> ZFS can very conservative about caching data and twice might not be enough. >> I suggest you try 8-10 times, or until the time stops reducing. > > Timing doing an "ls" in large directories 20 times, the first is the slowest, then all subsequent listings are roughly the same. There doesn't appear to be any gain after 20 repetitions You might consider that the bottleneck might be in 'ls' or something outside of zfs. Make sure that you are doing 'ls -f' or else you are just measuring its sorting performance. On a Solaris 10 system in a zfs directory with a million files: % time ls -f |wc -l 1000002 /bin/ls -F -f 0.76s user 0.93s system 89% cpu 1.897 total wc -l 0.08s user 0.02s system 5% cpu 1.697 total % time ls |wc -l 1000000 /bin/ls -F 4.32s user 8.10s system 97% cpu 12.682 total wc -l 0.08s user 0.02s system 0% cpu 12.432 total Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.GSO.2.01.1302212046040.11141>