Date: Thu, 21 Jul 2011 21:40:18 +0200 From: Ivan Voras <ivoras@freebsd.org> To: Martin Matuska <mm@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and large directories - caveat report Message-ID: <CAF-QHFX_KmPQjX0AahwE-RTFt-TznRhKzah_eK6tce3dnhomZg@mail.gmail.com> In-Reply-To: <4E286F1F.6010502@FreeBSD.org> References: <j09hk8$svj$1@dough.gmane.org> <4E286F1F.6010502@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21 July 2011 20:25, Martin Matuska <mm@freebsd.org> wrote: > Quoting: > ... The default record size ZFS utilizes is 128K, which is good for many > storage servers that will harbor larger files. However, when dealing with > many files that are only a matter of tens of kilobytes, or even bytes, > considerable slowdown will result. ZFS can easily alter the record size of > the data to be written through the use of attributes. These attributes can > be set at any time through the use of the "zfs set" command. To set the > record size attribute perform "zfs set recordsize=32K pool/share". This will > set the recordsize to 32K on share "share" within pool "pool". This type of > functionality can even be implemented on nested shares for even more > flexibility. ... > > Read more: > http://www.articlesbase.com/information-technology-articles/improving-file-system-performance-utilizing-dynamic-record-sizes-in-zfs-4565092.html#ixzz1SlWZ7BM5 Thank you very much - now if only you took as much effort to explain the possible connection between your quote and my post as it took you to find the quote :) As others explained, ZFS definitely does not use fixed block sizes, which I can easily confirm for you by running iostat: www:~> iostat 1 tty ad4 ad6 ad8 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 32 5.87 123 0.70 5.88 122 0.70 5.99 121 0.71 4 0 8 0 88 0 233 1.54 171 0.26 1.54 166 0.25 1.53 172 0.26 10 0 17 1 72 0 78 7.34 181 1.30 7.02 191 1.31 7.46 177 1.29 0 0 8 0 92 0 78 5.50 234 1.25 5.59 232 1.26 5.27 247 1.27 0 0 10 1 89 ^C KB/t varies and is small. Now I'm working under the hypothesis that the directory pseudo-file itself is hugely fragmented and ZFS fragments it even more every time it adds or removes from it. Any ideas how to verify this?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF-QHFX_KmPQjX0AahwE-RTFt-TznRhKzah_eK6tce3dnhomZg>