Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 2 Feb 2013 06:24:16 +1100
From:      Peter Jeremy <peter@rulingia.com>
To:        Kevin Day <toasty@dragondata.com>
Cc:        FreeBSD Filesystems <freebsd-fs@freebsd.org>
Subject:   Re: Improving ZFS performance for large directories
Message-ID:  <20130201192416.GA76461@server.rulingia.com>
In-Reply-To: <F4420A8C-FB92-4771-B261-6C47A736CF7F@dragondata.com>
References:  <19DB8F4A-6788-44F6-9A2C-E01DEA01BED9@dragondata.com> <CAJjvXiE%2B8OMu_yvdRAsWugH7W=fhFW7bicOLLyjEn8YrgvCwiw@mail.gmail.com> <F4420A8C-FB92-4771-B261-6C47A736CF7F@dragondata.com>

next in thread | previous in thread | raw e-mail | index | archive | help

--envbJBWh7q8WU6mo
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2013-Jan-29 18:06:01 -0600, Kevin Day <toasty@dragondata.com> wrote:
>On Jan 29, 2013, at 5:42 PM, Matthew Ahrens <mahrens@delphix.com> wrote:
>> On Tue, Jan 29, 2013 at 3:20 PM, Kevin Day <toasty@dragondata.com> wrote:
>> I'm prepared to try an L2arc cache device (with secondarycache=3Dmetadat=
a),
>>=20
>> You might first see how long it takes when everything is cached.  E.g. b=
y doing this in the same directory several times.  This will give you a low=
er bound on the time it will take (or put another way, an upper bound on th=
e improvement available from a cache device).
>> =20
>
>Doing it twice back-to-back makes a bit of difference but it's still slow =
either way.

ZFS can very conservative about caching data and twice might not be enough.
I suggest you try 8-10 times, or until the time stops reducing.

>I think some of the issue is that nothing is being allowed to stay cached =
long.

Well ZFS doesn't do any time-based eviction so if things aren't
staying in the cache, it's because they are being evicted by things
that ZFS considers more deserving.

Looking at the zfs-stats you posted, it looks like your workload has
very low locality of reference (the data hitrate is very) low.  If
this is not what you expect then you need more RAM.  OTOH, your
vfs.zfs.arc_meta_used being above vfs.zfs.arc_meta_limit suggests that
ZFS really wants to cache more metadata (by default ZFS has a 25%
metadata, 75% data split in ARC to prevent metadata caching starving
data caching).  I would go even further than the 50:50 split suggested
later and try 75:25 (ie, triple the current vfs.zfs.arc_meta_limit).

Note that if there is basically no locality of reference in your
workload (as I suspect), you can even turn off data caching for
specific filesystems with zfs set primarycache=3Dmetadata tank/foo
(note that you still need to increase vfs.zfs.arc_meta_limit to
allow ZFS to use the the ARC to cache metadata).

--=20
Peter Jeremy

--envbJBWh7q8WU6mo
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)

iEYEARECAAYFAlEMFmAACgkQ/opHv/APuIecWACgn5H+MWNyBmOSD6dCkZOrkIF7
mUgAn0tVC7elSQq2Z22FqQ5/wNi+0Fvn
=u4yZ
-----END PGP SIGNATURE-----

--envbJBWh7q8WU6mo--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130201192416.GA76461>