From owner-freebsd-fs@FreeBSD.ORG  Wed Jan 30 16:34:48 2013
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 8D63B27E
 for <freebsd-fs@freebsd.org>; Wed, 30 Jan 2013 16:34:48 +0000 (UTC)
 (envelope-from ndenev@gmail.com)
Received: from mail-bk0-f51.google.com (mail-bk0-f51.google.com
 [209.85.214.51]) by mx1.freebsd.org (Postfix) with ESMTP id E7728774
 for <freebsd-fs@freebsd.org>; Wed, 30 Jan 2013 16:34:47 +0000 (UTC)
Received: by mail-bk0-f51.google.com with SMTP id ik5so922431bkc.38
 for <freebsd-fs@freebsd.org>; Wed, 30 Jan 2013 08:34:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=x-received:subject:mime-version:content-type:from:in-reply-to:date
 :cc:content-transfer-encoding:message-id:references:to:x-mailer;
 bh=8RTCp/ASRpz9O9eFd1VM9MVuj+AlGo/ex9Alocsrpy0=;
 b=P0iibY5ikgCzzIMWXojRZpi357R608ZbSCwecaIvvR6Z6TO6BzbzMfJOkGq6HBKKM0
 SwaMEO/Q0T1U1nEkWz5ujhv+5GzuO2dG0vWcoPcq+JptER9vrgTBttVEjakDLp8vI/BH
 YuThDDe7qcw8MKSeXNrmPqXDM6bblgDlL728ZKAnqqUhfGRmU920keaNDAmIyHtstEfO
 7lLulsLYq/Sl4foceTHANPoEG7Zc0o0jvx8NuU32ewdA/lIfzWBlgha5vozRTo16vCH7
 0D3dO5jt4UVrfkAZSBjIsxhDzhtwQwnV7hn7wnzACBdggU9zkmg/aWbJo/r2IlCSWo2p
 NeaA==
X-Received: by 10.204.11.78 with SMTP id s14mr1431340bks.118.1359563681583;
 Wed, 30 Jan 2013 08:34:41 -0800 (PST)
Received: from imba-brutale-3.totalterror.net ([93.152.184.10])
 by mx.google.com with ESMTPS id z5sm819545bkv.11.2013.01.30.08.34.39
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Wed, 30 Jan 2013 08:34:40 -0800 (PST)
Subject: Re: Improving ZFS performance for large directories
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Content-Type: text/plain; charset=windows-1252
From: Nikolay Denev <ndenev@gmail.com>
In-Reply-To: <47975CEB-EA50-4F6C-8C47-6F32312F34C4@dragondata.com>
Date: Wed, 30 Jan 2013 18:34:41 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <23E6691A-F30C-4731-9F78-FD8ADDDA09AE@gmail.com>
References: <19DB8F4A-6788-44F6-9A2C-E01DEA01BED9@dragondata.com>
 <5267B97C-ED47-4AAB-8415-12D6987E9371@gmail.com>
 <47975CEB-EA50-4F6C-8C47-6F32312F34C4@dragondata.com>
To: Kevin Day <toasty@dragondata.com>
X-Mailer: Apple Mail (2.1499)
Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org>
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Jan 2013 16:34:48 -0000

On Jan 30, 2013, at 5:19 PM, Kevin Day <toasty@dragondata.com> wrote:
>=20
> On Jan 30, 2013, at 4:36 AM, Nikolay Denev <ndenev@gmail.com> wrote:
>>=20
>>=20
>> What are your : vfs.zfs.arc_meta_limit and vfs.zfs.arc_meta_used =
sysctls?
>> Maybe increasing the limit can help?
>=20
>=20
> vfs.zfs.arc_meta_limit: 8199079936
> vfs.zfs.arc_meta_used: 13965744408
>=20
> Full output of zfs-stats: [=85snipped=85]

Looks like you can try to increase arc_meta_limit to be let's say : half =
of arc_max. (16398159872 in your case).