From owner-freebsd-performance@FreeBSD.ORG  Thu Dec 20 16:56:50 2007
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: freebsd-performance@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7F03616A417
	for <freebsd-performance@freebsd.org>;
	Thu, 20 Dec 2007 16:56:50 +0000 (UTC) (envelope-from _pppp@mail.ru)
Received: from mx28.mail.ru (mx28.mail.ru [194.67.23.67])
	by mx1.freebsd.org (Postfix) with ESMTP id 2F9EA13C458
	for <freebsd-performance@freebsd.org>;
	Thu, 20 Dec 2007 16:56:49 +0000 (UTC) (envelope-from _pppp@mail.ru)
Received: from f156.mail.ru (f156.mail.ru [194.67.57.217])
	by mx28.mail.ru (mPOP.Fallback_MX) with ESMTP id 25DD9772DB5
	for <freebsd-performance@freebsd.org>;
	Thu, 20 Dec 2007 19:41:16 +0300 (MSK)
Received: from mail by f156.mail.ru with local 
	id 1J5OSL-000Fl1-00; Thu, 20 Dec 2007 19:41:09 +0300
Received: from [89.208.20.114] by koi.mail.ru with HTTP;
	Thu, 20 Dec 2007 19:41:09 +0300
From: dima <_pppp@mail.ru>
To: Alexandre Biancalana <biancalana@gmail.com>
Mime-Version: 1.0
X-Mailer: mPOP Web-Mail 2.19
X-Originating-IP: [89.208.20.114]
Date: Thu, 20 Dec 2007 19:41:09 +0300
References: <8e10486b0712191207r5e256b84xf6cb0bbafc6cfd20@mail.gmail.com>
In-Reply-To: <8e10486b0712191207r5e256b84xf6cb0bbafc6cfd20@mail.gmail.com>
Content-Type: text/plain; charset=koi8-r
Content-Transfer-Encoding: 8bit
Message-Id: <E1J5OSL-000Fl1-00._pppp-mail-ru@f156.mail.ru>
Cc: freebsd-performance@freebsd.org, Mike Tancsa <mike@sentex.net>
Subject: Re[2]: Bad performance when accessing a lot of small files
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: dima <_pppp@mail.ru>
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Dec 2007 16:56:50 -0000

>>>The behavior that I'm observing and that want your help is when the
>>>system is accessing some directory with many small files ( directories
>>>with ~ 1 million of ~30kb files), the performance is very poor.
>>
>> Hi,
> 
> I'm using zfs, I think this change the things.. no ?
> 
> >          Have you adjusted the dirhash value ? What does
> >
> >   sysctl -a vfs.ufs | grep dirhash
> 
> # sysctl -a vfs.ufs | grep dirhash
> vfs.ufs.dirhash_docheck: 0
> vfs.ufs.dirhash_mem: 1410338
> vfs.ufs.dirhash_maxmem: 2097152
> vfs.ufs.dirhash_minsize: 2560

I agree with Mike.

Try to run this command while listing your large directory. You'll see that dirhash_mem will reach dirhash_maxmem. So, the value of dirhash_maxmem should be increased.

One more hint. Mounting filesystems with -noatime option greatly improves read performance. Apply this also, if you haven't done this yet.