From owner-freebsd-performance@FreeBSD.ORG Fri Dec 21 20:49:58 2007 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6CE616A421 for ; Fri, 21 Dec 2007 20:49:58 +0000 (UTC) (envelope-from biancalana@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.181]) by mx1.freebsd.org (Postfix) with ESMTP id 973DC13C4F2 for ; Fri, 21 Dec 2007 20:49:58 +0000 (UTC) (envelope-from biancalana@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so765097waf.3 for ; Fri, 21 Dec 2007 12:49:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=cbQGs15tXf1sOU2nastfwyfS2g2C43mgadUJiVcwTl4=; b=YGxOfSnAC9ctksj+o8Fqam0h81jrMUmAUNSH6QuXzEbzm46SGkVfUJmocsWEWn04op2qi/J3ArbLcJgqFfuX+Voybuj2J/TgHmx6XATqLvzz0Yua//j8FbYO7H3DudYKSJo3isSLb9rK3wKoYa551qwODJbgiwqCpVETtvm1Reo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=LvMpoDp9Jt6k3FTVBIP+GYi3dIXxbG4/1+EngfjGfuIrsfX8dnP0ZhU339mvAiWGeAnI4iuOqqujr6JqsvHvzuZrNztNNlBCufgCFQJ5zEdhH0lyDheY/L9klcJNXtY4BMSDd5Gek0F5ngINWeDhP47iANtUHteKlAJ4e94XqRc= Received: by 10.114.210.2 with SMTP id i2mr661090wag.36.1198270198099; Fri, 21 Dec 2007 12:49:58 -0800 (PST) Received: by 10.114.27.7 with HTTP; Fri, 21 Dec 2007 12:49:58 -0800 (PST) Message-ID: <8e10486b0712211249v4c5571ddud21b277f686992b2@mail.gmail.com> Date: Fri, 21 Dec 2007 17:49:58 -0300 From: "Alexandre Biancalana" To: "Alfred Perlstein" In-Reply-To: <20071221201625.GZ16982@elvis.mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <8e10486b0712191109n3d21b02cyf5183ee0cd01d8ce@mail.gmail.com> <20071221201625.GZ16982@elvis.mu.org> Cc: freebsd-performance@freebsd.org Subject: Re: Bad performance when accessing a lot of small files X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Dec 2007 20:49:58 -0000 On 12/21/07, Alfred Perlstein wrote: Hi Alfred ! > > There is a lot of very good tuning advice in this thread, however > one thing to note is that having ~1 million files in a directory > is not a very good thing to do on just about any filesystem. I think I was not clear, I will try explain better. This Backup Server has a /backup zfs filesystem of 4TB. Each host that do backups to this server has a /backup/ and /backup//YYYYMMDD zfs filesystems, the last contains the backups for some day of that server. My problem is with some hosts that have in your directory structure a lot of small files, independent of the hierarchy. > > One trick that a lot of people do is hashing the directories themselves > so that you use some kind of computation to break this huge dir into > multiple smaller dirs. I have the two cases, when you have a lot of files inside on directory without any directory organization/distribution but I also have problems with hosts that have files organized in a hierarchy like YYYY/MM/DD/ having no more that 200 files in the day directory level, but almost one million of files in total. Just for info, I made the previous suggested tuning (raise dirhash, maxvnodes) but this improve nothing. Thanks for your hint!