From owner-freebsd-performance@FreeBSD.ORG  Sat Dec 22 00:27:05 2007
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: freebsd-performance@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 5196B16A468
	for <freebsd-performance@freebsd.org>;
	Sat, 22 Dec 2007 00:27:05 +0000 (UTC)
	(envelope-from bright@elvis.mu.org)
Received: from elvis.mu.org (elvis.mu.org [192.203.228.196])
	by mx1.freebsd.org (Postfix) with ESMTP id 0632313C47E
	for <freebsd-performance@freebsd.org>;
	Sat, 22 Dec 2007 00:27:03 +0000 (UTC)
	(envelope-from bright@elvis.mu.org)
Received: by elvis.mu.org (Postfix, from userid 1192)
	id 928C51A4D7C; Fri, 21 Dec 2007 16:25:35 -0800 (PST)
Date: Fri, 21 Dec 2007 16:25:35 -0800
From: Alfred Perlstein <alfred@freebsd.org>
To: Alexandre Biancalana <biancalana@gmail.com>
Message-ID: <20071222002535.GL16982@elvis.mu.org>
References: <8e10486b0712191109n3d21b02cyf5183ee0cd01d8ce@mail.gmail.com>
	<20071221201625.GZ16982@elvis.mu.org>
	<8e10486b0712211249v4c5571ddud21b277f686992b2@mail.gmail.com>
	<20071221212808.GE16982@elvis.mu.org>
	<8e10486b0712211555n3efe8729qff14387be128cf10@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8e10486b0712211555n3efe8729qff14387be128cf10@mail.gmail.com>
User-Agent: Mutt/1.4.2.3i
Cc: freebsd-performance@freebsd.org
Subject: Re: Bad performance when accessing a lot of small files
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Dec 2007 00:27:05 -0000

* Alexandre Biancalana <biancalana@gmail.com> [071221 15:53] wrote:
> On 12/21/07, Alfred Perlstein <alfred@freebsd.org> wrote:
> >
> > What application are you scanning these files with?  I know I had
> > issues with rsync in particular where I had to have it rsync
> > smaller pieces of a collection for it to work nicely instead of
> > going for the whole heirarchy.
> 
> tar
> 
> I run tar in the /backup/<hostname>/YYYYMMDD writing to LTO3 tape
> drive, the problem is that when origin directory contains a lot of
> small files the process is *much* more slow.... this is my question
> since the thread start.

Have you tried the 'noatime' mount option?  That should help.

Can you provide a histogram of the count of files per directory?

-- 
- Alfred Perlstein