From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 08:51:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEB08106564A for ; Tue, 2 Aug 2011 08:51:36 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out1.tiscali.nl (smtp-out1.tiscali.nl [195.241.79.176]) by mx1.freebsd.org (Postfix) with ESMTP id 6BAD78FC08 for ; Tue, 2 Aug 2011 08:51:36 +0000 (UTC) Received: from [212.182.167.131] (helo=sjakie.klop.ws) by smtp-out1.tiscali.nl with esmtp (Exim) (envelope-from ) id 1QoALa-0002q0-T0; Tue, 02 Aug 2011 10:29:06 +0200 Received: from 212-182-167-131.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id B07E34182; Tue, 2 Aug 2011 10:29:02 +0200 (CEST) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-stable@freebsd.org, "seanrees@gmail.com" References: Date: Tue, 02 Aug 2011 10:29:02 +0200 MIME-Version: 1.0 From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/11.50 (FreeBSD) Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 08:51:36 -0000 Not an in depth solution for ZFS, but maybe a solution for you. mkdir images2 mv images/* images2 rmdir images Ronald. On Tue, 02 Aug 2011 09:39:03 +0200, seanrees@gmail.com =20 wrote: > Hi there, > > I Googled around and checked the PRs and wasn't successful in finding > any reports of what I'm seeing. I'm hoping someone here can help me > debug what's going on. > > On my FreeBSD 8.2-S machine (built circa 12th June), I created a > directory and populated it over the course of 3 weeks with about 2 > million individual files. As you might imagine, a 'ls' of this > directory took quite some time. > > The files were conveniently named with a timestamp in the filename > (still images from a security camera, once per second) so I've since > moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I > found though was the original directory the images were in is still > very slow to ls -- and it only has 1 file in it, another directory. > > To clarify: > % ls second > [lots of time and many many files enumerated] > % # rename files using rename script > % ls second > [wait ages] > 2011 dead > % mkdir second2 && mv second/2011 second2 > % ls second2 > [fast!] > 2011 > % ls second > [still very slow] > dead > % time ls second > dead/ > gls -F --color 0.00s user 1.56s system 0% cpu 3:09.61 total > > (timings are similar for /bin/ls) > > This data is stored on a striped ZFS pool (version 15, though the > kernel reports version 28 is available but zpool upgrade seems to > disagree), 2T in size. I've run zpool scrub with no effect. ZFS is > busily driving the disks away; my iostat monitoring has all three > drives in the zpool running at 40-60% busy for the duration of the ls > (it was quiet before). > > I've attached truss to the ls process. It spends a lot of time here: > fstatfs(0x5,0x7fffffffe0d0,0x800ad5548,0x7fffffffdfd8,0x0,0x0) =3D 0 (0= x0) > > I'm thinking there's some old ZFS metadata that it's looking into, but > I'm not sure how to best dig into this to understand what's going on > under the hood. > > Can anyone perhaps point me the right direction on this? > > Thanks, > > Sean > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g"