From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 15:15:10 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 799B41B8 for ; Wed, 30 Jan 2013 15:15:10 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-ie0-x22c.google.com (ie-in-x022c.1e100.net [IPv6:2607:f8b0:4001:c03::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 490C9F94 for ; Wed, 30 Jan 2013 15:15:10 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id c10so1338113ieb.17 for ; Wed, 30 Jan 2013 07:15:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer; bh=4jStnLPiLJlFlXWlcVd2u39urNx9ecuHBzR1WKnGeAY=; b=duvti6SSYFdBAFGyj87Qf3PSjr9+9PgNgdCCDIVygZjzzEunXnEYmsQum1cwd6iHmS hxaifO0JuzuzVGzG3suqMY+pYVVrvhXeCBQEUP07lo6YxguAMFciAqerdqFf3rBGwXda 1ugktxPbzKf/p3np0p0Hsakib/9Uf4SZzGjUo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=4jStnLPiLJlFlXWlcVd2u39urNx9ecuHBzR1WKnGeAY=; b=oiSUN8jI/7GqNIRyYkCUIoLAyygk/qwDOYYkAM2hsjW/smXQaqv9d1EowrzQYJ7sOK 9oxQaGph4jiSpE9xDXliaiikZRI8ht2XjGnT+7dGny1Cd5imThT8Dxl/WnvZEiXANAzz lnxV+uM3S2DvfC+TtDgHX8B6UeT6fTfNUdkW8Iylylmd/ohX/XekYTQmk5Z4r9ZuAOMq /DiWoJXuj2A4DCuauu7kBOndTGXDmhkg0PNRRHc81c21jVjbceCLu8bCLDZW3H+KJLtX VKgeTfh8Sx8Jo42fhlVCCzar3ZAvablXw6b95w1hxWcVYSxzf0E/01hyD1A1FVXXJG3c thrg== X-Received: by 10.50.47.200 with SMTP id f8mr3951203ign.98.1359558909991; Wed, 30 Jan 2013 07:15:09 -0800 (PST) Received: from vpn132.rw1.your.org (vpn132.rw1.your.org. [204.9.51.132]) by mx.google.com with ESMTPS id bg10sm4495947igc.6.2013.01.30.07.15.07 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Jan 2013 07:15:08 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Improving ZFS performance for large directories From: Kevin Day In-Reply-To: Date: Wed, 30 Jan 2013 09:15:04 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <19DB8F4A-6788-44F6-9A2C-E01DEA01BED9@dragondata.com> To: "Ronald Klop" X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQkruxYgS03kacjD54zkMx4V3qRKoiNaTXmQywWTjh4H6FRcoDDHNGsh5gUccTzxo4AX8ktz Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2013 15:15:10 -0000 On Jan 30, 2013, at 4:20 AM, "Ronald Klop" = wrote: > On Wed, 30 Jan 2013 00:20:15 +0100, Kevin Day = wrote: >=20 >>=20 >> I'm trying to improve performance when using ZFS in large (>60000 = files) directories. A common activity is to use "getdirentries" to = enumerate all the files in the directory, then "lstat" on each one to = get information about it. Doing an "ls -l" in a large directory like = this can take 10-30 seconds to complete. Trying to figure out why, I = did: >>=20 >> ktrace ls -l /path/to/large/directory >> kdump -R |sort -rn |more >=20 > Does ls -lf /pat/to/large/directory make a difference. It makes ls not = to sort the directory so it can use a more efficient way of traversing = the directory. >=20 > Ronald. Nope, the sort seems to add a trivial amount of extra time to the entire = operation. Nearly all the time is spent in lstat() or getdirentries(). = Good idea though! -- Kevin