From owner-freebsd-fs@FreeBSD.ORG Thu Jul 21 17:03:54 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 838D2106566B for ; Thu, 21 Jul 2011 17:03:54 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 427778FC12 for ; Thu, 21 Jul 2011 17:03:54 +0000 (UTC) Received: by ywf7 with SMTP id 7so884757ywf.13 for ; Thu, 21 Jul 2011 10:03:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=4B19JgmgUjK/pawyQ0hzh/7ANZ3Njt1avoZPtRu3Bao=; b=Knh72L+ygFvhthwQamjBYGSAzIB2Y+fbwJbIB97q7xQv/X/gNYGGtKp2AH5d8km8wo eYcnyGoGcSMYggliqylsn6HS2JqSORS3ytLvt13+6a1WR1BSOXu9eqbb/UA1umPFgqSI 7kHGK3jXuknYtX/t9NeqHGJbzd7k2zfDLh0hE= Received: by 10.101.180.22 with SMTP id h22mr491681anp.149.1311266323120; Thu, 21 Jul 2011 09:38:43 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.100.198.5 with HTTP; Thu, 21 Jul 2011 09:38:02 -0700 (PDT) In-Reply-To: References: From: Ivan Voras Date: Thu, 21 Jul 2011 18:38:02 +0200 X-Google-Sender-Auth: RIvcVctju_TcNfF67mhlqJX8VOs Message-ID: To: Freddie Cash Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ZFS and large directories - caveat report X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jul 2011 17:03:54 -0000 On 21 July 2011 17:50, Freddie Cash wrote: > On Thu, Jul 21, 2011 at 8:45 AM, Ivan Voras wrote: >> >> Is there an equivalent of UFS dirhash memory setting for ZFS? (i.e. the >> size of the metadata cache) > > vfs.zfs.arc_meta_limit > > This sets the amount of ARC that can be used for metadata.=C2=A0 The defa= ult is > 1/8th of ARC, I believe.=C2=A0 This setting lets you use "primarycache=3D= all" > (store metadata and file data in ARC) but then tune how much is used for > each. > > Not sure if that will help in your case or not, but it's a sysctl you can > play with. I don't think that it works, or at least is not as efficient as dirhash: www:~> sysctl -a | grep meta kern.metadelay: 28 vfs.zfs.mfu_ghost_metadata_lsize: 129082368 vfs.zfs.mfu_metadata_lsize: 116224 vfs.zfs.mru_ghost_metadata_lsize: 113958912 vfs.zfs.mru_metadata_lsize: 16384 vfs.zfs.anon_metadata_lsize: 0 vfs.zfs.arc_meta_limit: 322412800 vfs.zfs.arc_meta_used: 506907792 kstat.zfs.misc.arcstats.demand_metadata_hits: 4471705 kstat.zfs.misc.arcstats.demand_metadata_misses: 2110328 kstat.zfs.misc.arcstats.prefetch_metadata_hits: 27 kstat.zfs.misc.arcstats.prefetch_metadata_misses: 51 arc_meta_used is nearly 500 MB which should be enough even in this case. With filenames of 32 characters, all the filenames alone for 130,000 files in a directory take about 4 MB - I doubt the ZFS introduces so much extra metadata it doesn't fit in 500 MB. I am now deleting the session files, and I hope it will not take days to complete...