From owner-freebsd-fs@FreeBSD.ORG Mon Oct 13 10:14:20 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DC4B1065687 for ; Mon, 13 Oct 2008 10:14:20 +0000 (UTC) (envelope-from snb@moduli.net) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170]) by mx1.freebsd.org (Postfix) with ESMTP id 0C9118FC1D for ; Mon, 13 Oct 2008 10:14:19 +0000 (UTC) (envelope-from snb@moduli.net) Received: by ug-out-1314.google.com with SMTP id m2so406979uge.39 for ; Mon, 13 Oct 2008 03:14:18 -0700 (PDT) Received: by 10.103.207.18 with SMTP id j18mr3415025muq.21.1223891580203; Mon, 13 Oct 2008 02:53:00 -0700 (PDT) Received: by 10.102.228.7 with HTTP; Mon, 13 Oct 2008 02:53:00 -0700 (PDT) Message-ID: Date: Mon, 13 Oct 2008 11:53:00 +0200 From: "Nick Barkas" To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: CFT: vm_lowmem event handler patch for dirhash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2008 10:14:20 -0000 Hello These past few months I worked with David Malone on a Google summer of code project for allocating memory for dirhash dynamically. The hope is that this will allow more memory to go to dirhashes on systems that have memory to spare, so that performance working with large directories can be improved. We decided to actually keep the current memory allocation scheme for dirhash unchanged, but I added a vm_lowmem event handler (see EVENTHANDLER(9)) so that older dirhashes will be deleted when the kernel signals that it needs more memory. This should allow vfs.ufs.dirhash_maxmem to be increased quite a bit above its current 2MB default. I have patches against FreeBSD 7-STABLE (I've only tested this one on 7.0-RELEASE) here: http://www.nada.kth.se/~barkas/dirhash_lowmem_7-stable_2008-8-14.patch and 8-CURRENT here: http://www.nada.kth.se/~barkas/dirhash_lowmem_head_2008-10-12.patch Please try these out if you can, and let me know if you see any performance benefits! I've only tested this code with some benchmark scripts, so I am very interested to see how it does under real workloads. Note that the patches do not yet change the default vfs.ufs.dirhash_maxmem because I haven't figured out what would be a new reasonable default yet. To allow for more than 2MB of dirhashes you'll need to set that yourself. In most of my testing on a system with 1GB of memory, I set dirhash_maxmem to 64MB. This seems to be more than enough space to fit, for example, dirhashes for a million email messages in maildirs. There are some other parameters that one could tune and potentially achieve better performance gains. The new vfs.ufs.dirhash_reclaimage sysctl sets the number of seconds dirhashes can remain unused before a vm_lowmem event will unconditionally delete them. The default of 5s works reasonably well in all my tests, although it is somewhat workload dependent. If you change this value and see different performance under your workload, I would definitely like to hear about it. For more information and a bunch of graphs with results from my benchmarking, take a look at http://wiki.freebsd.org/DirhashDynamicMemory. Also, I'll be giving a talk about this project quite soon now at EuroBSDCon 2008. Nick