From owner-freebsd-current@FreeBSD.ORG Thu Mar 10 17:12:15 2005 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2D35B16A4CF for ; Thu, 10 Mar 2005 17:12:15 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 90FDB43D49 for ; Thu, 10 Mar 2005 17:12:14 +0000 (GMT) (envelope-from peadar.edwards@gmail.com) Received: by wproxy.gmail.com with SMTP id 37so510916wra for ; Thu, 10 Mar 2005 09:12:14 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=YyMSQJJGRHLJfk6Zmbq8DcVHPXgWmPNgS2u+odT5BCwIA+LxkRnuBna7AQMLU14mp1HLq4OqYMBFpMvv4wzWILhwTMqLhywZFVwdli1zjUQYRggCzBEAyBMJvHq9Fa68IvXLSk3jgXh7NQHQwYVY44kU9hk90ft01EE5brNgKHU= Received: by 10.54.45.14 with SMTP id s14mr1921133wrs; Thu, 10 Mar 2005 09:12:14 -0800 (PST) Received: by 10.54.57.20 with HTTP; Thu, 10 Mar 2005 09:12:14 -0800 (PST) Message-ID: <34cb7c84050310091243cdc342@mail.gmail.com> Date: Thu, 10 Mar 2005 17:12:14 +0000 From: Peter Edwards To: Sergey Matveychuk In-Reply-To: <423027B1.8080503@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <87is46kzk1.fsf@neva.vlink.ru> <41C26F23F7DF023CB3DF35C5@cc-171.int.t-online.fr> <20050305151903.GC26240@hub.freebsd.org> <87mzth18e2.fsf@neva.vlink.ru> <1DE178D508C1D70D1B5F9E87@cc-171.int.t-online.fr> <874qfpupk5.fsf@neva.vlink.ru> <423027B1.8080503@FreeBSD.org> cc: Denis Shaposhnikov cc: freebsd-current@freebsd.org cc: Mathieu Arnold Subject: Re: unionfs 5.4 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Peter Edwards List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Mar 2005 17:12:15 -0000 > I've found nullfs toooo slow in jail. There seems to be a common consensus that nullfs is a bit of a tortoise. Stab in the dark: I haven't looked at this in great detail, but the very top of nullfs_subr .c defines the size of a hashtables used for storing all the null nodes into. It's only 16 buckets. That seems awfully under provisioned to me: there might be tens of thousands of "underlying" vnodes, giving a best-case scenario of thousands of iterations over one of these buckets to find an upper "null" node. Even the hash function seems pretty ropey... unionfs has a similar hash table (32 buckets instead of 16, and separate locks on each bucket) but it might not invoke the searches as often: I'm not as familliar with its design. If someone experiencing these issues could try rebuilding after changing #define NNULLNODECACHE 16 to #define NNULLNODECACHE 1024 in null_subr.c (line 50 in my version, 1.45) And just see if it makes much of a difference. If so, it might be worth making more flexible. (Sizing proportionally to maxvnodes or something, as UFS does, or replacing with a more scalable data structure.)