From owner-freebsd-current@FreeBSD.ORG Thu Jul 12 23:39:49 2007 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CD48C16A400 for ; Thu, 12 Jul 2007 23:39:49 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.180]) by mx1.freebsd.org (Postfix) with ESMTP id 642A013C45A for ; Thu, 12 Jul 2007 23:39:49 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: by ik-out-1112.google.com with SMTP id c21so296759ika for ; Thu, 12 Jul 2007 16:39:48 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=BuxmsIilUQlm4g3TyTvvSFPQ0+nq/DU+wFj6i+x9/BL3MenlkXqL3SnQJxhJ9CQFBbcxRBtvl6OfGxfbCFv6j7flziZe/IjfkoAJ4oRKKLJ4y6S+yagzA9lpqz7FFQEhd7Ni6xiYHz5/yHBt1x+yoPHjelb5AI+EDKMRPtM/+jA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=ZfI+cqO4KTGkbWO32tV02upqeYX4wrrTgZHK5tF/0YO6RYzAY9sB2HQeCZVdE/sAqt6P0sbivwIXchVemZ6ATRnB5bkahXNDkgHTYdodpOmb2tyqCFHkMA5veSqq8LY8b85affFWRyipJDfqfaHxJXt+rfUDtcrrmg2VeF6GQ38= Received: by 10.78.142.14 with SMTP id p14mr335926hud.1184281848759; Thu, 12 Jul 2007 16:10:48 -0700 (PDT) Received: by 10.78.187.16 with HTTP; Thu, 12 Jul 2007 16:10:48 -0700 (PDT) Message-ID: <70e8236f0707121610v65bacaa0pcaf45c62516ab424@mail.gmail.com> Date: Fri, 13 Jul 2007 00:10:48 +0100 From: "Joao Barros" To: "Pawel Jakub Dawidek" In-Reply-To: <20070709000918.GD1208@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200707071426.18202.dfr@rabson.org> <20070709000918.GD1208@garage.freebsd.pl> Cc: current@freebsd.org Subject: Re: ZFS leaking vnodes (sort of) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2007 23:39:49 -0000 On 7/9/07, Pawel Jakub Dawidek wrote: > On Sat, Jul 07, 2007 at 02:26:17PM +0100, Doug Rabson wrote: > > I've been testing ZFS recently and I noticed some performance issues > > while doing large-scale port builds on a ZFS mounted /usr/ports tree. > > Eventually I realised that virtually nothing ever ended up on the vnode > > free list. This meant that when the system reached its maximum vnode > > limit, it had to resort to reclaiming vnodes from the various > > filesystem's active vnode lists (via vlrureclaim). Since those lists > > are not sorted in LRU order, this led to pessimal cache performance > > after the system got into that state. > > > > I looked a bit closer at the ZFS code and poked around with DDB and I > > think the problem was caused by a couple of extraneous calls to vhold > > when creating a new ZFS vnode. On FreeBSD, getnewvnode returns a vnode > > which is already held (not on the free list) so there is no need to > > call vhold again. > > Whoa! Nice catch... The patch works here - I did some pretty heavy > tests, so please commit it ASAP. > > I also wonder if this can help with some of those 'kmem_map too small' > panics. I was observing that ARC cannot reclaim memory and this may be > because all vnodes and thus associated data are beeing held. > > To ZFS users having problems with performance and/or stability of ZFS: > Can you test the patch and see if it helps? > I've recompiled my system after Doug committed this patch 3 days ago and I can panic my machine as soon as I don't set kern.maxvnodes to 50000 while doing a ls -R after a recursive chown on some thousands of files and dirs: panic: kmem_malloc(16384): kmem_map too small: 326066176 total allocated I noticed that before this patch the system panicked very easily and soon in the chown process. Now it completes the chown on the thousands of files and dirs I have in it and only panics after in the ls -R process. It's an improvement, but something else is still there... -- Joao Barros