From owner-freebsd-current@FreeBSD.ORG Mon Sep 27 17:19:16 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CA86816A4CE for ; Mon, 27 Sep 2004 17:19:16 +0000 (GMT) Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB3C243D41 for ; Mon, 27 Sep 2004 17:19:15 +0000 (GMT) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.1/8.13.1) with ESMTP id i8RHIsrP032923; Mon, 27 Sep 2004 10:18:59 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <200409271718.i8RHIsrP032923@gw.catspoiler.org> Date: Mon, 27 Sep 2004 10:18:54 -0700 (PDT) From: Don Lewis To: scrappy@hub.org In-Reply-To: <20040924160535.N43964@ganymede.hub.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii cc: freebsd-current@FreeBSD.org cc: keramida@linux.gr Subject: Re: what is fsck's "slowdown"? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2004 17:19:16 -0000 On 24 Sep, Marc G. Fournier wrote: > > Just curious as to whether or not this is going to get applied to the > source tree ... ? Just checked my -current, and it isn't there yet, just > wanted to see if maybe its been forgotten? :( There are a couple of tweaks that I want to do. My plan is to re-post the patch to current@ for further testing before I commit it. After it has had time to be sufficiently exercised in -CURRENT, I'll MFC it to RELENG_5 (after 5.3-RELEASE), and RELENG_4. The feedback I've gotten from those who reviewed the patch has been positive, but I want to take it slow because of the critical nature of fsck. > On Sun, 5 Sep 2004, Don Lewis wrote: > >> On 4 Sep, To: scrappy@hub.org wrote: >>> On 4 Sep, Don Lewis wrote: >>>> On 4 Sep, Marc G. Fournier wrote: >>>>> On Fri, 3 Sep 2004, Don Lewis wrote: >>>> >>>>>> Would the file system in question happen to be full of UNREF files that >>>>>> fsck is deleting? >>>>> >>>>> mostly 'ZERO LENGTH DIRECTORY' ... >>>> >>>> I'm pretty sure that I understand the problem now. During pass 4, fsck >>>> looks at each inode. It checks each inode in the FSTATE and DFOUND >>>> states to see if their link counts need to be adjusted. If the link >>>> count does not need to be adjusted, fsck checks to see if the inode is >>>> on the list of inodes whose initial link counts were zero, and if it >>>> finds the inode on this list, it clears the inode. >>>> >>>> The problem is that the zero length directories get added to this list >>>> if their initial link count is zero, and they also don't get removed >>>> from the list because they are in the DCLEAR state, so the list doesn't >>>> shrink. This bloats the list, which greatly slows down processing of >>>> normal files and directories. >>>> >>>> Deleting unreferenced files is not the biggest bottleneck, so reversing >>>> the order of the list isn't going to help much. Probably the biggest >>>> speedup could be gained by keeping the zero length directories off the >>>> list. >>> >>> An even better solution would be to dispense with the zln list entirely >>> and just set a bit for these inodes in their struct inostat. This >>> change is a bit more intrusive because of the need for some sort of >>> packing strategy because of the need to keep this structure small. My >>> initial inclination would be to add two new states, FZERO and DZERO, >>> that pass1() would use to mark inodes with a zero link count. >> >> Here's a patch that eliminates the zln list and adds two new inode >> states to tag zero files and directories that have a zero link count. It >> seems to work in the light testing that I have done, but it needs >> further and review before it gets committed. >> [snip]