From owner-freebsd-current Fri Sep 28 1: 0:14 2001 Delivered-To: freebsd-current@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 6F65F37B403 for ; Fri, 28 Sep 2001 00:59:59 -0700 (PDT) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id f8S7wTi34939; Fri, 28 Sep 2001 10:58:29 +0300 (EEST) (envelope-from ru) Date: Fri, 28 Sep 2001 10:58:29 +0300 From: Ruslan Ermilov To: Poul-Henning Kamp Cc: Julian Elischer , current@FreeBSD.ORG Subject: Re: RFC: mod for 'du' Message-ID: <20010928105829.F30062@sunbay.com> References: <3BB3936A.D1E1F6B3@vicor-nb.com> <69645.1001658209@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <69645.1001658209@critter>; from phk@critter.freebsd.dk on Fri, Sep 28, 2001 at 08:23:29AM +0200 Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG We even have a working hcreate(3) and friends in 5.0-CURRENT, that could probably be tried as well. Will MFC shortly. On Fri, Sep 28, 2001 at 08:23:29AM +0200, Poul-Henning Kamp wrote: > In message <3BB3936A.D1E1F6B3@vicor-nb.com>, Julian Elischer writes: > >This is a multi-part message in MIME format. > >--------------5937F02517145C712E28C299 > >Content-Type: text/plain; charset=us-ascii > >Content-Transfer-Encoding: 7bit > > > >'du' keeps an array of files it has encountered that have > 1 link. > >Whenever it encounters another, it checks to see if it's one it has > >already seen > >and thus can avoid counting its space twice.. > > > >This is ok for small filesystems, however VICOR maintains > >500GB filesystems on which much of the data has several links. > > > >The following patch to replace the linear array (which it realocs if too > >small) > >(which it scans linearly) with a hash-table can makle a DRASTIC change > >to how DU perfomrs for us in this environment. > > > >In a small test, we made a linked copy of /usr/src > > > >the run times were: > >old: 0.410u 2.221s 1:55.41 2.2% 12+1355k 6325+0io 2pf+0w > >new: 8.610u 2.665s 2:09.23 8.7% 10+718k 6367+0io 2pf+0 > > Have you examined if the hask key is evenly distributed ? > > >Does anyone have objections to me committing this (or a variant of it)? > > I would certainly have objections of you show us one patch and > commit another.. > > > * The Regents of the University of California. All rights reserved. > > * > >+ * This version of du has been modified by Andre de Bruin and Scott Macy for Vicor. > > Too long line ? > > >+ * The change is related to the handling of file links, in the official > >+ * release, du keeps a simple linked list of all visited i-nodes with > >+ * multiple links. This list is now implemented as a "hash table" (actually > >+ * a fixed size array) with link list nodes providing a > >+ * performance of up to 30% better than the original version. > >+ * The only changes are in file create.c and marked with "AdB". > > We put such stuff in the CVS commit message, not in the source file. > > >+typedef struct _ID { > >+ dev_t dev; > >+ ino_t inode; > >+ struct _ID *next; > >+} ID; > > Why not use db(3) ? > > It has a known good hash, it autosizes to different task-sets and > it could be setup with an option to put the hash-table in a diskfile > ("du -t" ?) for known bad cases of many hardlinks -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message