From owner-freebsd-hackers Sun Aug 17 19:34:40 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.5/8.8.5) id TAA13792 for hackers-outgoing; Sun, 17 Aug 1997 19:34:40 -0700 (PDT) Received: from dg-rtp.dg.com (dg-rtp.rtp.dg.com [128.222.1.2]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id TAA13785 for ; Sun, 17 Aug 1997 19:34:34 -0700 (PDT) Received: by dg-rtp.dg.com (5.4R3.10/dg-rtp-v02) id AA28157; Sun, 17 Aug 1997 21:50:11 -0400 Received: from ponds by dg-rtp.dg.com.rtp.dg.com; Sun, 17 Aug 1997 21:50 EDT Received: from lakes.dignus.com (lakes [10.0.0.3]) by ponds.dignus.com (8.8.5/8.7.3) with ESMTP id VAA00310; Sun, 17 Aug 1997 21:58:15 -0400 (EDT) Received: (from root@localhost) by lakes.dignus.com (8.8.5/8.6.9) id VAA03041; Sun, 17 Aug 1997 21:12:38 -0400 (EDT) Date: Sun, 17 Aug 1997 21:12:38 -0400 (EDT) From: Thomas David Rivers Message-Id: <199708180112.VAA03041@lakes.dignus.com> To: ponds!freefall.cdrom.com!freebsd-hackers, ponds!lakes.dignus.com!rivers Subject: "rm" speeds (2.1.7.1 vs. 2.2.1) Cc: ponds!root.com!dg Content-Type: text Sender: owner-freebsd-hackers@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Ok - I've been spending the afternoon verifying my findings on 2.1.7.1 "rm times" vs. 2.2.1 "rm times." Here's what I've done so far: 1) I have another large directory (well, it's the same one, "control" from my news spool - 1061888 bytes of directory space with 42881 entries.. 2) I tar'd that up and copied it to another directory, to determine just how many "empty" entries there were in the directory. About 1/2 of them are empty. (that is, the directory size when everything is untarr'd is about 1/2 the original size.) 3) I booted 2.1.7 and used the "fixit" floppy to remove some files. I noticed that the disk activity was rapid, and the files were deleted fairly quickly. About 10,000 files took less than 5 minutes (wall time.) I was trying removing files that occur later in the directory; so as to incur the delay associated with skipping most of the entries. "Later" was by timestamp, which may not actually be later in the entry list, perhaps I should have done this by inode. 4) I booted 2.2.1, into single user mode to avoid any potential issue there - the directory size was unchanged, so no coalescing of directory entries had occured when I deleted files using 2.1.7. Using the same directory with the remaining files; I attempted to delete about the same number of files; starting toward the beginning of the directory (by time stamp) to give 2.2.1 "an edge." The disk activity has a markedly different pattern - much more bursty, as if each delete is causing a flush of some kind. After 45 minutes of trying to delete ~10,000 files - I gave up. 5) I then rebooted 2.1.7 and tried to delete the same ~10,000 files (which I presume is somewhat less than 10,000 - but I didn't actually count them) - less than 5 minutes, they were gone. From this, I'm guessing that 2.2.1 is doing something different with the directory entries - causing a lot more I/O per deletion, which is a serious drag on performance. Does anyone have a notion about why 2.2.1 would be doing more I/O? Is it writing pieces of the directory back when 2.1.7 didn't? [By the way, these were all synchronous mounts - my previous experiments verified that mounting asynchronously with 2.2.1 didn't improve the unlink performance.] I did start looking into unlink() - it really hasn't changed for 2.2.1 (vs. 2.1.7), at least the code paths would be the same. I also tried running the 2.1.7 "rm" command on 2.2.1 (the one from /stand on the 2.1.7 boot floppy) to ensure that the "rm" command hadn't somehow changed - I got the same less-than-stellar performance. Again, f.y.i. - this is a 486dx66 with 24meg of RAM, a typical IDE drive (1+gig)... The 2.2.1 kernel has NBUF defined at 128; to see if that's the problem... where the 2.1.7 kernel was from the boot floppy off of a 2.1.7 CDROM. - Opinions? - - Dave Rivers -