From owner-freebsd-questions Fri May 21 4: 4:47 1999 Delivered-To: freebsd-questions@freebsd.org Received: from loviatar.webcom.com (loviatar.webcom.com [209.1.28.41]) by hub.freebsd.org (Postfix) with ESMTP id 9582F14E67 for ; Fri, 21 May 1999 04:04:43 -0700 (PDT) (envelope-from graeme@echidna.com) Received: from kigal.webcom.com (kigal.webcom.com [209.1.28.57]) by loviatar.webcom.com (8.9.1/8.9.1) with SMTP id EAA17542 for ; Fri, 21 May 1999 04:04:42 -0700 Received: from [204.143.69.17] by inanna.webcom.com (WebCom SMTP 1.2.1) with SMTP id 27988926; Fri May 21 04:00 PDT 1999 Message-Id: <374567E9.4854@echidna.com> Date: Fri, 21 May 1999 07:04:25 -0700 From: Graeme Tait Organization: Echidna X-Mailer: Mozilla 2.02 (Win16; I) Mime-Version: 1.0 To: freebsd-questions@freebsd.org Cc: info@boatbooks.com Subject: File system gets too fragmented ??? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I previously posted about a filesystem that was reported full when df -ik showed plenty of space remaining, but got no useful answers. After some further investigation, here's what I think is happening. The filesystem is built with 4096 byte blocks, 512 byte fragments, and 2048 bytes/inode, and is mounted 'async noatime'. It contains about 900,000 files, most of which are small, occupying around 2-5 fragments. The small files are updated monthly from a tar archive generated offline. In the course of updates, some files may grow or shrink by a fragment or so, and some new files are created, a few deleted. Apparently over many months of this, the filesystem became badly fragmented. I'm guessing that when larger files are written, FreeBSD looks for intact blocks to use, but of the around 600MB df -ik reported as "Avail", only about 300MB were in a form it wanted to use for a large files. It was in creating large files that I fouln the problem with "missing" filespace. With the filesystem reported full, 'df -ik' showed Filesystem 1K-blocks Used Avail Capacity iused ifree %iused /dev/da1s1f 3563104 2941142 336913 90% 908255 733055 55% Presumably more space could have been utilized if small files were being written. I "cured" the problem by archiving, deleting and restoring first 10%, then another 10% of the small files. fsck -n now reports: 906229 files, 5827112 used, 1299096 free (366048 frags, 116631 blocks, 5.1% fragmentation) There were 441481 frags after the first 10% were deleted and restored; I didn't think to look at this data before the first delete. It's now possible to write large files that utilize all available space. However, I don't understand how the FFS works, so I'm just probing and guessing as to what's going on here. Could someone please shed a little light on this? Is FreeBSD not able to self-manage a filesystem operated in this way? Is there some way of preventing the problem, or of periodically cleaning things up (rebuilding the whole filesystem from backup means being down for over an hour on a 24x7 server)? In addition, I have a new problem: fsck -n now reports: ** /dev/rda1s1f (NO WRITE) ** Last Mounted on /usr/www ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts UNREF FILE I=48947 OWNER=open MODE=100644 SIZE=13109876 MTIME=May 19 21:55 1999 CLEAR? no UNREF FILE I=48990 OWNER=open MODE=100644 SIZE=13783569 MTIME=May 21 02:26 1999 CLEAR? no UNREF FILE I=48992 OWNER=open MODE=100644 SIZE=20427240 MTIME=May 20 17:32 1999 CLEAR? no ** Phase 5 - Check Cyl groups CLEAN FLAG NOT SET IN SUPERBLOCK FIX? no This never happened before to my knowledge. The UNREF FILE's above are apparently .tgz archives used to update the filesystem, which are normally expanded and deleted without incident. -- Graeme Tait - Echidna To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message