Date: Thu, 13 May 1999 09:28:08 +0930 From: Greg Lehey <grog@lemis.com> To: "Mark J. Taylor" <mtaylor@cybernet.com> Cc: Matthew Jacob <mjacob@feral.com>, freebsd-hackers@FreeBSD.ORG, Jim Carroll <jim@carroll.com> Subject: Re: fsck and large file system Message-ID: <19990513092808.X89091@freebie.lemis.com> In-Reply-To: <XFMail.990512151822.mtaylor@cybernet.com>; from Mark J. Taylor on Wed, May 12, 1999 at 03:18:22PM -0400 References: <Pine.LNX.4.04.9905121113090.22991-100000@feral.com> <XFMail.990512151822.mtaylor@cybernet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday, 12 May 1999 at 15:18:22 -0400, Mark J. Taylor wrote: > On 12-May-99 Matthew Jacob wrote: >> >> >>>> I was wondering if anyone has done any work on fsck and very large file >>>> systems. We have a system that has 126 GB RAID Array. As you can imagine, >>>> fsck chokes trying to alloc enough blocks to store it's internal data >>>> structures (128 MB RAM, 128 MB Swap) >> >> Huh- I remember fixing this for NetBSD. You have to do a setrlimit within >> fsck so it can malloc enough space and have enough swap to back that. We >> were fsck'ing 600GB+ filesystems. >> >>> >>>> We would like to treat this array as a single large disk, and was wondering >>>> if anyone else had run into this situation, and had a work around. >>> >> >> I've been doing 120GB+ filesystems for FreeBSD for quite some time. The >> real fun will be the 1TB filesystems. > > The problem that we ran into in a system with several 130 MB RAID5 arrays > is that the fsck was running out of RAM+swap. We had to add a vnode to swap > to before the fsck would complete (basically added more swap space). > We had to have over 100 MB swap space to fsck the 130 MB volume, and the > system has 64 MB RAM. This was is 2.2.8 (haven't upgraded it yet). Why can't you just use more swap rather than resort to vnodes? > BTW: this system is getting VERY poor I/O performance, using the DPT SCSI RAID > controller and three arrays of four 49 GB Seagate drives. "iozone" reports > 340,000 bytes/sec write and 9,800,000 bytes/sec read. That's particularly bad. Which model was it? I've done some comparisons with Vinum and found that the write performance of Vinum (RAID-5) was round 25% of read performance, while it was below 10% on the SmartRAID IV. You should also try rawio (ftp://ftp.lemis.com/pub/rawio.tar.gz). iozone and bonnie use block devices, and they measure the total system, not just the storage device. > This horrendous write rate makes the system virtually unusable. > Anyone have any ideas on improving the performance? Use Vinum :-) > Would an upgrade from its 2.2.8 to 3.{1,2} help? It doesn't seem to be a FreeBSD issue. > It is a Pentium 166. During the "iozone" test, there seems to be > only a few (less than 10) interrupts from the DPT card per second > ("systat -vm 1"). Am I losing interrupts (it would seem so)... ? I think you'd get a message if you lost interrupts. It would be interesting to correlate the interrupt load with the test you're doing. Greg -- See complete headers for address, home page and phone numbers finger grog@lemis.com for PGP public key To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19990513092808.X89091>