Date: Mon, 29 Sep 2008 21:32:50 -0700 From: Jeremy Chadwick <koitsu@FreeBSD.org> To: Andrew Snow <andrew@modulus.org> Cc: freebsd-stable@freebsd.org Subject: Re: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY Message-ID: <20080930043250.GA36878@icarus.home.lan> In-Reply-To: <48E1839E.3060006@modulus.org> References: <765067435.20080926223557@takeda.tk> <20080927064417.GA43638@icarus.home.lan> <588787159.20080927003750@takeda.tk> <5f67a8c40809282030l7888d942q548d570cd0b33be9@mail.gmail.com> <20080929040025.GA97332@icarus.home.lan> <48E080C0.9070103@modulus.org> <5f67a8c40809290809j58639df8ka65184151161cab6@mail.gmail.com> <5f67a8c40809290849m413eebe6sd31a493aea506932@mail.gmail.com> <200809291744.m8THiBlR034739@apollo.backplane.com> <48E1839E.3060006@modulus.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Sep 30, 2008 at 11:40:46AM +1000, Andrew Snow wrote: > > Matthew Dillon wrote: >> It can take 6 hours to fsck a full 1TB HD. It can >> take over a day to fsck larger setups. Putting in a few sleeps here >> and there just makes the run time even longer and perpetuates the pain. > > We have a box with millions of files spread over 2TB, on a 16 disk RAID. > Foreground fsck takes almost 8 hours, so background fsck, which takes > almost 24 hours or more, is my only option when I want to bring the box > back online quickly. And UFS Snapshots are so slow as to be completely > useless. > > I've now converted the volume to ZFS, and am now enjoying instant boot > time and higher speed I/O under heavy load, at the expense of memory > consumption. > >> My recommendation? Default UFS back to a synchronous fsck and stop >> treating ZFS (your only real alternative) as being so ultra-alpha that >> it shouldn't be used. > > Completely agree. ZFS is the way of the future for FreeBSD. In my > latest testing, the memory problems are now under control, there is just > stability problems with random lockups after days of heavy load unless I > turn off ZIL. So its nearly there. It just now occurred to me that this entire conversation should've been moved to freebsd-fs weeks ago. *laugh* Oh well. :-) You're the first person I've encountered who has had to disable the ZIL to get stability in ZFS; ouch, that must hurt. ZFS stability has been discussed on freebsd-fs numerous times, but the answers provided are always penultimate; no one (AFAIK) has examined how to solve this from the start (specifically new FreeBSD installations). Yes, I know sysinstall/sade doesn't support ZFS (though the PC-BSD folks have apparently implemented this), but that's not what I'm talking about. I'm talking about the most commonly-encountered problem: kmem exhaustion. People want to be able to install FreeBSD then say "Okay! Time to give ZFS a try!" on some separate disks, and have it work. They don't want to encounter kmem exhaustion half way through the migration process; that's just going to dishearten them. I'll be starting up a new topic on freebsd-fs later tonight with an idea I came up with for solving this out-of-the-box. I have a feeling I'm going to get told "so who's going to do all the work?" or downright flamed, but I hope it induces a discussion of ideas, specifically with regards to new FreeBSD installations. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080930043250.GA36878>