Date: Thu, 30 Oct 2008 19:33:47 -0700 From: Jeremy Chadwick <koitsu@FreeBSD.org> To: Rich Winkel <rich@math.missouri.edu> Cc: freebsd-questions@FreeBSD.org Subject: Re: Filesystem, RAID Question Message-ID: <20081031023347.GA19474@icarus.home.lan> In-Reply-To: <20081031014159.GI22915@pencil.math.missouri.edu> References: <9f3798c00810291118i1c80cb8cw8d4995eabe6a4f8f@mail.gmail.com> <4908BE2C.7010505@infracaninophile.co.uk> <20081030231207.GA56260@pencil.math.missouri.edu> <20081030233849.GA16747@icarus.home.lan> <20081031014159.GI22915@pencil.math.missouri.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Oct 30, 2008 at 08:41:59PM -0500, Rich Winkel wrote: > On Thu, Oct 30, 2008 at 04:38:49PM -0700, Jeremy Chadwick wrote: > > On Thu, Oct 30, 2008 at 06:12:07PM -0500, Rich Winkel wrote: > > > Doesn't hw.ata.wc affect only card-level caching? > > > > hw.ata.wc causes the ata(4) subsystem to disable write caching on all > > disks attached to the subsystem. It does not disable card features. > > I mean, the individual disks are invisible to the OS unless the > card's driver (and the card itself) specifically supports it. Correct. With regards to ATA: ata(4) has support for pass-through on some RAID cards, such as Promise. FreeBSD will see the individual disks (e.g. ad4, ad6, etc.) as well as the array (e.g. ar0). With regards to SCSI: pass(4) provides this capability. I don't think in the case of SCSI that the disks will appear in FreeBSD (e.g. da0) though. Instead, pass(4) can be used to query individual disks on an array, e.g. smartctl's -d flag (e.g. -d 3ware, -d marvell, etc.). In both cases (ATA and SCSI), the card itself has to support pass-through, *and* the FreeBSD driver has to have code to allow for such, otherwise no go. > > There's also the below PR, which extends atacontrol to permit disabling > > and enabling write caching on a per-disk basis. > > > > http://www.freebsd.org/cgi/query-pr.cgi?pr=127717 > > But not on disks which are behind hardware raid cards, correct? Correct. For FreeBSD to be able to disable write caching on disks behind a RAID controller, one of two things is needed: 1) Pass-through support (see above), 2) A native CLI program that interfaces with the card directly (usually written by the vendor). Sadly, #2 appears to be the most common choice when a RAID card is used. I say "sadly" because many vendors do not support FreeBSD, and only offer Linux CLI programs -- requiring an administrator to install Linux emulation, Linux libraries, etc., and *hoping* that it works. If the neither of the above options are available, then your only choice is to go into the RAID card's BIOS and disable write caching in there, assuming the option exists (on many cards it does). > > What gives you the impression that during a power outage your data is > > going to be intact? > > One of the main functions of softupdates is to order disk updates in such > a way that the fs organizational integrity is maintained at all times. And we've recently found that this is simply not the case. The benefits of SU are applicable to very specific environments; desktop PCs are the main ones, offering great performance improvements there. But there's a known problem with the "background fsck" feature of FreeBSD, which is only applicable to filesystems which use SU; sometimes fsck does not correct all errors, causing the filesystem to be marked clean, even though there are actual problems with it. There's a thread from about a month ago discussing why background_fsck="no" is highly recommended when using SU. > Of course this doesn't protect against actual sector corruption, but if > the disk is between writes at the time it loses power, the fs structure > is supposed to still be internally consistent. At least that's my > understanding of it. Yep, that's how I understand it as well. But this is a different topic than what we were discussing 2-3 replies ago, talking about how a RAID controller with cache + BBU is sufficient enough to guarantee data integrity even when power is lost -- that's incorrect. Back to write caching: Disabling write caching on disks does not guarantee integrity in the case of such failures either -- on the other hand, by disabling an extra layer of caching, you've essentially diminished the risk by only a nominal amount. I personally believe disabling write caching is not a plausible option for users; the performance hit is major (I have done tests) -- write speeds drop to 12% of total capability. Meaning: 70MB/sec with WC enabled, 8.4MB/sec with WC disabled. This is *without* a controller that does caching of any kind. Essentially you can use this to benchmark which is faster: write caching disabled on disks + caching enabled on a controller, or write caching enabled on disks + caching disabled on a controller. It would be interesting to see benchmark comparisons of different controllers. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20081031023347.GA19474>