Date: Thu, 29 Nov 2001 19:18:57 -0600 From: Bob Willcox <bob@immure.com> To: "Brian T. Schellenberger" <bts@babbleon.org> Cc: "Patrick O'Reilly" <patrick@mip.co.za>, Nils Holland <nils@tisys.org>, FreeBSD Question List <freebsd-questions@FreeBSD.ORG> Subject: Re: Softupdates Message-ID: <20011129191857.A78385@luke.immure.com> In-Reply-To: <01112918422800.00733@i8k.babbleon.org>; from bts@babbleon.org on Thu, Nov 29, 2001 at 06:42:28PM -0500 References: <NDBBIMKICMDGDMNOOCAIOENHDPAA.patrick@mip.co.za> <01112623573601.00696@i8k.babbleon.org> <20011127082804.A4524@luke.immure.com> <01112918422800.00733@i8k.babbleon.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Nov 29, 2001 at 06:42:28PM -0500, Brian T. Schellenberger wrote: > On Tuesday 27 November 2001 09:28, Bob Willcox wrote: > > On Mon, Nov 26, 2001 at 11:57:36PM -0500, Brian T. Schellenberger wrote: > > > On Monday 26 November 2001 18:20, Bob Willcox wrote: > > > > On Mon, Nov 26, 2001 at 09:19:52AM -0500, Brian T.Schellenberger wrote: > > > > > > > > [snip] > > > > > > > > > Also, you most definately should turn off write-caching if you turn > > > > > on softupdates. In fact, you should do this anyway: softupdates are > > > > > really rather safe, but write caching is quite dangerous, and doubly > > > > > so with softupdates enabled. > > > > > > > > > > To do this, set > > > > > > > > > > hw.ata.wc=0 > > > > > > > > > > in your /boot/loader.conf (assuming IDE devices). > > > > > > If you use softupdates *and* enable write caching then you will screw up > > > your disk with a simple sequence like > > > > > > rm -r foo/* > > > shutdown -p now > > > > > > because the softupdates process will just finish writing its updates > > > before the power is clobbered. > > > > Well, my systems don't power down on shutdown (sounds like yours maybe > > do). > > They do. > > > I have to push the power off button so my disk usually has quite a > > bit of time to flush its cache. > > > > > At least that happens with my hardward, so softupdates is not compatible > > > with write-cache-enabled. > > > > > > And from what I can tell (no rigorous testing), softupdates w/o write > > > cache is just as fast as non-softupdates w/ write cache, but lots safer. > > > > Try some benchmarks. An IBM DTLA 45GB drive that I tested with bonnie > > gave me these results with and without write caching: > > > > -------Sequential Output-------- ---Sequential Input-- > > --Random-- IBM DTLA -Per Char- --Block--- -Rewrite-- -Per Char- > > --Block--- --Seeks--- > > WITH WC 1000 26068 34.4 26180 12.7 13990 10.3 27597 > > 57.0 29819 12.4 344.3 1.3 > > W/O WC 1000 9768 13.1 9784 4.8 7547 5.5 > > 26801 55.9 31608 13.8 172.5 0.8 > > Mine are not quite so dramatic, but it does make a big difference. However, > bonnie and real world have essentially nothing in common: as you point out, > turning on and off softupdates has essentially no impact on the bonnie > results but a huge impact on the results in real life. My claim was not that > caching did nothing. It does (almost 3x for you; almost 2x for me); my claim > is that > > softupdates on + write cache Off > and > softupdates off + write cache on > > yield simlar performance, and the latter is a lot safer for your data. > I *have* lost data with write caching on and I don't care to do it again. Depends on what you're doing. If you are simply writing a large file (which I do alot of and is really all that bonnie is doing) softupdates helps little to none (not much meta-data writing going on). However, turning on write caching makes a _big_ difference. > > Now, > softupdates on + write cache on > > is the ultimate in speed, but least safe for your data. Never lost any yet. :-) > > If you never play around with less-than-perfectly mainstream kernel drivers, I have spent the past two years developing a GSN driver on FreeBSD that wasn't always perfect (nor was the hardware)! :-) > and your shutdown doesn't power down for you, and you remember to wait from > shutdown to powerdown, and your data isn't terribly critical, then this might > work for you. As I stated in the quoted text below, I feel that as long as you don't lose power before the disk has a chance to complete the writing of its cache you're okay. For me, that would pretty much only occur if I went up to it and either pulled the plug or pushed the power off button while it was running. > > But, man, the time you waste when you have a signficant data loss even *once* > really makes you think about whether the daily speed increase is worth it > after all . . Hasn't happened yet...and all of my important data is backed up to tape every night. :-) > > > > This was on the exact same hardware (a 1.33 GHz Athlon system). As you > > can see in the second case (w/o write caching) the write performance > > is considerably down. Since bonnie is a simple benchmark program that > > writes a single file softupdates won't help. This is where I saw the > > performance penalty, writing large files. I suspect that without write > > caching enabled in the drive the system was not able to send the > > next block of data to the disk in time for it to not have to take an > > additional revolution in order to write it. > > > > > I know > > > that I've crashed 8 times in the last 36 hours and I'm darn glad I don't > > > enable write caching! (Of course that's not the norm, but sometimes I > > > manage that sort of thing.) > > > > I still contend that as long as you don't suddenly lose power and the > > drive's firmware has a reasonable write behind behavior (i.e., it > > will eventually write the data within a reasonable period...I can't > > confirm this one), then the blocks that reside in the disk's cache will > > be written to the disk even with a system crash (not involving power > > failure). > > I *has* happened to me, so I'm not speculating about theoretical possibities. > Your statement might be correct, of course; my drive's firmware might have > "unreasonable" write behavior, but that's reason enough to turn off write > caching, isn't it? And if you can't tell, do you really want to take chances? > > Perhaps it's all a matter of how risk-averse you are . . . or how badly > you've gotten burned. But I experienced both data loss (twice) and > inapproriate fsck invocation at startup (many times) when I had write cache > enabled. If I felt sufficiently risk-adverse to run my ide drives with write caching disabled I would rather scrap them for SCSI disks. The performance penalty for me is simply unbearable. I backup often, keep critical data on proven to be reliable production systems with UPSes (and use SCSI disks on these systems). I only use IDE drives on largely "don't care" systems where the worst that will happen is that I have to reinstall and reconfigure the machine (a pain, but not that bad). Indeed, the only data loss that I've had in the past 3 or 4 years has been the result of sudden and catastrophic failure of some disks (all IDE, I might add). Perhaps I'm just lucky... > > > > > > BTW, all of my really critical data is kept on a separate system on SCSI > > disks. Also, usually none of my "production" systems crash as many as 8 > > times in a year. :-) > > I was using a customized local arla (AFS) driver that got out of sync, but > the symptoms didn't make that very obvious at first. The crashes are now > under control. My other "crash clusters" were when I was trying to get an > unsupported PCMCIA card driver to work. If you stay away from this sort of > thing, you'll be pretty safe with FreeBSD. > > > Of course, it's your data. :-) If you don't feel safe doing it then by > > all means don't. I'm certainly not trying to talk you into it...just > > expressing a different point of view. > > Ditto. > > *Lots* of people run with write caching enabled all the time and have no > trouble. There is some risk. It will be faster. Linux does it regularly > and loads of people use it all the time without difficulty. (Of course > modern Linux uses a journaling filesystem, which allows for safe write > caching as long the write cache writes data in-order.) If you run other than the default ext2 filesystem (I would bet that most users don't run the journaled FS). And I feel that what the ext2 linux FS does is much higher risk than softupdates with write caching turned on in FreeBSD. They do async meta-data writes which can result in a trashed filesystem on simple system crashes, without the requirement of sudden power failure (meta-data writes may written out of order and never even get sent to the disk at all). Sigh...I guess we should simply agree to disagree on this one... Bob > > > > > Bob > > -- > Brian T. Schellenberger . . . . . . . bts@wnt.sas.com (work) > Brian, the man from Babble-On . . . . bts@babbleon.org (personal) > http://www.babbleon.org > > -------> Free Dmitry Sklyarov! (let him go home) <----------- > > http://www.eff.org http://www.programming-freedom.org -- Bob Willcox Boucher's Observation: bob@vieo.com He who blows his own horn always plays the music Austin, TX several octaves higher than originally written. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20011129191857.A78385>