Date: Wed, 28 Dec 2011 11:14:19 -0800 From: mdf@FreeBSD.org To: Maxim Khitrov <max@mxcrypt.com> Cc: Matthias Andree <matthias.andree@gmx.de>, freebsd-current@freebsd.org Subject: Re: SU+J systems do not fsck themselves Message-ID: <CAMBSHm-%2BJGQiXHRh0ona3qZN2O2s=7dgnQ_3q_NoLe-CWM2yww@mail.gmail.com> In-Reply-To: <CAJcQMWeDeCC-pvz4tfwZKAUqxZnpPBOpa9r12o0_w4=YyGf4zA@mail.gmail.com> References: <20111227215330.GI45484@redundancy.redundancy.org> <4EFB470D.3070309@gmx.de> <CAJcQMWeDeCC-pvz4tfwZKAUqxZnpPBOpa9r12o0_w4=YyGf4zA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Dec 28, 2011 at 8:54 AM, Maxim Khitrov <max@mxcrypt.com> wrote: > On Wed, Dec 28, 2011 at 11:42 AM, Matthias Andree > <matthias.andree@gmx.de> wrote: >> Am 27.12.2011 22:53, schrieb David Thiel: >>> I've had multiple machines now (9.0-RC3, amd64, i386 and earlier >>> 9-CURRENT on ppc) running SU+J that have had unexplained panics and >>> crashes start happening relating to disk I/O. When I end up running a >>> full fsck, it keeps turning out that the disk is dirty and corrupted, >>> but no mechanism is in place with SU+J to detect and fix this. A bgfsck >>> never happens, but a manual fsck in single-user does indeed fix the >>> crashing and weird behavior. Others have tested their SU+J volumes and >>> found them to have errors as well. This makes me super nervous. >> >> The one thing I figured is that in the light of power outages, or >> crashing virtualization hosts, you really really really need to disable >> disk write caches, and this affects softupdates, journalling, asynch >> file systems, just about everything. >> >> The fact that makes matters worse is that journalling or softupdates >> allow you to mount a silently-corrupted file system, whereas the >> traditional UFS/UFS2 sync/asynch mounts will fsck themselves in the >> foreground, so they get fixed before the FS panics. >> >> So can you be sure that: >> >> - your driver, chip set and hard disk execute ordered writes in order, If they don't, it's a bug. Not that there isn't buggy firmware out there, but each layer of software does need to rely on the one below actually doing what it's promised. >> - your driver, chip set and hard disk actually write data to permanent >> storage BEFORE acknowledging a successful write? Not required by SU as they use an explicit BIO_FLUSH which should be handled by the driver. >> Whenever I fixed these issues, I had no more corruptions. >> >> For ata and sata, there are loader tunables you will want to set, >> hw.ata.wc=3D0 and kern.cam.ada.write_cache=3D0. This should not be necessary if the driver and firmware are not buggy. >> If your drives are under ada, ad, or ahci related control, try these >> settings. =A0For SCSI, use camcontrol to turn the write cache off. >> softupdates is supposed to rectify most of the performance penalties >> incurred. >> >> Note also that you needed to set ahci_load=3DYES and atapicam_load=3DYES= in >> 8.X, I've never bothered to check 7.X or 9.X WRT these settings. > > This is a bit off-topic, but I'm curious what the effect of NCQ is on > softupdates? Since that too has the ability to reorder writes to disk, > should it be disabled in addition to cache? SU doesn't care about write ordering, as long as everything before a BIO_FLUSH is really flushed by the time the BIO_FLUSH is acknowledged. Cheers, matthew > Also, I would say that if you are using a hardware raid controller > with a BBU, then allowing the use of controller's cache and write-back > policy should be safe for use with softupdates. Any caching mechanism, > for that matter, that has a separate power supply source should be ok. > For example, the Intel 320 SSDs have a few on-board capacitors that > are used to flush the cache in the event of a power loss. > > - Max > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= "
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMBSHm-%2BJGQiXHRh0ona3qZN2O2s=7dgnQ_3q_NoLe-CWM2yww>