From owner-freebsd-fs Mon Oct 12 06:22:09 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA20253 for freebsd-fs-outgoing; Mon, 12 Oct 1998 06:22:09 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA20235; Mon, 12 Oct 1998 06:22:07 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id GAA12095; Mon, 12 Oct 1998 06:18:46 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id GAA14367; Mon, 12 Oct 1998 06:18:45 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id GAA09733; Mon, 12 Oct 1998 06:18:43 -0700 (PDT) From: Don Lewis Message-Id: <199810121318.GAA09733@salsa.gv.tsc.tdk.com> Date: Mon, 12 Oct 1998 06:18:43 -0700 In-Reply-To: Eivind Eklund "Re: filesystem safety and SCSI disk write caching" (Oct 4, 3:25pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: Eivind Eklund , Mike Smith , Bruce Evans Subject: Re: filesystem safety and SCSI disk write caching Cc: Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG, gibbs@plutotech.com, tlambert@primenet.com Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 4, 3:25pm, Eivind Eklund wrote: } Subject: Re: filesystem safety and SCSI disk write caching } On Sun, Oct 04, 1998 at 12:06:15AM -0700, Mike Smith wrote: } > > >> Yes, the default configuration may be much slower than mine. } > > > } > > >I can definitely back your basic point ('make world' is CPU bound) up. } > > >On a 4-way Xeon system with slow disks we were still able to get down } > > >around 40 minutes. } > > } > > Er, that shows that it is i/o bound on systems with so much CPU. I } > > got it down to 75 minutes on 1-way K6-233 with 1 IDE disk before it } > > was bloated by perl5 and transition to elf. } > } > Moving to an MFS only saved about 15% of the build time. My point was } > that a faster CPU let you go faster. If the build was I/O bound, it } > wouldn't. } } My hypothesis is that for the high end boxes, 'make world' is mostly } bound by memory bandwidth. This is what seems to best match the speed } patterns people have been reporting. I don't think that's it either. One would certainly hope that a 4-way Xeon system would have more than twice as much memory bandwidth as a K6-233. I'm getting times down around 92 minutes with a Pentium II - 266 and a single Seagate Hawk when building FreeBSD post elf and perl5. That should get the times down to under 60 minutes with a Pentium II - 450. My conclusion is that running FreeBSD on a 4-way Xeon is not a cost effective way to get buildworld to run faster ... Here are some statistics that I gathered by varying the mount options on /usr/obj and rerunning make buildworld. It now looks like enabling SCSI write caching makes even less difference in the timing than I originally thought (a maximum of about 4 minutes with standard mount options, and a maximum of about 1 minute with either softupdates or async, typically much less). Also, softupdates is generally faster than async. I haven't yet had time to try to track down why "make -j4 buildworld" fails with the standard mount options. make -j1 -j2 -j3 -j4 -j6 -j8 -j12 /usr/obj mount options + SCSI write caching standard 158:04 133:33 129:56 FAIL *126:09 127:58 128:42 standard+write-cache 154:54 131:58 126:13 FAIL *115:44 123:44 126:24 softupdates 126:23 96:36 93:33 92:38 92:02 91:50 92:12 softupdates+write-cache 125:58 95:10 92:32 91:54 91:18 91:10 91:35 async 127:06 96:40 93:27 92:45 91:59 92:07 92:15 async+write-cache 125:18 95:54 92:56 92:05 91:26 91:24 91:36 Times in MMM:SS. All "make buildworld" runs started with a full object tree except: * partial object tree because of failure during previous run Hardware: Pentium II - 266 64 MB RAM Adaptec 2940UW Seagate ST-32151N Hawk 2XL Fast SCSI-2 (10.0MB/s transfer rate) Software: FreeBSD 3.0-BETA from September 26 with softupdates fixes and CAM. The softupdates times may be somewhat pessimistic because the code still has a number of sanity checks that I inserted to track down the newdirrem panic. All partitions mounted with standard mount options except /tmp which used mfs (and "setenv TMPDIR /tmp") and /usr/obj whose mount options were varied. All partitions were on the same spindle. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 06:26:32 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA20853 for freebsd-fs-outgoing; Mon, 12 Oct 1998 06:26:32 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA20835; Mon, 12 Oct 1998 06:26:30 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id GAA12141; Mon, 12 Oct 1998 06:26:08 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id GAA14505; Mon, 12 Oct 1998 06:26:07 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id GAA09753; Mon, 12 Oct 1998 06:26:06 -0700 (PDT) From: Don Lewis Message-Id: <199810121326.GAA09753@salsa.gv.tsc.tdk.com> Date: Mon, 12 Oct 1998 06:26:05 -0700 In-Reply-To: "Justin T. Gibbs" "Re: filesystem safety and SCSI disk write caching" (Oct 4, 9:51pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: "Justin T. Gibbs" , Terry Lambert Subject: Re: filesystem safety and SCSI disk write caching Cc: julian@whistle.com, Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 4, 9:51pm, "Justin T. Gibbs" wrote: } Subject: Re: filesystem safety and SCSI disk write caching } >> >I told you so. } >> } >> You told me some things that were in-correct and some things that } >> I already knew. Par for the course. } > } >Feel free to make his setup work with SCSI write caching enabled. } } I gave the recipe for this on freebsd-alpha near the end of september. } } 1) Use a UPS. Even with an UPS, a large percentage of our unclean shutdowns are power related. Most of these are due to power outages that last longer than our UPS batteries. I don't think there is enough room in our building to store enough UPS batteries to last through our typical winter power outages. I don't think we'll be getting a backup generator anytime soon, and even then I've heard quite a few stories on freebsd-isp about problems getting generators to reliably start. } 2) Use a drive with non-bogus firmware. Recent Seagate and IBM } drives should work just fine. I haven't validated any Quantum } drives in this regard yet. But how can tell if the firmware is non-bogus? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 06:30:10 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA21580 for freebsd-fs-outgoing; Mon, 12 Oct 1998 06:30:10 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA21574 for ; Mon, 12 Oct 1998 06:30:09 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id GAA12164; Mon, 12 Oct 1998 06:29:47 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id GAA14515; Mon, 12 Oct 1998 06:29:46 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id GAA09765; Mon, 12 Oct 1998 06:29:40 -0700 (PDT) From: Don Lewis Message-Id: <199810121329.GAA09765@salsa.gv.tsc.tdk.com> Date: Mon, 12 Oct 1998 06:29:39 -0700 In-Reply-To: Hinrich Eilts "Re: filesystem safety and SCSI disk write caching" (Oct 4, 4:00pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: Hinrich Eilts , freebsd-fs@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 4, 4:00pm, Hinrich Eilts wrote: } Subject: Re: filesystem safety and SCSI disk write caching } > I can post (once again) the results of a Novell study on server usage } > patterns. The 30,000 foot view for a typical server breaks down to: } > } > 75% reads } > 15% writes } > 8% directory search operations } > 2% other } } How does this translate to disk usage? I think, VM/buffer cache will } reduce reads and directory search operations quite a lot. I've actually seen some figures quoted for this, but darned if I can figure out where. The numbers I saw showed writes dominating the mix on the other side of the buffer cache. Obviously this will depend on the workload, and I would expect that softupdates would further perturb the numbers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 07:08:25 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA26475 for freebsd-fs-outgoing; Mon, 12 Oct 1998 07:08:25 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from ifi.uio.no (ifi.uio.no [129.240.64.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA26434; Mon, 12 Oct 1998 07:08:16 -0700 (PDT) (envelope-from dag-erli@ifi.uio.no) Received: from urdarbrunni.ifi.uio.no (2602@urdarbrunni.ifi.uio.no [129.240.64.184]) by ifi.uio.no (8.8.8/8.8.7/ifi0.2) with ESMTP id QAA27138; Mon, 12 Oct 1998 16:07:52 +0200 (MET DST) Received: (from dag-erli@localhost) by urdarbrunni.ifi.uio.no ; Mon, 12 Oct 1998 16:07:51 +0200 (MET DST) Mime-Version: 1.0 To: Don Lewis Cc: "Justin T. Gibbs" , Terry Lambert , julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching References: <199810121326.GAA09753@salsa.gv.tsc.tdk.com> Organization: University of Oslo, Department of Informatics X-url: http://www.stud.ifi.uio.no/~dag-erli/ X-other-addresses: 'finger dag-erli@ifi.uio.no' for a list X-disclaimer-1: The views expressed in this article are mine alone, and do X-disclaimer-2: not necessarily coincide with those of any organisation or X-disclaimer-3: company with which I am or have been affiliated. X-Stop-Spam: http://www.cauce.org/ From: dag-erli@ifi.uio.no (Dag-Erling C. =?iso-8859-1?Q?Sm=F8rgrav?= ) Date: 12 Oct 1998 16:07:50 +0200 In-Reply-To: Don Lewis's message of "Mon, 12 Oct 1998 06:26:05 -0700" Message-ID: Lines: 13 X-Mailer: Gnus v5.5/Emacs 19.34 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hub.freebsd.org id HAA26449 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Don Lewis writes: > On Oct 4, 9:51pm, "Justin T. Gibbs" wrote: > > 1) Use a UPS. > Even with an UPS, a large percentage of our unclean shutdowns are power > related. Most of these are due to power outages that last longer than > our UPS batteries. 1.5) When the UPS reports that the battery is running low, shut everything down. DES -- Dag-Erling Smørgrav - dag-erli@ifi.uio.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 07:22:46 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA29401 for freebsd-fs-outgoing; Mon, 12 Oct 1998 07:22:46 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from magicnet.magicnet.net (magicnet.magicnet.net [204.96.116.9]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA29370 for ; Mon, 12 Oct 1998 07:22:39 -0700 (PDT) (envelope-from bill@bilver.magicnet.net) Received: (from root@localhost) by magicnet.magicnet.net (8.8.6/8.8.8) with UUCP id KAA26367 for freebsd-fs@freebsd.org; Mon, 12 Oct 1998 10:20:03 -0400 (EDT) Received: (from bill@localhost) by bilver.magicnet.net (8.9.1/8.9.1) id KAA15041 for freebsd-fs@freebsd.org; Mon, 12 Oct 1998 10:12:02 -0400 (EDT) From: Bill Vermillion Message-Id: <199810121412.KAA15041@bilver.magicnet.net> Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810121326.GAA09753@salsa.gv.tsc.tdk.com> from Don Lewis at "Oct 12, 98 06:26:05 am" To: freebsd-fs@FreeBSD.ORG Date: Mon, 12 Oct 1998 10:12:01 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Don Lewis recently said: > Even with an UPS, a large percentage of our unclean shutdowns are > power related. Most of these are due to power outages that last > longer than our UPS batteries. I don't think there is enough room > in our building to store enough UPS batteries to last through our > typical winter power outages. I don't think we'll be getting a > backup generator anytime soon, and even then I've heard quite a > few stories on freebsd-isp about problems getting generators to > reliably start. Today's UPSes typically have self-monitoring capabilities to put a signal on a pin when they reach a certain minimum capacity. At this point they will send a system shutdown command. If you have nothing as sophisticated as that, build your own. Have a normally open 110VAC relay. Connect two contacts to the TX/RX pins on an RS232. Push something through that ever minute or so. When the signal can't be passed the system will not get this signal - indicating the power has failed on the relay and it opened up (barring some other failure). If your battery is good for 20 minutes, use a routine like this. If no signal (eg no AC to relay), start timer. Set a variable to 10 Every minute check the connection. If not connected decrement the variable. If power comes up, stop countdown routine. If power still not there software monitor the relay will issue a shutdown when the counter decrements to zero. As to getting generators to reliably start - good systems from known vendors usually work well. It's the home-brew units that lack niceties. At a radio station I worked with a zillion years ago we had a motor-generator set that we were required to have (one of the EBS stations), and that had to be run once per week, also another legal requirment. It was a Ford V8 running on propane. Always started - no problems - but it was homebrew. At the end of the require hour on the air with it running all the clocks would be about 5 minutes slow and all the records play about 5-10% underspeed. Current offerings from companies such as Best - will use Onan or Sony for their small systems. Software automatically tests them once per week by starting them an running them. Options are large tanks to extend running time with no refills to days, etc. Good equipment almost never fails. Cheap equipment quite often fails - and Murphy's law dictates that it will fail when you most need it. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 08:43:58 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA11365 for freebsd-fs-outgoing; Mon, 12 Oct 1998 08:43:58 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA11345; Mon, 12 Oct 1998 08:43:55 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id JAA01328; Mon, 12 Oct 1998 09:43:19 -0600 (MDT) Message-Id: <199810121543.JAA01328@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: dag-erli@ifi.uio.no (Dag-Erling C. =?iso-8859-1?Q?Sm=F8rgrav?= ) cc: Don Lewis , "Justin T. Gibbs" , Terry Lambert , julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "12 Oct 1998 16:07:50 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Date: Mon, 12 Oct 1998 09:36:33 -0600 From: "Justin T. Gibbs" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hub.freebsd.org id IAA11350 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Don Lewis writes: >> On Oct 4, 9:51pm, "Justin T. Gibbs" wrote: >> > 1) Use a UPS. >> Even with an UPS, a large percentage of our unclean shutdowns are power >> related. Most of these are due to power outages that last longer than >> our UPS batteries. > >1.5) When the UPS reports that the battery is running low, shut > everything down. Exactly. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 08:58:12 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA14349 for freebsd-fs-outgoing; Mon, 12 Oct 1998 08:58:12 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA14196; Mon, 12 Oct 1998 08:58:01 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id JAA04320; Mon, 12 Oct 1998 09:57:28 -0600 (MDT) Message-Id: <199810121557.JAA04320@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Don Lewis cc: "Justin T. Gibbs" , Terry Lambert , julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Mon, 12 Oct 1998 06:26:05 PDT." <199810121326.GAA09753@salsa.gv.tsc.tdk.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 12 Oct 1998 09:50:42 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >} 2) Use a drive with non-bogus firmware. Recent Seagate and IBM >} drives should work just fine. I haven't validated any Quantum >} drives in this regard yet. > >But how can tell if the firmware is non-bogus? Ask Terry since he has stated that he 'doesn't have any drives with non-bogus firmware'. Seriously, the major complaint I've heard about firmware has to do with it not properly flushing the cache on a bus reset. I've never seen that failure mode here, and I've done quite a bit of "external bus reset" testing. You'll need sophisticated tools in order to perform these kinds of tests: 1) Find a paper clip 2) Find a ribbon cable that has enough connectors to attach to the device you want to test and the controller with a connector spare. 3) Start lots of writes 4) Ground pin 40 to pin 39 using the paper clip from step 1. 5) Verify data 6) goto 3 -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 08:59:15 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA14671 for freebsd-fs-outgoing; Mon, 12 Oct 1998 08:59:15 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA14636; Mon, 12 Oct 1998 08:59:10 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id JAA04452; Mon, 12 Oct 1998 09:58:32 -0600 (MDT) Message-Id: <199810121558.JAA04452@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "Justin T. Gibbs" cc: Don Lewis , Terry Lambert , julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Mon, 12 Oct 1998 09:50:42 MDT." <199810121557.JAA04320@pluto.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 12 Oct 1998 09:51:46 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This assumes a Single Ended SCSI configuration... >1) Find a paper clip > >2) Find a ribbon cable that has enough connectors to attach to the device >you want to test and the controller with a connector spare. > >3) Start lots of writes > >4) Ground pin 40 to pin 39 using the paper clip from step 1. > >5) Verify data > >6) goto 3 > >-- >Justin > > > -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 13:19:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA01680 for freebsd-fs-outgoing; Mon, 12 Oct 1998 13:19:20 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA01675 for ; Mon, 12 Oct 1998 13:19:19 -0700 (PDT) (envelope-from tlambert@usr07.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id NAA22149; Mon, 12 Oct 1998 13:19:05 -0700 (MST) Received: from usr07.primenet.com(206.165.6.207) via SMTP by smtp04.primenet.com, id smtpd022118; Mon Oct 12 13:18:58 1998 Received: (from tlambert@localhost) by usr07.primenet.com (8.8.5/8.8.5) id NAA21730; Mon, 12 Oct 1998 13:18:54 -0700 (MST) From: Terry Lambert Message-Id: <199810122018.NAA21730@usr07.primenet.com> Subject: Re: Optimizing space utilization To: francisco@natserv.com Date: Mon, 12 Oct 1998 20:18:53 +0000 (GMT) Cc: grog@lemis.com, freebsd-fs@FreeBSD.ORG In-Reply-To: <199810110304.XAA22392@federation.addy.com> from "Francisco Reyes" at Oct 10, 98 11:04:35 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >> On a 1 Gig partition by using 1/4 the number of inodes (sent -i 16384 > >> to newfs) I was able to save a little over 30MB. A 3% saving. :-) > >> I was still left with over 64K inodes, which I doubt I will use. > > > Interesting. At 262 bytes per inode, I'd only expect a saving of > > about 12 MB. Uh, what are you smoking? An inode is 128 bytes, exactly, including reserve space. Are you perhaps counting the mandatory 24 byte overhead on a directory entry, and assuming a 110 byte file name? If so, you should be aware that flexname allocations must round to 4 byte boundaries, so you are off by 2 (108 and 112 are evenly divisible by 4). > Something else I noticed.. > The total space reported by df (heading 1K-blocks) also went up > slightly. Don't recall exact number, but I think it was 3 to 5MB. Clearly, so long as we make a distinction between 512b physical blocks divided up between 4 inodes and 512b physical blocks that are agregated and represent portions of file system blocks, you can adjust the ratio of blocks used for inodes vs. blocks used for data. So yes, you could adjust this to get some space back. But if you go over an 85% fill, then even a perfect hash begins to have collisions. The historical 10% reserve was chosen as a compromise of diminishing returns for hash fill efficiency (read the FFS paper, and read Knuth's Seminumerical Algorithms). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 15:47:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA01786 for freebsd-fs-outgoing; Mon, 12 Oct 1998 15:47:34 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA01773; Mon, 12 Oct 1998 15:47:24 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id PAA28425; Mon, 12 Oct 1998 15:47:06 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp03.primenet.com, id smtpd028394; Mon Oct 12 15:47:03 1998 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id PAA10583; Mon, 12 Oct 1998 15:46:58 -0700 (MST) From: Terry Lambert Message-Id: <199810122246.PAA10583@usr02.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: Don.Lewis@tsc.tdk.com (Don Lewis) Date: Mon, 12 Oct 1998 22:46:57 +0000 (GMT) Cc: gibbs@plutotech.com, tlambert@primenet.com, julian@whistle.com, Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810121326.GAA09753@salsa.gv.tsc.tdk.com> from "Don Lewis" at Oct 12, 98 06:26:05 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > } 1) Use a UPS. > > Even with an UPS, a large percentage of our unclean shutdowns are power > related. Most of these are due to power outages that last longer than > our UPS batteries. I don't think there is enough room in our building > to store enough UPS batteries to last through our typical winter power > outages. I don't think we'll be getting a backup generator anytime soon, > and even then I've heard quite a few stories on freebsd-isp about problems > getting generators to reliably start. The point of a UPS is to give you room to perform an orderly shutdown prior to UPS failure. This requires using a UPS that support monitoring and software control of power-on-state and running a monitoring daemon capable of powering down the UPS. > } 2) Use a drive with non-bogus firmware. Recent Seagate and IBM > } drives should work just fine. I haven't validated any Quantum > } drives in this regard yet. > > But how can tell if the firmware is non-bogus? Someone you trust tells you it is non-bogus. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 15:59:09 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA03987 for freebsd-fs-outgoing; Mon, 12 Oct 1998 15:59:09 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA03916; Mon, 12 Oct 1998 15:58:48 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id PAA12815; Mon, 12 Oct 1998 15:58:30 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp02.primenet.com, id smtpd012733; Mon Oct 12 15:58:20 1998 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id PAA11377; Mon, 12 Oct 1998 15:58:15 -0700 (MST) From: Terry Lambert Message-Id: <199810122258.PAA11377@usr02.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: gibbs@plutotech.com (Justin T. Gibbs) Date: Mon, 12 Oct 1998 22:58:15 +0000 (GMT) Cc: Don.Lewis@tsc.tdk.com, gibbs@plutotech.com, tlambert@primenet.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810121557.JAA04320@pluto.plutotech.com> from "Justin T. Gibbs" at Oct 12, 98 09:50:42 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >} 2) Use a drive with non-bogus firmware. Recent Seagate and IBM > >} drives should work just fine. I haven't validated any Quantum > >} drives in this regard yet. > > > >But how can tell if the firmware is non-bogus? > > Ask Terry since he has stated that he 'doesn't have any drives with > non-bogus firmware'. A) Run soft updates B) Press "reset" occasionally C) Note any anomalies in the resulting fsck when the machine comes back up D) if count < 200, goto B E) if # of anomalies > 0, print "bad firmware". > Seriously, the major complaint I've heard about firmware has to do with it > not properly flushing the cache on a bus reset. I've never seen that > failure mode here, and I've done quite a bit of "external bus reset" > testing. You'll need sophisticated tools in order to perform these kinds > of tests: > > 1) Find a paper clip > > 2) Find a ribbon cable that has enough connectors to attach to the device > you want to test and the controller with a connector spare. > > 3) Start lots of writes > > 4) Ground pin 40 to pin 39 using the paper clip from step 1. > > 5) Verify data > > 6) goto 3 It's very hard to do this in software, without providing a mechanism to actually break into the latency link between the derive reporting a write cached operation has been written, and the actual writing. Such a latency link only exists on drives which Justin has identified as having broken firmware due to the behaviour reported by Don Lewis. I would be much more interested in knowing what drives and firmware revisions of those drives Justin has, since both mine and Don Lewis's are demonstrably broken. The drives I can demonstrate breakage on are a 9G IBM drive, a 2.1G Quantum drive, and two .5G DEC drives. I can get exact model numbers if necessary, but it seems to me, from empirical evidence so far, that the number of "broken firmware" drives, as defined by Justin, outnumber the number of non-broken firmware drives. This means that a "known good" table would be smaller than a "known rogues" table, and thus a better mechanism for implementing the decision about whether write caching should be enabled on the drive, or not. I'll be happy (well, actually I won't) to have it proven to me that my and Don Lewis' drives are the exceptions, rather than the rule. 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 19:14:31 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA05880 for freebsd-fs-outgoing; Mon, 12 Oct 1998 19:14:31 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from allegro.lemis.com (allegro.lemis.com [192.109.197.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA05871 for ; Mon, 12 Oct 1998 19:14:27 -0700 (PDT) (envelope-from grog@freebie.lemis.com) Received: from freebie.lemis.com (freebie.lemis.com [192.109.197.137]) by allegro.lemis.com (8.9.1/8.9.0) with ESMTP id LAA21309; Tue, 13 Oct 1998 11:44:11 +0930 (CST) Received: (from grog@localhost) by freebie.lemis.com (8.9.1/8.9.0) id LAA19951; Tue, 13 Oct 1998 11:44:11 +0930 (CST) Message-ID: <19981013114410.A21983@freebie.lemis.com> Date: Tue, 13 Oct 1998 11:44:10 +0930 From: Greg Lehey To: Terry Lambert , francisco@natserv.com Cc: freebsd-fs@FreeBSD.ORG Subject: Re: Optimizing space utilization References: <199810110304.XAA22392@federation.addy.com> <199810122018.NAA21730@usr07.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.91.1i In-Reply-To: <199810122018.NAA21730@usr07.primenet.com>; from Terry Lambert on Mon, Oct 12, 1998 at 08:18:53PM +0000 WWW-Home-Page: http://www.lemis.com/~grog Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Monday, 12 October 1998 at 20:18:53 +0000, Terry Lambert wrote: >>>> On a 1 Gig partition by using 1/4 the number of inodes (sent -i 16384 >>>> to newfs) I was able to save a little over 30MB. A 3% saving. :-) >>>> I was still left with over 64K inodes, which I doubt I will use. >> >>> Interesting. At 262 bytes per inode, I'd only expect a saving of >>> about 12 MB. > > Uh, what are you smoking? Cooked inodes. > An inode is 128 bytes, exactly, including reserve space. Oops. The in-core inode is 262 bytes, which isn't particularly relevant to the disk. Where should I have been looking (i.e. which file?). > Are you perhaps counting the mandatory 24 byte overhead on a directory > entry, and assuming a 110 byte file name? If so, you should be > aware that flexname allocations must round to 4 byte boundaries, > so you are off by 2 (108 and 112 are evenly divisible by 4). No. Here's a test program: #define MAXQUOTAS 2 #include "/T/vinum/vinumhdr.h" #include main () { printf ("inode size is %d\n", sizeof (struct inode)); } The MAXQUOTAS is because I didn't want a whole slew of knotted header files. vinumhdr is gratuitous, but save more knotted header files. Greg -- See complete headers for address, home page and phone numbers finger grog@lemis.com for PGP public key To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Oct 12 22:11:50 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA00672 for freebsd-fs-outgoing; Mon, 12 Oct 1998 22:11:50 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA00656; Mon, 12 Oct 1998 22:11:48 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id WAA22177; Mon, 12 Oct 1998 22:11:28 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id WAA00543; Mon, 12 Oct 1998 22:11:27 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id VAA14457; Mon, 12 Oct 1998 21:33:54 -0700 (PDT) From: Don Lewis Message-Id: <199810130433.VAA14457@salsa.gv.tsc.tdk.com> Date: Mon, 12 Oct 1998 21:33:53 -0700 In-Reply-To: dag-erli@ifi.uio.no (Dag-Erling C. =?iso-8859-1?Q?Sm=F8rgrav?= ) "Re: filesystem safety and SCSI disk write caching" (Oct 12, 4:07pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: dag-erli@ifi.uio.no (Dag-Erling C. =?iso-8859-1?Q?Sm=F8rgrav?= ), Don Lewis Subject: Re: filesystem safety and SCSI disk write caching Cc: "Justin T. Gibbs" , Terry Lambert , julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 12, 4:07pm, Dag-Erling C. =?iso-8859-1?Q?Sm=F8rgrav?= wrote: } Subject: Re: filesystem safety and SCSI disk write caching } Don Lewis writes: } > On Oct 4, 9:51pm, "Justin T. Gibbs" wrote: } > > 1) Use a UPS. } > Even with an UPS, a large percentage of our unclean shutdowns are power } > related. Most of these are due to power outages that last longer than } > our UPS batteries. } } 1.5) When the UPS reports that the battery is running low, shut } everything down. I'd like to do that at some point. I'd have to rig up some way of broadcasting that info on our network, since the UPS lives two floors below. So far as I know, it's not network aware. I haven't seen any info on how to interface to it, but then I haven't really looked, either. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 00:06:13 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA11466 for freebsd-fs-outgoing; Tue, 13 Oct 1998 00:06:13 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA11449; Tue, 13 Oct 1998 00:06:09 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id BAA12205; Tue, 13 Oct 1998 01:05:50 -0600 (MDT) Message-Id: <199810130705.BAA12205@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: gibbs@plutotech.com (Justin T. Gibbs), Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Mon, 12 Oct 1998 22:58:15 -0000." <199810122258.PAA11377@usr02.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Oct 1998 00:59:05 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >> >} 2) Use a drive with non-bogus firmware. Recent Seagate and IBM >> >} drives should work just fine. I haven't validated any Quantum >> >} drives in this regard yet. >> > >> >But how can tell if the firmware is non-bogus? >> >> Ask Terry since he has stated that he 'doesn't have any drives with >> non-bogus firmware'. > >A) Run soft updates >B) Press "reset" occasionally >C) Note any anomalies in the resulting fsck when the machine > comes back up >D) if count < 200, goto B >E) if # of anomalies > 0, print "bad firmware". You're missing a large step here. You can't prove that the 'anomaly' is related to the drive firmware without a trace of all transactions on the SCSI bus. It could well be a missing dependency in the soft update code. I'd be more than happy to reproduce your failure scenario while recording a SCSI bus trace so that the fault is easy to interpret. Just send me any *modern* drive that you think fails. You should also ensure that your reset button does not cause any power spikes on the drive power lines. That would be cheating. >It's very hard to do this in software, without providing a mechanism >to actually break into the latency link between the drive reporting >a write cached operation has been written, and the actual writing. If you can cause this a failure to occur by hitting your reset button, I should be able to cause it to occur by using a paper-clip if the reset condition (cased by the SCSI card BIOS in the reset button case) is the event that causes cache corruption. Both are non-deterministic methods of error injection. >Such a latency link only exists on drives which Justin has identified >as having broken firmware due to the behaviour reported by Don Lewis. I'm still unclear as to whether Don was turning off power or hitting what I consider the reset button. His comment about UPSes use makes me think he was testing power outage scenarios. >I would be much more interested in knowing what drives and firmware >revisions of those drives Justin has, since both mine and Don Lewis's >are demonstrably broken. Since you were able to test 4 drives so quickly, I'd love to see well documented information on exactly how the file system was inconsistent in the failure cases. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 00:22:36 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA13191 for freebsd-fs-outgoing; Tue, 13 Oct 1998 00:22:36 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA13172; Tue, 13 Oct 1998 00:22:34 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id AAA23196; Tue, 13 Oct 1998 00:22:16 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id AAA02758; Tue, 13 Oct 1998 00:22:15 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id AAA14971; Tue, 13 Oct 1998 00:22:13 -0700 (PDT) From: Don Lewis Message-Id: <199810130722.AAA14971@salsa.gv.tsc.tdk.com> Date: Tue, 13 Oct 1998 00:22:12 -0700 In-Reply-To: "Justin T. Gibbs" "Re: filesystem safety and SCSI disk write caching" (Oct 13, 12:59am) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: "Justin T. Gibbs" , Terry Lambert Subject: Re: filesystem safety and SCSI disk write caching Cc: Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 13, 12:59am, "Justin T. Gibbs" wrote: } Subject: Re: filesystem safety and SCSI disk write caching } I'm still unclear as to whether Don was turning off power or hitting what I } consider the reset button. His comment about UPSes use makes me think he } was testing power outage scenarios. I was hitting the reset button to test the softupdates code because I thought that write caching on the drive was disabled. It wasn't until I got the unexpected filesystem inconsistency that I actually checked the state of the write caching bit. Once I disable write caching, I didn't get any more unexpected inconsistencies, though I didn't repeat this experiment enough to totally convince myself that my results were conclusive. With write caching disabled, I'd expect the same results with either the reset button or the power switch, but I didn't test the latter since it's a lot rougher on the hardware. If the inconsistency was due to a missing softupdates dependency, I'd expect to see more inconsistencies in the many panics I provoked in some other testing I was doing. With the exception of some inconsistencies created by bugs in fsck that corrupted the filesystem while attempting to fix it, I did not see any unexpected inconsistencies caused by these panics. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 16:27:54 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA19489 for freebsd-fs-outgoing; Tue, 13 Oct 1998 16:27:54 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA19478 for ; Tue, 13 Oct 1998 16:27:50 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id QAA06191; Tue, 13 Oct 1998 16:27:35 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpd006104; Tue Oct 13 16:27:26 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA16801; Tue, 13 Oct 1998 16:27:07 -0700 (MST) From: Terry Lambert Message-Id: <199810132327.QAA16801@usr08.primenet.com> Subject: Re: Optimizing space utilization To: grog@lemis.com (Greg Lehey) Date: Tue, 13 Oct 1998 23:27:07 +0000 (GMT) Cc: tlambert@primenet.com, francisco@natserv.com, freebsd-fs@FreeBSD.ORG In-Reply-To: <19981013114410.A21983@freebie.lemis.com> from "Greg Lehey" at Oct 13, 98 11:44:10 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Uh, what are you smoking? > > Cooked inodes. 8-). > > An inode is 128 bytes, exactly, including reserve space. > > Oops. The in-core inode is 262 bytes, which isn't particularly > relevant to the disk. Where should I have been looking (i.e. which > file?). /sys/ufs/ufs/dinode.h (for struct dinode) or /sys/ufs/ufs/inode.h (for "struct dinode i_din; /* 128 bytes of the on-disk dinode. */"). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 16:59:28 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA25226 for freebsd-fs-outgoing; Tue, 13 Oct 1998 16:59:28 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA25212; Tue, 13 Oct 1998 16:59:26 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id QAA17796; Tue, 13 Oct 1998 16:59:10 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpd017768; Tue Oct 13 16:59:04 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA18137; Tue, 13 Oct 1998 16:58:59 -0700 (MST) From: Terry Lambert Message-Id: <199810132358.QAA18137@usr08.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: gibbs@plutotech.com (Justin T. Gibbs) Date: Tue, 13 Oct 1998 23:58:59 +0000 (GMT) Cc: tlambert@primenet.com, gibbs@plutotech.com, Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810130705.BAA12205@pluto.plutotech.com> from "Justin T. Gibbs" at Oct 13, 98 00:59:05 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >> Ask Terry since he has stated that he 'doesn't have any drives with > >> non-bogus firmware'. > > > >A) Run soft updates > >B) Press "reset" occasionally > >C) Note any anomalies in the resulting fsck when the machine > > comes back up > >D) if count < 200, goto B > >E) if # of anomalies > 0, print "bad firmware". > > You're missing a large step here. You can't prove that the 'anomaly' > is related to the drive firmware without a trace of all transactions > on the SCSI bus. It could well be a missing dependency in the soft > update code. If I turn off write caching on the drive, and repeat the test, and the evaluation (E) results in a "# of anomalies == 0", where with write caching enabled, the number was > 0, then I can say with high confidence that it's the write caching. This is the experiment Don Lewis ran. > I'd be more than happy to reproduce your failure scenario > while recording a SCSI bus trace so that the fault is easy to interpret. > Just send me any *modern* drive that you think fails. Sure; just define "modern" for me, since my personal definition is "not IDE". > You should also ensure that your reset button does not cause any power > spikes on the drive power lines. That would be cheating. It doesn't, since "# of anomalies == 0" with write caching disabled. > >It's very hard to do this in software, without providing a mechanism > >to actually break into the latency link between the drive reporting > >a write cached operation has been written, and the actual writing. > > If you can cause this a failure to occur by hitting your reset button, I > should be able to cause it to occur by using a paper-clip if the reset > condition (cased by the SCSI card BIOS in the reset button case) is the > event that causes cache corruption. Both are non-deterministic methods of > error injection. I'm not very confident that this will break in at the fragile point in the transaction. > >Such a latency link only exists on drives which Justin has identified > >as having broken firmware due to the behaviour reported by Don Lewis. > > I'm still unclear as to whether Don was turning off power or hitting what I > consider the reset button. His comment about UPSes use makes me think he > was testing power outage scenarios. Well, I know that this might sound insane, but we could ask Don, and I could get out of the middle of this whole thing... ;-). > >I would be much more interested in knowing what drives and firmware > >revisions of those drives Justin has, since both mine and Don Lewis's > >are demonstrably broken. > > Since you were able to test 4 drives so quickly, I'd love to see well > documented information on exactly how the file system was inconsistent > in the failure cases. There were directory dependencies which were committed out of order (the modified fsck reports these as soft dependency errors...). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 17:08:13 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA27200 for freebsd-fs-outgoing; Tue, 13 Oct 1998 17:08:13 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id RAA27098; Tue, 13 Oct 1998 17:07:58 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id SAA02391; Tue, 13 Oct 1998 18:07:37 -0600 (MDT) Message-Id: <199810140007.SAA02391@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: gibbs@plutotech.com (Justin T. Gibbs), Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Tue, 13 Oct 1998 23:58:59 -0000." <199810132358.QAA18137@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Oct 1998 18:00:52 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >> You're missing a large step here. You can't prove that the 'anomaly' >> is related to the drive firmware without a trace of all transactions >> on the SCSI bus. It could well be a missing dependency in the soft >> update code. > >If I turn off write caching on the drive, and repeat the test, and >the evaluation (E) results in a "# of anomalies == 0", where with >write caching enabled, the number was > 0, then I can say with >high confidence that it's the write caching. I disagree. You have demonstrable varied the rate at which write completions as seen by FreeBSD occur which means that the test is anything but conclusive. >> I'd be more than happy to reproduce your failure scenario >> while recording a SCSI bus trace so that the fault is easy to interpret. >> Just send me any *modern* drive that you think fails. > >Sure; just define "modern" for me, since my personal definition is >"not IDE". A drive manufactured within the last 3 years. >> You should also ensure that your reset button does not cause any power >> spikes on the drive power lines. That would be cheating. > >It doesn't, since "# of anomalies == 0" with write caching disabled. This doesn't follow. If the cache is disabled, it doesn't matter if the drive loses power due to hitting the reset button. We already know that losing power on a drive that cached data will not work. >> I'm still unclear as to whether Don was turning off power or hitting what I >> consider the reset button. His comment about UPSes use makes me think he >> was testing power outage scenarios. > >Well, I know that this might sound insane, but we could ask Don, and >I could get out of the middle of this whole thing... ;-). Well, if your offering, I'd be more than happy to take you up on your offer. >> Since you were able to test 4 drives so quickly, I'd love to see well >> documented information on exactly how the file system was inconsistent >> in the failure cases. > >There were directory dependencies which were committed out of order >(the modified fsck reports these as soft dependency errors...). Can you be more specific? Are you positive that the transactions were committed out of order or could it be that some transactions were never committed at all? What was the size of the directory. Was the failure in directory creation or destruction? Which portion of the dependency graph was violated? -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 17:49:57 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA03896 for freebsd-fs-outgoing; Tue, 13 Oct 1998 17:49:57 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id RAA03888; Tue, 13 Oct 1998 17:49:55 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id RAA04271; Tue, 13 Oct 1998 17:49:40 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpd004237; Tue Oct 13 17:49:37 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA20004; Tue, 13 Oct 1998 17:49:31 -0700 (MST) From: Terry Lambert Message-Id: <199810140049.RAA20004@usr08.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: gibbs@plutotech.com (Justin T. Gibbs) Date: Wed, 14 Oct 1998 00:49:31 +0000 (GMT) Cc: tlambert@primenet.com, gibbs@plutotech.com, Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810140007.SAA02391@pluto.plutotech.com> from "Justin T. Gibbs" at Oct 13, 98 06:00:52 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >It doesn't, since "# of anomalies == 0" with write caching disabled. > > This doesn't follow. If the cache is disabled, it doesn't matter if > the drive loses power due to hitting the reset button. We already > know that losing power on a drive that cached data will not work. We do? If writes are committed in dependency order, and the write is cached and there is no reordering of subsequent writes (ie: writes occur in tag order, even if they are cached), then I think this satisifies soft updates. > >> I'm still unclear as to whether Don was turning off power or hitting what I > >> consider the reset button. His comment about UPSes use makes me think he > >> was testing power outage scenarios. > > > >Well, I know that this might sound insane, but we could ask Don, and > >I could get out of the middle of this whole thing... ;-). > > Well, if your offering, I'd be more than happy to take you up on your > offer. No need; he's spoken up: He was using the front panel reset, not power loss. > >> Since you were able to test 4 drives so quickly, I'd love to see well > >> documented information on exactly how the file system was inconsistent > >> in the failure cases. > > > >There were directory dependencies which were committed out of order > >(the modified fsck reports these as soft dependency errors...). > > Can you be more specific? Are you positive that the transactions > were committed out of order or could it be that some transactions > were never committed at all? Transactions were committed out of order. If the transactions that had been committed had been committed in order, then the loss of cached transactions would only result in a rewind of state of the drive to a previous state. The previous state, by definition, must also be consistent. In other words, transaction dependency order as required for soft updates must cause commits to the drives to occur in dependency order, and the problem is the reordering of consecutive write requests by the drive itself if there is *ever* a case where the state is ever inconsistent. The only alternative is a dependency order bug, and such a bug would be very easy to reproduce using a break-to-debugger or a reset+fsck. This is not the case, from all observable data, so it *must* be that the soft updates code is being given the go-ahead on another transaction before the previous transaction on which it depends has been committed to stable storage, and the drive is subsequently committing them out of order. There is no other explanation for a soft dependency inconsistency, so long as dependencies are both correctly modelled and enqueued (which we believe to be the case). > What was the size of the directory. For me, it was a large directory; it was the full X11R6 source tree from the XFree86 distribution (this is what Julian has used as one of his tests at Whistle, so I did the same at home). > Was the failure in directory creation or destruction? Which portion > of the dependency graph was violated? I would have to manually examine the data on the disk after getting the fsck report to answer this. Unfortunately, fsck is a tool for returning the disk to a known stable state, and if the dependency order is not enforced by synchronous writes and/or soft update dependency order being honored by the drive, then I would need to be able to intuit what the correct state should have been vs. what the current state was (in other words, guess the outstanding cached transactions remaining uncommitted, and determine if they would be rolled forward or backward by fsck). My *hunch* from what I was doing at the time (rm -rf) is "destruction", but since I started this before all the creations were committed to stable storage (only enqueued in the dependency queue), I can't really say for sure what operation was in effect when I hit the reset button. Maybe Don can elaborate on what he was doing when he hit reset? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 21:40:18 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA05997 for freebsd-fs-outgoing; Tue, 13 Oct 1998 21:40:18 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA05980; Tue, 13 Oct 1998 21:40:12 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id VAA09333; Tue, 13 Oct 1998 21:39:53 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id VAA23764; Tue, 13 Oct 1998 21:39:52 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id VAA17463; Tue, 13 Oct 1998 21:39:50 -0700 (PDT) From: Don Lewis Message-Id: <199810140439.VAA17463@salsa.gv.tsc.tdk.com> Date: Tue, 13 Oct 1998 21:39:50 -0700 In-Reply-To: "Justin T. Gibbs" "Re: filesystem safety and SCSI disk write caching" (Oct 13, 6:00pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: "Justin T. Gibbs" , Terry Lambert Subject: Re: filesystem safety and SCSI disk write caching Cc: Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 13, 6:00pm, "Justin T. Gibbs" wrote: } Subject: Re: filesystem safety and SCSI disk write caching } >> I'd be more than happy to reproduce your failure scenario } >> while recording a SCSI bus trace so that the fault is easy to interpret. } >> Just send me any *modern* drive that you think fails. } > } >Sure; just define "modern" for me, since my personal definition is } >"not IDE". } } A drive manufactured within the last 3 years. The drive in my experiment is: da0 at ahc0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI2 device da0: 10.0MB/s transfers (10.0MHz, offset 15), Tagged Queueing Enabled da0: 2049MB (4197405 512 byte sectors: 255H 63S/T 261C) I don't know when it was manufactured, but I've only had access to it for less than a year. } >> You should also ensure that your reset button does not cause any power } >> spikes on the drive power lines. That would be cheating. } > } >It doesn't, since "# of anomalies == 0" with write caching disabled. } } This doesn't follow. If the cache is disabled, it doesn't matter if } the drive loses power due to hitting the reset button. We already } know that losing power on a drive that cached data will not work. I didn't hear the sound of any mechanical things spinning down. The machine just started going through its boot sequence. } >> I'm still unclear as to whether Don was turning off power or hitting what I } >> consider the reset button. His comment about UPSes use makes me think he } >> was testing power outage scenarios. } > } >Well, I know that this might sound insane, but we could ask Don, and } >I could get out of the middle of this whole thing... ;-). } } Well, if your offering, I'd be more than happy to take you up on your } offer. I was only playing with the reset button. Justin was the first to mention an UPS as the solution. } >> Since you were able to test 4 drives so quickly, I'd love to see well } >> documented information on exactly how the file system was inconsistent } >> in the failure cases. } > } >There were directory dependencies which were committed out of order } >(the modified fsck reports these as soft dependency errors...). } } Can you be more specific? Are you positive that the transactions } were committed out of order or could it be that some transactions } were never committed at all? What was the size of the directory. } Was the failure in directory creation or destruction? Which portion } of the dependency graph was violated? The symptom was a directory entry that referenced an unallocated inode. When creating a new file (or adding a link), softupdates writes the new inode to disk (and waits for the driver to tell it the write is complete) before writing the block containing the new directory entry to disk. When doing an unlink, softupdates clears the directory entry, writes the directory block to disk, and waits for the driver to tell it the directory block has been written before writing the inode to disk (either cleared or just with decreased reference count as appropriate). If the writes that actually ended up on the platters were done in the correct order, the only inconsistency that could result from a failure to commit all the transactions would be an unreferenced inode on disk that might get reconnected under lost+found by fsck. The problem I saw indicates that the writes to the platters were done in the wrong order and not all of them were completed. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 22:05:32 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA09478 for freebsd-fs-outgoing; Tue, 13 Oct 1998 22:05:32 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA09454; Tue, 13 Oct 1998 22:05:28 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id WAA09465; Tue, 13 Oct 1998 22:05:09 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id WAA24188; Tue, 13 Oct 1998 22:05:08 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id WAA17506; Tue, 13 Oct 1998 22:05:07 -0700 (PDT) From: Don Lewis Message-Id: <199810140505.WAA17506@salsa.gv.tsc.tdk.com> Date: Tue, 13 Oct 1998 22:05:06 -0700 In-Reply-To: Terry Lambert "Re: filesystem safety and SCSI disk write caching" (Oct 14, 12:49am) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: Terry Lambert , gibbs@plutotech.com (Justin T. Gibbs) Subject: Re: filesystem safety and SCSI disk write caching Cc: Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 14, 12:49am, Terry Lambert wrote: } Subject: Re: filesystem safety and SCSI disk write caching } > >It doesn't, since "# of anomalies == 0" with write caching disabled. } > } > This doesn't follow. If the cache is disabled, it doesn't matter if } > the drive loses power due to hitting the reset button. We already } > know that losing power on a drive that cached data will not work. } } We do? I think we do. } If writes are committed in dependency order, and the write is cached } and there is no reordering of subsequent writes (ie: writes occur in } tag order, even if they are cached), then I think this satisifies soft } updates. I believe the situation is that if you have write caching enabled, the drive will reorder the writes and it will tell you that the writes are done before they have been really gotten all the way to the platters. This might allow softupdates to issue more writes that might actually end up on the platters before the writes that they are dependent on. If the host CPU panics, this is still ok, because the combination of the pending writes in the drive cache and the data on the platters is in a safe state from the standpoint of what softupdates expects, and if the drive is not perturbed (by shutting off its power or doing something else that causes it to fail to commit the pending writes to stable storage), all the pending writes will complete and the bits on the platters will eventually end up in a "good enough" if not consistent state. } > What was the size of the directory. } } For me, it was a large directory; it was the full X11R6 source tree } from the XFree86 distribution (this is what Julian has used as one } of his tests at Whistle, so I did the same at home). I don't know in my case. } > Was the failure in directory creation or destruction? Which portion } > of the dependency graph was violated? } My *hunch* from what I was doing at the time (rm -rf) is "destruction", } but since I started this before all the creations were committed to } stable storage (only enqueued in the dependency queue), I can't } really say for sure what operation was in effect when I hit the reset } button. Maybe Don can elaborate on what he was doing when he hit } reset? I was doing the usual buildworld. I don't remember if it was in a creation or destruction phase when I hit reset. This was a couple weeks ago. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 22:18:54 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA10728 for freebsd-fs-outgoing; Tue, 13 Oct 1998 22:18:54 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA10712; Tue, 13 Oct 1998 22:18:49 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id XAA15040; Tue, 13 Oct 1998 23:18:26 -0600 (MDT) Message-Id: <199810140518.XAA15040@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: gibbs@plutotech.com (Justin T. Gibbs), Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Wed, 14 Oct 1998 00:49:31 -0000." <199810140049.RAA20004@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Oct 1998 23:11:41 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >> >It doesn't, since "# of anomalies == 0" with write caching disabled. >> >> This doesn't follow. If the cache is disabled, it doesn't matter if >> the drive loses power due to hitting the reset button. We already >> know that losing power on a drive that cached data will not work. > >We do? Of course we do. Why do you think my position on this subject has always been, "Write caching is fine so long as you use a UPS"? >If writes are committed in dependency order, and the write is cached >and there is no reordering of subsequent writes (ie: writes occur in >tag order, even if they are cached), then I think this satisifies soft >updates. You have no guarantee that the writes will be committed to the media in the order that they were reported as completed to the host if you use write caching. You do have a guarantee, however, that the cache contents are always consistent and if you allow the drive to flush it's cache, the media will eventually be consistent as well. >> >There were directory dependencies which were committed out of order >> >(the modified fsck reports these as soft dependency errors...). >> >> Can you be more specific? Are you positive that the transactions >> were committed out of order or could it be that some transactions >> were never committed at all? > >Transactions were committed out of order. If the transactions that >had been committed had been committed in order, then the loss of cached >transactions would only result in a rewind of state of the drive to >a previous state. The previous state, by definition, must also be >consistent. This is consistent with losing transactions. Certainly the writes could have been written out of order, but the failure should only occur if the drive loses transactions that were considered complete by the host. This should only happen by way of a power loss or power spike. >In other words, transaction dependency order as required for soft >updates must cause commits to the drives to occur in dependency >order, and the problem is the reordering of consecutive write >requests by the drive itself if there is *ever* a case where the >state is ever inconsistent. This is why you must have a UPS. I thought we already went over this before... >The only alternative is a dependency order bug, and such a bug >would be very easy to reproduce using a break-to-debugger or a >reset+fsck. Only if your break or reset+fsck hits at exactly the right time. You've used this argument yourself. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 22:23:32 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA11369 for freebsd-fs-outgoing; Tue, 13 Oct 1998 22:23:32 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA11344; Tue, 13 Oct 1998 22:23:29 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id XAA15206; Tue, 13 Oct 1998 23:23:12 -0600 (MDT) Message-Id: <199810140523.XAA15206@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Don Lewis cc: "Justin T. Gibbs" , Terry Lambert , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Tue, 13 Oct 1998 21:39:50 PDT." <199810140439.VAA17463@salsa.gv.tsc.tdk.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Oct 1998 23:16:27 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >} This doesn't follow. If the cache is disabled, it doesn't matter if >} the drive loses power due to hitting the reset button. We already >} know that losing power on a drive that cached data will not work. > >I didn't hear the sound of any mechanical things spinning down. The >machine just started going through its boot sequence. The drive will reinitialize to the 'power on state' if the power fluctuates into a zone that might invalidate it's run-time state. It doesn't take a very long spike for the drive's power-glitch sensor to go off. In this case, dropping cached contents on the floor is much safer than attempting to continue from an unknown state. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 22:59:53 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA15235 for freebsd-fs-outgoing; Tue, 13 Oct 1998 22:59:53 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA15215; Tue, 13 Oct 1998 22:59:45 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id WAA09857; Tue, 13 Oct 1998 22:59:27 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id WAA25064; Tue, 13 Oct 1998 22:59:26 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id WAA17612; Tue, 13 Oct 1998 22:59:24 -0700 (PDT) From: Don Lewis Message-Id: <199810140559.WAA17612@salsa.gv.tsc.tdk.com> Date: Tue, 13 Oct 1998 22:59:24 -0700 In-Reply-To: "Justin T. Gibbs" "Re: filesystem safety and SCSI disk write caching" (Oct 13, 11:16pm) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: "Justin T. Gibbs" , Don Lewis Subject: Re: filesystem safety and SCSI disk write caching Cc: Terry Lambert , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 13, 11:16pm, "Justin T. Gibbs" wrote: } Subject: Re: filesystem safety and SCSI disk write caching } >} This doesn't follow. If the cache is disabled, it doesn't matter if } >} the drive loses power due to hitting the reset button. We already } >} know that losing power on a drive that cached data will not work. } > } >I didn't hear the sound of any mechanical things spinning down. The } >machine just started going through its boot sequence. } } The drive will reinitialize to the 'power on state' if the power fluctuates } into a zone that might invalidate it's run-time state. It doesn't take a } very long spike for the drive's power-glitch sensor to go off. In this } case, dropping cached contents on the floor is much safer than attempting } to continue from an unknown state. If that's the reason for the problem that I saw, then the UPS the system was plugged into wasn't sufficient to prevent the problem. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Oct 13 23:08:59 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id XAA16607 for freebsd-fs-outgoing; Tue, 13 Oct 1998 23:08:59 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id XAA16591; Tue, 13 Oct 1998 23:08:56 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id AAA16953; Wed, 14 Oct 1998 00:08:38 -0600 (MDT) Message-Id: <199810140608.AAA16953@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Don Lewis cc: "Justin T. Gibbs" , Terry Lambert , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Tue, 13 Oct 1998 22:59:24 PDT." <199810140559.WAA17612@salsa.gv.tsc.tdk.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 00:01:53 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >} The drive will reinitialize to the 'power on state' if the power fluctuates >} into a zone that might invalidate it's run-time state. It doesn't take a >} very long spike for the drive's power-glitch sensor to go off. In this >} case, dropping cached contents on the floor is much safer than attempting >} to continue from an unknown state. > >If that's the reason for the problem that I saw, then the UPS the >system was plugged into wasn't sufficient to prevent the problem. Why is that? Do you have gremlins walking around hitting the reset buttons on your machines? The UPS should isolate the machine from any drop in power that would cause it to lose its brain other than that caused by a hardware failure or an administrator hitting the reset or power switch. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 07:01:30 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA26014 for freebsd-fs-outgoing; Wed, 14 Oct 1998 07:01:30 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from magicnet.magicnet.net (magicnet.magicnet.net [204.96.116.9]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA26007 for ; Wed, 14 Oct 1998 07:01:26 -0700 (PDT) (envelope-from bill@bilver.magicnet.net) Received: (from root@localhost) by magicnet.magicnet.net (8.8.6/8.8.8) with UUCP id KAA20978 for freebsd-fs@freebsd.org; Wed, 14 Oct 1998 10:00:05 -0400 (EDT) Received: (from bill@localhost) by bilver.magicnet.net (8.9.1/8.9.1) id JAA10202 for freebsd-fs@freebsd.org; Wed, 14 Oct 1998 09:48:52 -0400 (EDT) From: Bill Vermillion Message-Id: <199810141348.JAA10202@bilver.magicnet.net> Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810140518.XAA15040@pluto.plutotech.com> from "Justin T. Gibbs" at "Oct 13, 98 11:11:41 pm" To: freebsd-fs@FreeBSD.ORG Date: Wed, 14 Oct 1998 09:48:52 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Justin T. Gibbs recently said: > >> >It doesn't, since "# of anomalies == 0" with write caching > >> >disabled. > >> > >> This doesn't follow. If the cache is disabled, it doesn't > >> matter if the drive loses power due to hitting the reset > >> button. We already know that losing power on a drive that > >> cached data will not work. > >We do? > Of course we do. Why do you think my position on this subject has > always been, "Write caching is fine so long as you use a UPS"? There was at least one model of IBM's high-end SCSI drives which when power was lost, would use the inertia from the spinning platters to generate current from the motor, in order to flush the on-board drive cache. Anyone recall that model number? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 08:02:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA03032 for freebsd-fs-outgoing; Wed, 14 Oct 1998 08:02:20 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from ns.mt.sri.com (sri-gw.MT.net [206.127.105.141]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA03010; Wed, 14 Oct 1998 08:02:15 -0700 (PDT) (envelope-from nate@mt.sri.com) Received: from mt.sri.com (rocky.mt.sri.com [206.127.76.100]) by ns.mt.sri.com (8.8.8/8.8.8) with SMTP id JAA05240; Wed, 14 Oct 1998 09:01:47 -0600 (MDT) (envelope-from nate@rocky.mt.sri.com) Received: by mt.sri.com (SMI-8.6/SMI-SVR4) id JAA04936; Wed, 14 Oct 1998 09:01:46 -0600 Date: Wed, 14 Oct 1998 09:01:46 -0600 Message-Id: <199810141501.JAA04936@mt.sri.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Justin T. Gibbs" Cc: Terry Lambert , Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810140518.XAA15040@pluto.plutotech.com> References: <199810140049.RAA20004@usr08.primenet.com> <199810140518.XAA15040@pluto.plutotech.com> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >If writes are committed in dependency order, and the write is cached > >and there is no reordering of subsequent writes (ie: writes occur in > >tag order, even if they are cached), then I think this satisifies soft > >updates. > > You have no guarantee that the writes will be committed to the media in the > order that they were reported as completed to the host if you use write > caching. You do have a guarantee, however, that the cache contents are > always consistent and if you allow the drive to flush it's cache, the > media will eventually be consistent as well. ... > This is why you must have a UPS. I thought we already went over this > before... Bottom line is that by default FreeBSD w/SoftUpdates is more *unstable* now with CAM than it was w/out CAM for 99% of the users. You can argue all day long that people have broken firmware, crappy power supplies, or sun spots, but the bottom line is that CAM is making it so people's file systems are hosed when they don't have to be. Architecturally it would be nice if everyone had a UPS, a great power supply that didn't 'spike' the power line, and perfect firmware but this isn't a perfect world. Far from it. We live in an imperfect world and that won't change just because CAM likes perfection. IMO, CAM should disable write caching by default, and allow people to add it back by hand if they know how. I don't know how this would be done, but it's *ALWAYS* a better idea to be safe than to be sorry. Never optimize when the optimizations are a pessisimization for most of the users. Nate ps. I have both good firmware and a appropriately configured UPS that correctly powers down the system, so don't be yelling at me for having bad hardware. But, I'm an exception to the norm, in many ways. :) :) :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 08:09:59 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA04192 for freebsd-fs-outgoing; Wed, 14 Oct 1998 08:09:59 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA04169; Wed, 14 Oct 1998 08:09:50 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id JAA07471; Wed, 14 Oct 1998 09:09:26 -0600 (MDT) Message-Id: <199810141509.JAA07471@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Nate Williams cc: "Justin T. Gibbs" , Terry Lambert , Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Wed, 14 Oct 1998 09:01:46 MDT." <199810141501.JAA04936@mt.sri.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 09:02:41 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Bottom line is that by default FreeBSD w/SoftUpdates is more *unstable* >now with CAM than it was w/out CAM for 99% of the users. Please don't add any more FUD to this thread than there is already. CAM has nothing to do with this issue. Neither the old SCSI code nor the new CAM code has ever modified the caching behavior of the devices attached on the bus. My position on this is that we should *never* modify device mode parameters unless instructed to do so by the end user. If the user community insists that we add a warning about the cache being enabled, now, after years of silently ignoring the effects of the cache on filesystem integrity, so be it. My opinion is that if this was a problem in practice, we'd have heard about it by now from our user community. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 08:19:21 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA05550 for freebsd-fs-outgoing; Wed, 14 Oct 1998 08:19:21 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from ns.mt.sri.com (sri-gw.MT.net [206.127.105.141]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA05529; Wed, 14 Oct 1998 08:19:18 -0700 (PDT) (envelope-from nate@mt.sri.com) Received: from mt.sri.com (rocky.mt.sri.com [206.127.76.100]) by ns.mt.sri.com (8.8.8/8.8.8) with SMTP id JAA05367; Wed, 14 Oct 1998 09:18:57 -0600 (MDT) (envelope-from nate@rocky.mt.sri.com) Received: by mt.sri.com (SMI-8.6/SMI-SVR4) id JAA05051; Wed, 14 Oct 1998 09:18:56 -0600 Date: Wed, 14 Oct 1998 09:18:56 -0600 Message-Id: <199810141518.JAA05051@mt.sri.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Justin T. Gibbs" Cc: Nate Williams , Terry Lambert , Don.Lewis@tsc.tdk.com, julian@whistle.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810141509.JAA07471@pluto.plutotech.com> References: <199810141501.JAA04936@mt.sri.com> <199810141509.JAA07471@pluto.plutotech.com> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >Bottom line is that by default FreeBSD w/SoftUpdates is more *unstable* > >now with CAM than it was w/out CAM for 99% of the users. > > Please don't add any more FUD to this thread than there is already. > > CAM has nothing to do with this issue. Neither the old SCSI code nor the > new CAM code has ever modified the caching behavior of the devices attached > on the bus. Previous email imply that it has the ability to disable/enable write-caching the addition of quirks entries for certain devices. > My position on this is that we should *never* modify device > mode parameters unless instructed to do so by the end user. If the user > community insists that we add a warning about the cache being enabled, now, > after years of silently ignoring the effects of the cache on filesystem > integrity, so be it. My opinion is that if this was a problem in practice, > we'd have heard about it by now from our user community. Up till this point, we never had any FS code that attempted to deal with 'crashing' robustly. I for one knew that if my machine crashed, an fsck was expected and it was going to repair my disk. This is no longer the case with SoftUpdates, so what was previously attributed to 'the crash' can now be more fine-tuned to 'write caching screwed up' or 'I have a bad power supply which breaks my drive when I hit the reset button' or even simply 'when I lose power, write-caching spams my disk'. With better information more informed decisions can be made. (Now is that a redundant statement or what? *grin*) Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 11:16:00 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA01926 for freebsd-fs-outgoing; Wed, 14 Oct 1998 11:16:00 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA01907; Wed, 14 Oct 1998 11:15:56 -0700 (PDT) (envelope-from julian@whistle.com) Received: (from daemon@localhost) by alpo.whistle.com (8.8.5/8.8.5) id LAA08932; Wed, 14 Oct 1998 11:08:18 -0700 (PDT) Received: from current1.whistle.com(207.76.205.22) via SMTP by alpo.whistle.com, id smtpdXn8918; Wed Oct 14 18:08:11 1998 Date: Wed, 14 Oct 1998 11:08:05 -0700 (PDT) From: Julian Elischer To: "Justin T. Gibbs" cc: Terry Lambert , Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810140518.XAA15040@pluto.plutotech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hey everybody.. look, we're in violent agreement here but haven't noticed it.... 1/ Soft updates produces a consistent view of the filesystem at the level where reported completions are generated. 2/ If there is another layer below that (a write cache) then if the transactions are re-ordered in that layer, then lower layers may not get a consistent view of the filesystem at every instant. 3/ Given time and power, the layer that generates the completion reports will sync down to the lower layers making them consistent. Lack of either will have the lower layers in an inconsitent state. 4/ Drives that do not sync down after there is a scsi reset are in danger of producing corrupted filesystems after a warm reboot. This should be considered a firmware bug. 5/ Systems that run without UPS, or want to be able to withstand more drastic failures (e.g. PSU explodes) should run without write caching, or should ensure that writes to media occur in the same order as completions are generated (not neccesarily the same as write requests as they could be re-ordered before the completions are issued). Since the latter is unlikely, the former is preferable. 6/ Since Softupdates makes all writes asychronous, the penalty for turning off write caching is very minimal. My experiments show a 1% difference in normal operation. THere are some applications however where this is not true, and these application should not consider soft updates alone to be a guarantee of FS consitency. They should have some way of guaranteeing 'time and power' (and good firmware). 7/ to allow for this to be achieved easily, there should be an easy way to ensure that the write cache is turned off. Possibly as simple as a good example in camctl.8 . julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 11:53:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA08212 for freebsd-fs-outgoing; Wed, 14 Oct 1998 11:53:34 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatekeeper.tsc.tdk.com (gatekeeper.tsc.tdk.com [207.113.159.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA08194; Wed, 14 Oct 1998 11:53:31 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from sunrise.gv.tsc.tdk.com (root@sunrise.gv.tsc.tdk.com [192.168.241.191]) by gatekeeper.tsc.tdk.com (8.8.8/8.8.8) with ESMTP id LAA21952; Wed, 14 Oct 1998 11:53:01 -0700 (PDT) (envelope-from gdonl@tsc.tdk.com) Received: from salsa.gv.tsc.tdk.com (salsa.gv.tsc.tdk.com [192.168.241.194]) by sunrise.gv.tsc.tdk.com (8.8.5/8.8.5) with ESMTP id LAA07794; Wed, 14 Oct 1998 11:53:00 -0700 (PDT) Received: (from gdonl@localhost) by salsa.gv.tsc.tdk.com (8.8.5/8.8.5) id LAA19279; Wed, 14 Oct 1998 11:52:59 -0700 (PDT) From: Don Lewis Message-Id: <199810141852.LAA19279@salsa.gv.tsc.tdk.com> Date: Wed, 14 Oct 1998 11:52:59 -0700 In-Reply-To: "Justin T. Gibbs" "Re: filesystem safety and SCSI disk write caching" (Oct 14, 12:01am) X-Mailer: Mail User's Shell (7.2.6 alpha(3) 7/19/95) To: "Justin T. Gibbs" , Don Lewis Subject: Re: filesystem safety and SCSI disk write caching Cc: Terry Lambert , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Oct 14, 12:01am, "Justin T. Gibbs" wrote: } Subject: Re: filesystem safety and SCSI disk write caching } >} The drive will reinitialize to the 'power on state' if the power fluctuates } >} into a zone that might invalidate it's run-time state. It doesn't take a } >} very long spike for the drive's power-glitch sensor to go off. In this } >} case, dropping cached contents on the floor is much safer than attempting } >} to continue from an unknown state. } > } >If that's the reason for the problem that I saw, then the UPS the } >system was plugged into wasn't sufficient to prevent the problem. } } Why is that? Do you have gremlins walking around hitting the reset } buttons on your machines? The UPS should isolate the machine from } any drop in power that would cause it to lose its brain other than } that caused by a hardware failure or an administrator hitting the } reset or power switch. >From my point of view, I can cut the run time of something like "make buildworld" by about 25 percent by turning on softupdates. If I disable write caching on the drive, it increases the time by about 1 percent, but I don't have to worry about filesystem corruption caused by: gremlins hitting the reset button power glitches that cause the drive to forget uncommitted transactions firmware bugs that cause the drive to forget uncommitted transactions when the drive sees a SCSI reset the power supply fuse blowing the power cord getting knocked loose the UPS shutting down before the system does a clean shutdown Having an UPS and wiring it so that the system does a clean shutdown before the UPS shuts down is pretty desirable, but there are enough other common failure modes that can result in a corrupted filesystem if write caching is enabled that I don't feel it's worth the minimal performance gain that I see. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 14:54:37 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA03107 for freebsd-fs-outgoing; Wed, 14 Oct 1998 14:54:37 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from quark.ChrisBowman.com (crbowman.erols.com [209.122.47.155]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA03089; Wed, 14 Oct 1998 14:54:31 -0700 (PDT) (envelope-from crb@ChrisBowman.com) Received: from fermion (fermion [10.0.1.2]) by quark.ChrisBowman.com (8.8.8/8.8.8) with SMTP id SAA17170; Wed, 14 Oct 1998 18:13:35 -0500 (EST) (envelope-from crb@ChrisBowman.com) Message-Id: <199810142313.SAA17170@quark.ChrisBowman.com> X-Sender: crb@quark X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.1 Date: Wed, 13 May 1998 17:54:00 -0400 To: Julian Elischer From: "Christopher R. Bowman" Subject: Re: filesystem safety and SCSI disk write caching Cc: "Justin T. Gibbs" , Terry Lambert , Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: References: <199810140518.XAA15040@pluto.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 11:08 AM 10/14/98 -0700, Julian Elischer wrote: >7/ to allow for this to be achieved easily, there should be an easy way to >ensure that the write cache is turned off. Possibly as simple as >a good example in camctl.8 . > > >julian Could we make this a mount time option, say if -wc to turn write caching on, -nowc to turn it off, and if neither flag is present use whatever the drive is already set for. -------- Christopher R. Bowman crb@ChrisBowman.com http://www.ChrisBowman.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 14:59:26 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA03703 for freebsd-fs-outgoing; Wed, 14 Oct 1998 14:59:26 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA03687; Wed, 14 Oct 1998 14:59:23 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id PAA00513; Wed, 14 Oct 1998 15:58:31 -0600 (MDT) Message-Id: <199810142158.PAA00513@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "Christopher R. Bowman" cc: Julian Elischer , "Justin T. Gibbs" , Terry Lambert , Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Wed, 13 May 1998 17:54:00 EDT." <199810142313.SAA17170@quark.ChrisBowman.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 15:51:41 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Could we make this a mount time option, say if -wc to turn write caching on, >-nowc to turn it off, and if neither flag is present use whatever the drive is >already set for. There is no facility to pass mount flags all the way down to a device driver. This also is not a per mount instance type of feature since several mountable partitions may reside on the same device. CAM currently lacks code to modify mode pages and maintain those modifications across bus reset and BDR events. This could be added, but it would be IMHO lots of complexity for limited gain. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 15:29:03 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA08032 for freebsd-fs-outgoing; Wed, 14 Oct 1998 15:29:03 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from dt053nb4.san.rr.com (dt053nb4.san.rr.com [204.210.34.180]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA07994; Wed, 14 Oct 1998 15:28:39 -0700 (PDT) (envelope-from Studded@gorean.org) Received: from gorean.org (Studded@localhost [127.0.0.1]) by dt053nb4.san.rr.com (8.8.8/8.8.8) with ESMTP id PAA08969; Wed, 14 Oct 1998 15:28:11 -0700 (PDT) (envelope-from Studded@gorean.org) Message-ID: <3625257B.B7C3B935@gorean.org> Date: Wed, 14 Oct 1998 15:28:11 -0700 From: Studded Organization: Triborough Bridge & Tunnel Authority X-Mailer: Mozilla 4.5b2 [en] (X11; I; FreeBSD 2.2.7-STABLE-1009 i386) X-Accept-Language: en MIME-Version: 1.0 To: "Justin T. Gibbs" CC: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching References: <199810141509.JAA07471@pluto.plutotech.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org "Justin T. Gibbs" wrote: > If the user > community insists that we add a warning about the cache being enabled, now, > after years of silently ignoring the effects of the cache on filesystem > integrity, so be it. My opinion is that if this was a problem in practice, > we'd have heard about it by now from our user community. Which user community are you referring to? Has this write caching always been present, or was it introduced by CAM? If you're referring to the community of all freebsd users, I think your statement is reasonable. If you are referring to the community of CAM users, that is an extremely small subset of freebsd's installed base, so expecting serious problems introduced in CAM to have all cropped up now is not reasonable. Please note that nothing in this post should be construed as critical to the CAM team. It's a huge undertaking and all signs are that it will be a very beneficial addition to freebsd. Doug -- *** Chief Operations Officer, DALnet IRC network *** Go PADRES! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 15:55:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA12559 for freebsd-fs-outgoing; Wed, 14 Oct 1998 15:55:06 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA12538; Wed, 14 Oct 1998 15:55:03 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id QAA04647; Wed, 14 Oct 1998 16:54:39 -0600 (MDT) Message-Id: <199810142254.QAA04647@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Studded cc: "Justin T. Gibbs" , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Wed, 14 Oct 1998 15:28:11 PDT." <3625257B.B7C3B935@gorean.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 16:47:49 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Which user community are you referring to? All of FreeBSD. >Has this write caching always been present, or was it introduced by CAM? It has always been present. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 16:01:43 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA13777 for freebsd-fs-outgoing; Wed, 14 Oct 1998 16:01:43 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA13761; Wed, 14 Oct 1998 16:01:40 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id RAA06139; Wed, 14 Oct 1998 17:01:12 -0600 (MDT) Message-Id: <199810142301.RAA06139@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: gibbs@plutotech.com (Justin T. Gibbs), Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Wed, 14 Oct 1998 22:40:09 -0000." <199810142240.PAA01660@usr04.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 16:54:22 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >I think the moral of this story is that you can't rely on UPS's, >and instead should only rely on single-state-transition based >finite state automatons if you need to be able to rely on anything. The moral of this story is that everyone should decide what kinds of performance/safety tradeoffs they are willing to make and design their systems accordingly. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 18:07:11 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA01670 for freebsd-fs-outgoing; Wed, 14 Oct 1998 18:07:11 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from mail.HiWAAY.net (fly.HiWAAY.net [208.147.154.56]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA01655; Wed, 14 Oct 1998 18:07:09 -0700 (PDT) (envelope-from dkelly@n4hhe.ampr.org) Received: from nospam.hiwaay.net (tnt1-47.HiWAAY.net [208.147.147.47]) by mail.HiWAAY.net (8.9.0/8.9.0) with ESMTP id UAA31706; Wed, 14 Oct 1998 20:06:50 -0500 (CDT) Received: from n4hhe.ampr.org (localhost.ampr.org [127.0.0.1]) by nospam.hiwaay.net (8.8.8/8.8.8) with ESMTP id TAA12721; Wed, 14 Oct 1998 19:15:36 -0500 (CDT) (envelope-from dkelly@n4hhe.ampr.org) Message-Id: <199810150015.TAA12721@nospam.hiwaay.net> X-Mailer: exmh version 2.0.2 2/24/98 To: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG From: David Kelly Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Message from Don Lewis of "Tue, 13 Oct 1998 22:59:24 PDT." <199810140559.WAA17612@salsa.gv.tsc.tdk.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 19:15:36 -0500 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Don Lewis writes: > On Oct 13, 11:16pm, "Justin T. Gibbs" wrote: > } Subject: Re: filesystem safety and SCSI disk write caching > } >} This doesn't follow. If the cache is disabled, it doesn't matter if > } >} the drive loses power due to hitting the reset button. We already > } >} know that losing power on a drive that cached data will not work. > } > > } >I didn't hear the sound of any mechanical things spinning down. The > } >machine just started going through its boot sequence. > } > } The drive will reinitialize to the 'power on state' if the power fluctuates > } into a zone that might invalidate it's run-time state. It doesn't take a > } very long spike for the drive's power-glitch sensor to go off. In this > } case, dropping cached contents on the floor is much safer than attempting > } to continue from an unknown state. > > If that's the reason for the problem that I saw, then the UPS the > system was plugged into wasn't sufficient to prevent the problem. Before you fight it too much more, replace the power supply. I've cured a number of "impossible" problems with a new power supply. One spectacular example was a Power Mac 7200/120. Crash, crash, crash. Sometimes it would run for 30 minutes. Sometimes overnight. Technician replaced everything several times over a couple of weeks. Everything but the plastic case and the power supply. I insisted on a new PS the last time back. And it worked like a charm. Power supply filter capacitors age with heat. And lose their ability to be good capacitors. No telling what kind of noise is on your DC power wires inside the case. Your PS could be generating a spike of its own on RESET when/if something suddenly demands a lot of current. Or if something suddenly quits demanding the current it was using. Switching power supplies need to be able to run thru a missed cycle or so. Or at least for 4 milliseconds. Inexpensive UPS's are "on-demand" and can take as long as 4 milliseconds to detect dropped power and come to the rescue. Otherwise on-demand UPS's are doing nothing. They might be acting as "surge suppressors". Bought a Best Power Inc, FerrUPS 700 a couple of years ago at a hamfest for $125. This UPS uses a ferro-resonant autotransformer and is online 100%. My voltmeter says its putting out 125VAC while my line voltage is 119. It also keeps my feet warm. -- David Kelly N4HHE, dkelly@nospam.hiwaay.net ===================================================================== The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 18:07:15 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA01701 for freebsd-fs-outgoing; Wed, 14 Oct 1998 18:07:15 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from mail.HiWAAY.net (fly.HiWAAY.net [208.147.154.56]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA01651; Wed, 14 Oct 1998 18:07:06 -0700 (PDT) (envelope-from dkelly@n4hhe.ampr.org) Received: from nospam.hiwaay.net (tnt1-47.HiWAAY.net [208.147.147.47]) by mail.HiWAAY.net (8.9.0/8.9.0) with ESMTP id UAA00627; Wed, 14 Oct 1998 20:06:48 -0500 (CDT) Received: from n4hhe.ampr.org (localhost.ampr.org [127.0.0.1]) by nospam.hiwaay.net (8.8.8/8.8.8) with ESMTP id TAA12754; Wed, 14 Oct 1998 19:32:40 -0500 (CDT) (envelope-from dkelly@n4hhe.ampr.org) Message-Id: <199810150032.TAA12754@nospam.hiwaay.net> X-Mailer: exmh version 2.0.2 2/24/98 To: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG From: David Kelly Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Message from Julian Elischer of "Wed, 14 Oct 1998 11:08:05 PDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 14 Oct 1998 19:32:40 -0500 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Julian Elischer writes: > Hey everybody.. > look, we're in violent agreement here but haven't noticed it.... [...] > 3/ Given time and power, the layer that generates the completion reports > will sync down to the lower layers making them consistent. Lack of either > will have the lower layers in an inconsitent state. > > 4/ Drives that do not sync down after there is a scsi reset are in danger > of producing corrupted filesystems after a warm reboot. This should be > considered a firmware bug. This reminds me of PowerMac thing that cropped up a couple of years ago. MacUser and MacWorld published "comparison" tests of HD's and would happily rate Brand A's superior to Brand B's when both were the same HD simply because A's was faster in their benchmarks. Then people blindly bought the "superior" package over the other. So vendors shipped drives to maximize benchmark performance. Including enable of whatever write caching the drive could do internally. The problem that cropped up was that PowerMacs can turn themselves off. Including the internal HD. And on shutdown would turn themselves off so fast the HD lost unwritten cached data. About the last thing a Mac writes to a disk on shutdown is its clean flag. Users were being reminded on reboot that they were supposed to do a "Shutdown from the Special... menu" rather than yank the power. Eventually the driver writers (Apple's SCSI driver won't talk to non-Apple OEM'ed HD's) implemented a flush on final close. So the driver busy-waited until it thought the data was synced. If somebody is punching the CPU RESET, then there isn't much software can do to overcome deficiencies of firmware. Users with external HD's didn't have this "problem" because they had to turn off their HD's manually. -- David Kelly N4HHE, dkelly@nospam.hiwaay.net ===================================================================== The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Oct 14 21:20:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA29265 for freebsd-fs-outgoing; Wed, 14 Oct 1998 21:20:06 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA29178; Wed, 14 Oct 1998 21:19:56 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.9.1/8.9.1) id XAA18802; Wed, 14 Oct 1998 23:19:26 -0500 (CDT) Date: Wed, 14 Oct 1998 23:19:25 -0500 From: Dan Nelson To: Julian Elischer , "Justin T. Gibbs" Cc: Terry Lambert , Don.Lewis@tsc.tdk.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching Message-ID: <19981014231925.A18446@emsphone.com> References: <199810140518.XAA15040@pluto.plutotech.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=HlL+5n6rz5pIUxbD X-Mailer: Mutt 0.94.3i In-Reply-To: ; from "Julian Elischer" on Wed Oct 14 11:08:05 GMT 1998 X-OS: FreeBSD 2.2.7-STABLE Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii In the last episode (Oct 14), Julian Elischer said: > Hey everybody.. > look, we're in violent agreement here but haven't noticed it.... > > 7/ to allow for this to be achieved easily, there should be an easy > way to ensure that the write cache is turned off. Possibly as simple > as a good example in camctl.8 . I humbly submit the following script, to be added to /etc/security, or periodic/weekly, /etc/rc, or wherever. It's dependant on the exact output of "camcontrol inquiry" and "camcontrol modepage", but does the job. -Dan Nelson dnelson@emsphone.com --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=cachecheck #! /bin/sh echo Checking SCSI drives for write-cache: numbad=0 # get a list of direct-access devices known to the system units=$(camcontrol devlist | sed -n -e "/,da[0-9]*/s/.*,da\(.*\))/\1/p") for i in $units do result=$(camcontrol modepage -m 8 -P 3 -u $i 2> /dev/null | grep WCE) if [ "$result" = "WCE: 1 " ] ; then camcontrol devlist | grep ",da${i})" numbad=$(($numbad + 1)) fi done if [ $numbad -gt 0 ] ; then s= [ $numbad -ne 1 ] && s=s echo " $numbad device$s with WCE set. To reset, run \"camcontrol modepage -e -P 3 -m 8 -u \" where is the disk number (i.e. da0 is unit #0), and set WCE to 0." fi echo --HlL+5n6rz5pIUxbD-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Oct 15 01:13:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id BAA21413 for freebsd-fs-outgoing; Thu, 15 Oct 1998 01:13:34 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id BAA21397; Thu, 15 Oct 1998 01:13:32 -0700 (PDT) (envelope-from julian@whistle.com) Received: (from daemon@localhost) by alpo.whistle.com (8.8.5/8.8.5) id BAA03171; Thu, 15 Oct 1998 01:10:45 -0700 (PDT) Received: from current1.whistle.com(207.76.205.22) via SMTP by alpo.whistle.com, id smtpdze3167; Thu Oct 15 08:10:40 1998 Date: Thu, 15 Oct 1998 01:10:37 -0700 (PDT) From: Julian Elischer To: "Justin T. Gibbs" cc: Studded , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <199810142254.QAA04647@pluto.plutotech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The reason it comes up now is because the promise of soft updates to make a 'safer and faster filesystem' is threatenned by it. Inthe pas the 'sync' filesystem was known to have problem with crashes. and everybody just took that in their stride and didn't try find a reason for it. However with soft updates, there is a theoretical behaviour that is easy to understand and when a filesystem doesn't behae that way it become obvious.. the difference between 45 filesystem eros and 47 is not that great. The difference between 0 and 2 is much more noticeable, even though it's still only 2 errors. These problems have probably always occured. they were just not so obvious. On Wed, 14 Oct 1998, Justin T. Gibbs wrote: > > Which user community are you referring to? > > All of FreeBSD. > > >Has this write caching always been present, or was it introduced by CAM? > > It has always been present. > > -- > Justin > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-scsi" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Oct 15 11:22:30 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA04648 for freebsd-fs-outgoing; Thu, 15 Oct 1998 11:22:30 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA04633; Thu, 15 Oct 1998 11:22:24 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id LAA21062; Thu, 15 Oct 1998 11:22:02 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp03.primenet.com, id smtpd021045; Thu Oct 15 11:21:58 1998 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id LAA15497; Thu, 15 Oct 1998 11:21:55 -0700 (MST) From: Terry Lambert Message-Id: <199810151821.LAA15497@usr02.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: dkelly@hiwaay.net (David Kelly) Date: Thu, 15 Oct 1998 18:21:55 +0000 (GMT) Cc: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810150015.TAA12721@nospam.hiwaay.net> from "David Kelly" at Oct 14, 98 07:15:36 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > If that's the reason for the problem that I saw, then the UPS the > > system was plugged into wasn't sufficient to prevent the problem. > > Before you fight it too much more, replace the power supply. I've cured > a number of "impossible" problems with a new power supply. Uh, you won't cure "Don punching the reset button to simulate a particular set of hardware failures" with a new supply. 8-). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Oct 16 04:09:43 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id EAA14680 for freebsd-fs-outgoing; Fri, 16 Oct 1998 04:09:43 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from mail.HiWAAY.net (fly.HiWAAY.net [208.147.154.56]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id EAA14672; Fri, 16 Oct 1998 04:09:39 -0700 (PDT) (envelope-from dkelly@n4hhe.ampr.org) Received: from nospam.hiwaay.net (tnt4-28.HiWAAY.net [208.166.127.28]) by mail.HiWAAY.net (8.9.0/8.9.0) with ESMTP id GAA30422; Fri, 16 Oct 1998 06:09:18 -0500 (CDT) Received: from n4hhe.ampr.org (localhost.ampr.org [127.0.0.1]) by nospam.hiwaay.net (8.8.8/8.8.8) with ESMTP id GAA16802; Fri, 16 Oct 1998 06:09:15 -0500 (CDT) (envelope-from dkelly@n4hhe.ampr.org) Message-Id: <199810161109.GAA16802@nospam.hiwaay.net> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG From: David Kelly Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Message from Terry Lambert of "Thu, 15 Oct 1998 18:21:55 -0000." <199810151821.LAA15497@usr02.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 16 Oct 1998 06:09:15 -0500 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Terry Lambert writes: > > > If that's the reason for the problem that I saw, then the UPS the > > > system was plugged into wasn't sufficient to prevent the problem. > > > > Before you fight it too much more, replace the power supply. I've cured > > a number of "impossible" problems with a new power supply. > > Uh, you won't cure "Don punching the reset button to simulate a > particular set of hardware failures" with a new supply. > > 8-). Why not? It might be interesting to put a recording voltmeter such as a digital storage oscilloscope on the HD power leads when Don is punching the reset. No telling what kind of voltage surges are generated when the load on the power supply is altered. No telling *if* there are changes in the PS load when reset is punched either. Think my PPro-166 CPU pulls 10A. If it suddenly stops pulling that much current when RESET is active then what does the PS do? -- David Kelly N4HHE, dkelly@nospam.hiwaay.net ===================================================================== The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Oct 16 12:58:49 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA28305 for freebsd-fs-outgoing; Fri, 16 Oct 1998 12:58:49 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA28285; Fri, 16 Oct 1998 12:58:41 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id MAA21313; Fri, 16 Oct 1998 12:58:22 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp01.primenet.com, id smtpd021284; Fri Oct 16 12:58:18 1998 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id MAA24426; Fri, 16 Oct 1998 12:58:17 -0700 (MST) From: Terry Lambert Message-Id: <199810161958.MAA24426@usr04.primenet.com> Subject: Re: filesystem safety and SCSI disk write caching To: dkelly@hiwaay.net (David Kelly) Date: Fri, 16 Oct 1998 19:58:17 +0000 (GMT) Cc: tlambert@primenet.com, freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG In-Reply-To: <199810161109.GAA16802@nospam.hiwaay.net> from "David Kelly" at Oct 16, 98 06:09:15 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Why not? It might be interesting to put a recording voltmeter such as a > digital storage oscilloscope on the HD power leads when Don is punching > the reset. No telling what kind of voltage surges are generated when > the load on the power supply is altered. > > No telling *if* there are changes in the PS load when reset is punched > either. Think my PPro-166 CPU pulls 10A. If it suddenly stops pulling > that much current when RESET is active then what does the PS do? The errors seen are a result of uncommitted data in the drive cache, not power spikes and gremlins. The interaction is well understood, and on firm footing unrelated to Stephan King novels. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Oct 16 16:50:44 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA17740 for freebsd-fs-outgoing; Fri, 16 Oct 1998 16:50:44 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA17567; Fri, 16 Oct 1998 16:49:54 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id RAA11679; Fri, 16 Oct 1998 17:49:25 -0600 (MDT) Message-Id: <199810162349.RAA11679@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert cc: dkelly@hiwaay.net (David Kelly), freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-reply-to: Your message of "Fri, 16 Oct 1998 19:58:17 -0000." <199810161958.MAA24426@usr04.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 16 Oct 1998 17:42:34 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >The errors seen are a result of uncommitted data in the drive cache, >not power spikes and gremlins. The interaction is well understood, >and on firm footing unrelated to Stephan King novels. And why do you think the drive didn't bother to commit the data even though power was constantly supplied to the drive and only a few, recent transactions were lost? Most likely because hitting the reset switch caused a power glitch that reverted the drive to its power on state. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Oct 17 00:56:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA10998 for freebsd-fs-outgoing; Sat, 17 Oct 1998 00:56:20 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from mailhost.rmci.net (mail.rmci.net [205.162.184.20]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id AAA10971 for ; Sat, 17 Oct 1998 00:56:09 -0700 (PDT) (envelope-from bcbent@az.rmci.net) Received: (qmail 29363 invoked from network); 17 Oct 1998 01:48:30 -0600 Received: from usr-az-15.rmci.net (HELO inficad.inficad.com) (209.210.32.15) by mail.rmci.net with SMTP; 17 Oct 1998 01:48:30 -0600 From: "Brian Jones" To: Subject: LIVE ENTERTAINMENT !!!!!!!!!!!!!!!! Date: Sat, 17 Oct 1998 00:33:36 -0700 Message-ID: <01bdf9a0$7367ada0$7520d2d1@inficad.inficad.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_031F_01BDF965.D286BAA0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.71.1712.3 X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This is a multi-part message in MIME format. ------=_NextPart_000_031F_01BDF965.D286BAA0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Come visit the next generation in adult entertainment. Where 35 of the = Internets most beautiful women are ready and willing to fulfill all of = your fantasies and desires. You can visit us 24 hours a day, 7 days a = week at; http://www.xoticfantasies.com IF YOU HAVE ANY COMPLAINTS REGARDING THIS SERVICE, OR YOU WOULD LIKE US = TO REMOVE YOUR NAME FROM OUR E-MAIL LIST, PLEASE CONTACT US IMMEDIATELY = AT 1-888-747-9191, OR YOU CAN E-MAIL US AT BCBENT@AZ.RMCI.NET=20 YOUR NAME WILL BE REMOVED IMMEDIATELY UPON CORRESPONDENCE! ------=_NextPart_000_031F_01BDF965.D286BAA0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Come visit the next generation in adult entertainment. = Where 35=20 of the Internets most beautiful women are ready and willing to fulfill = all of=20 your fantasies and desires. You can visit us 24 hours a day, 7 days a = week at;=20 http://www.xoticfantasies.com

IF YOU HAVE ANY COMPLAINTS REGARDING THIS SERVICE, OR = YOU WOULD=20 LIKE US TO REMOVE YOUR NAME FROM OUR E-MAIL LIST, PLEASE CONTACT US = IMMEDIATELY=20 AT 1-888-747-9191, OR YOU CAN E-MAIL US AT = BCBENT@AZ.RMCI.NET

YOUR NAME WILL BE REMOVED IMMEDIATELY UPON=20 CORRESPONDENCE!

------=_NextPart_000_031F_01BDF965.D286BAA0-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Oct 17 10:18:46 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA03922 for freebsd-fs-outgoing; Sat, 17 Oct 1998 10:18:46 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gvr.gvr.org (gvr.gvr.org [194.151.74.97]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id KAA03906; Sat, 17 Oct 1998 10:18:37 -0700 (PDT) (envelope-from guido@gvr.org) Received: (from guido@localhost) by gvr.gvr.org (8.8.8/8.8.5) id TAA13178; Sat, 17 Oct 1998 19:17:58 +0200 (MET DST) Message-ID: <19981017191758.A13174@gvr.org> Date: Sat, 17 Oct 1998 19:17:58 +0200 From: Guido van Rooij To: Terry Lambert , David Kelly Cc: freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching References: <199810161109.GAA16802@nospam.hiwaay.net> <199810161958.MAA24426@usr04.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.1i In-Reply-To: <199810161958.MAA24426@usr04.primenet.com>; from Terry Lambert on Fri, Oct 16, 1998 at 07:58:17PM +0000 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, Oct 16, 1998 at 07:58:17PM +0000, Terry Lambert wrote: > > The errors seen are a result of uncommitted data in the drive cache, > not power spikes and gremlins. The interaction is well understood, > and on firm footing unrelated to Stephan King novels. I always thought a drive will always be able to flush its write cache to disk, even when power fails. -Guido To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Oct 17 11:02:35 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA07550 for freebsd-fs-outgoing; Sat, 17 Oct 1998 11:02:35 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from magicnet.magicnet.net (magicnet.magicnet.net [204.96.116.9]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA07544 for ; Sat, 17 Oct 1998 11:02:34 -0700 (PDT) (envelope-from bill@bilver.magicnet.net) Received: (from uucp@localhost) by magicnet.magicnet.net (8.8.6/8.8.8) with UUCP id NAA20237 for freebsd-fs@freebsd.org; Sat, 17 Oct 1998 13:59:59 -0400 (EDT) Received: (from bill@localhost) by bilver.magicnet.net (8.9.1/8.9.1) id NAA26674 for freebsd-fs@freebsd.org; Sat, 17 Oct 1998 13:48:25 -0400 (EDT) From: Bill Vermillion Message-Id: <199810171748.NAA26674@bilver.magicnet.net> Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <19981017191758.A13174@gvr.org> from Guido van Rooij at "Oct 17, 98 07:17:58 pm" To: freebsd-fs@FreeBSD.ORG Date: Sat, 17 Oct 1998 13:48:25 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Guido van Rooij recently said: > On Fri, Oct 16, 1998 at 07:58:17PM +0000, Terry Lambert wrote: > > The errors seen are a result of uncommitted data in the drive cache, > > not power spikes and gremlins. The interaction is well understood, > > and on firm footing unrelated to Stephan King novels. > I always thought a drive will always be able to flush its write cache > to disk, even when power fails. Not all do. The high-end IBM's do/did. They used the inertia of the spinnging platters to generate enough current to flush the buffers to disk. There aren't a lot of drives that do it. With most drives, power off means driving in the dark. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Oct 17 19:03:54 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA18885 for freebsd-fs-outgoing; Sat, 17 Oct 1998 19:03:54 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA18866; Sat, 17 Oct 1998 19:03:52 -0700 (PDT) (envelope-from julian@whistle.com) Received: (from daemon@localhost) by alpo.whistle.com (8.8.5/8.8.5) id SAA08821; Sat, 17 Oct 1998 18:59:48 -0700 (PDT) Received: from current1.whistle.com(207.76.205.22) via SMTP by alpo.whistle.com, id smtpdwZ8817; Sun Oct 18 01:59:48 1998 Date: Sat, 17 Oct 1998 18:59:41 -0700 (PDT) From: Julian Elischer To: Guido van Rooij cc: Terry Lambert , David Kelly , freebsd-fs@FreeBSD.ORG, freebsd-scsi@FreeBSD.ORG Subject: Re: filesystem safety and SCSI disk write caching In-Reply-To: <19981017191758.A13174@gvr.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, 17 Oct 1998, Guido van Rooij wrote: > I always thought a drive will always be able to flush its write cache > to disk, even when power fails. no, That's a myth. > > -Guido > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-scsi" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message