From owner-freebsd-fs@FreeBSD.ORG Tue May 2 19:39:04 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3905F16A42C for ; Tue, 2 May 2006 19:39:04 +0000 (UTC) (envelope-from pho@holm.cc) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.FreeBSD.org (Postfix) with SMTP id 9773343D53 for ; Tue, 2 May 2006 19:39:03 +0000 (GMT) (envelope-from pho@holm.cc) Received: (qmail 21156 invoked from network); 2 May 2006 19:39:02 -0000 Received: from unknown (HELO peter.osted.lan) (unknown) by unknown with SMTP; 2 May 2006 19:39:02 -0000 X-pair-Authenticated: 83.95.197.184 Received: from peter.osted.lan (localhost.osted.lan [127.0.0.1]) by peter.osted.lan (8.13.4/8.13.4) with ESMTP id k42Jd1d0007696 for ; Tue, 2 May 2006 21:39:01 +0200 (CEST) (envelope-from pho@peter.osted.lan) Received: (from pho@localhost) by peter.osted.lan (8.13.4/8.13.4/Submit) id k42Jd1Ou007695 for fs@freebsd.org; Tue, 2 May 2006 21:39:01 +0200 (CEST) (envelope-from pho) Date: Tue, 2 May 2006 21:39:00 +0200 From: Peter Holm To: fs@freebsd.org Message-ID: <20060502193900.GA94069@peter.osted.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: Subject: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 19:39:05 -0000 I had a chance to look some more at how the UFS2 filesystem code handles a corrupt filesystem. I have made a web page describing the tests and my findings: http://people.freebsd.org/~pho/baddir.html My daytime job will probably prevent me from looking further at this any time soon, so if anyone finds this of interest I can make the corrupted filesystems available. -- Peter Holm From owner-freebsd-fs@FreeBSD.ORG Tue May 2 20:32:36 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5DEA716A404 for ; Tue, 2 May 2006 20:32:36 +0000 (UTC) (envelope-from freebsd-fs@merdin.com) Received: from smtp.mailix.net (smtp.mailix.net [216.148.213.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id D228243D49 for ; Tue, 2 May 2006 20:32:35 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from [62.148.144.242] (helo=[62.148.144.242]) by smtp.mailix.net with asmtp (Exim 4.24-CA) id 1Fb1Xr-0001Gs-Ge for freebsd-fs@freebsd.org; Tue, 02 May 2006 13:32:31 -0700 Date: Wed, 3 May 2006 00:32:29 +0400 From: Pavel Merdine X-Mailer: The Bat! (v3.71.04) Professional X-Priority: 3 (Normal) Message-ID: <1541458526.20060503003229@merdin.com> To: Peter Holm In-Reply-To: <20060502193900.GA94069@peter.osted.lan> References: <20060502193900.GA94069@peter.osted.lan> MIME-Version: 1.0 X-SA-Exim-Mail-From: freebsd-fs@merdin.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-SA-Exim-Version: 3.1 (built Thu Oct 23 13:26:47 PDT 2003) X-SA-Exim-Scanned: No; Unknown failure X-uvscan-result: clean (1Fb1Xr-0001Gs-Ge) Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 20:32:36 -0000 Hello , Thank you for raising this problem again. I already tried to do that in that list, but received an answer that kernel is intended to do that. For example, you have a faulty disk. And you have a faulty sector which happened to occur on the directory place. So each time kernel reads this sector it panics. So it's initially hard to even understand what happens. And also it leads to corruption and lost files on other file system (each time). Imagine if you have 15 disks. In this case you have many files lost just because of a small (and not significant) fault. It's just a nonsense. Personally, I just replaced bad_dir with error return. By the way, there was some bug in fs in kernel that could lead to panic even on clean filesystem (bad_dir as far as I remember). It is very rare and it was fixed on DragonFly. As far as I remember a fix for this was also commited to current recently. I think that Linux is usually much smarter on this. By default it remounts a file system as read-only in case it detects a filesystem corruption. I would be very happy if FreeBSD could do the same, because fs panics really hurt when you have many systems with disks. Of course I think we could do patches to overcome corrupting panics, but the core FreeBSD team would not accept this, as they are happy with panics and corruptions they make to other filesystems. Tuesday, May 2, 2006, 11:39:00 PM, you wrote: > I had a chance to look some more at how the UFS2 filesystem code > handles a corrupt filesystem. I have made a web page describing the > tests and my findings: > http://people.freebsd.org/~pho/baddir.html > My daytime job will probably prevent me from looking further at > this any time soon, so if anyone finds this of interest I can make > the corrupted filesystems available. -- / Pavel Merdine From owner-freebsd-fs@FreeBSD.ORG Tue May 2 22:13:10 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F2DF16A400 for ; Tue, 2 May 2006 22:13:10 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B24B43D46 for ; Tue, 2 May 2006 22:13:09 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 6D38D1A4DF6; Tue, 2 May 2006 15:13:09 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 6AC05515A6; Tue, 2 May 2006 18:13:07 -0400 (EDT) Date: Tue, 2 May 2006 18:13:07 -0400 From: Kris Kennaway To: Pavel Merdine Message-ID: <20060502221306.GD95348@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rz+pwK2yUstbofK6" Content-Disposition: inline In-Reply-To: <1541458526.20060503003229@merdin.com> User-Agent: Mutt/1.4.2.1i Cc: Peter Holm Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 22:13:10 -0000 --rz+pwK2yUstbofK6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: > Hello , >=20 > Thank you for raising this problem again. I already tried to do that > in that list, but received an answer that kernel is intended to do > that. For example, you have a faulty disk. And you have a faulty > sector which happened to occur on the directory place. So each time > kernel reads this sector it panics. So it's initially hard to even > understand what happens. And also it leads to corruption and lost > files on other file system (each time). Imagine if you have 15 disks. > In this case you have many files lost just because of a small (and not > significant) fault. It's just a nonsense. > Personally, I just replaced bad_dir with error return. > By the way, there was some bug in fs in kernel that could lead to > panic even on clean filesystem (bad_dir as far as I remember). It is > very rare and it was fixed on DragonFly. As far as I remember a fix > for this was also commited to current recently. >=20 > I think that Linux is usually much smarter on this. By default it > remounts a file system as read-only in case it detects a filesystem > corruption. I would be very happy if FreeBSD could do the same, > because fs panics really hurt when you have many systems with disks. >=20 > Of course I think we could do patches to overcome corrupting panics, > but the core FreeBSD team would not accept this, as they are happy > with panics and corruptions they make to other filesystems. Of course not, don't make silly accusations :-) The problem is much more difficult to solve than "making the panic an error return". Kris --rz+pwK2yUstbofK6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEV9lyWry0BWjoQKURAmKvAKCUqutakqkiSFOI9cE6smyEhLKg1QCggx0L sR1ctm8qukBIbheUzHr4Bac= =Fwc/ -----END PGP SIGNATURE----- --rz+pwK2yUstbofK6-- From owner-freebsd-fs@FreeBSD.ORG Wed May 3 00:21:45 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AD25A16A401 for ; Wed, 3 May 2006 00:21:45 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2502243D45 for ; Wed, 3 May 2006 00:21:42 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k430Lfmw042721; Tue, 2 May 2006 18:21:41 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4457F7A0.9050502@samsco.org> Date: Tue, 02 May 2006 18:21:52 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051230 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pavel Merdine References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> In-Reply-To: <1541458526.20060503003229@merdin.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: Peter Holm Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 00:21:45 -0000 Pavel Merdine wrote: > Hello , > > Thank you for raising this problem again. I already tried to do that > in that list, but received an answer that kernel is intended to do > that. For example, you have a faulty disk. And you have a faulty > sector which happened to occur on the directory place. So each time > kernel reads this sector it panics. So it's initially hard to even > understand what happens. And also it leads to corruption and lost > files on other file system (each time). Imagine if you have 15 disks. > In this case you have many files lost just because of a small (and not > significant) fault. It's just a nonsense. > Personally, I just replaced bad_dir with error return. > By the way, there was some bug in fs in kernel that could lead to > panic even on clean filesystem (bad_dir as far as I remember). It is > very rare and it was fixed on DragonFly. As far as I remember a fix > for this was also commited to current recently. > > I think that Linux is usually much smarter on this. By default it > remounts a file system as read-only in case it detects a filesystem > corruption. I would be very happy if FreeBSD could do the same, > because fs panics really hurt when you have many systems with disks. > > Of course I think we could do patches to overcome corrupting panics, > but the core FreeBSD team would not accept this, as they are happy > with panics and corruptions they make to other filesystems. You were so close to having an interesting email, and then you decided to expose yourself for the troll that you are. Thanks for the input, it will be ignored appropriately in the future. Scott From owner-freebsd-fs@FreeBSD.ORG Wed May 3 05:48:15 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8D1116A410 for ; Wed, 3 May 2006 05:48:15 +0000 (UTC) (envelope-from bkoenig@cs.tu-berlin.de) Received: from efacilitas.de (smtp.efacilitas.de [85.10.196.108]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27F1C43D4C for ; Wed, 3 May 2006 05:48:14 +0000 (GMT) (envelope-from bkoenig@cs.tu-berlin.de) Received: from eurystheus.local (port-212-202-39-242.dynamic.qsc.de [212.202.39.242]) by efacilitas.de (Postfix) with ESMTP id 8CCC34D410; Wed, 3 May 2006 07:59:42 +0200 (CEST) Received: from [192.168.1.2] (muhkuh.local [192.168.1.2]) by eurystheus.local (Postfix) with ESMTP id 340905285C; Wed, 3 May 2006 07:47:41 +0200 (CEST) Message-ID: <44584421.3000807@cs.tu-berlin.de> Date: Wed, 03 May 2006 07:48:17 +0200 From: =?ISO-8859-15?Q?Bj=F6rn_K=F6nig?= User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) X-Accept-Language: de-DE, de, en-us, en MIME-Version: 1.0 To: Kris Kennaway References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> In-Reply-To: <20060502221306.GD95348@xor.obsecurity.org> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 05:48:16 -0000 Kris Kennaway schrieb: > On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: >>Of course I think we could do patches to overcome corrupting panics, >>but the core FreeBSD team would not accept this, as they are happy >>with panics and corruptions they make to other filesystems. > > Of course not, don't make silly accusations :-) > > The problem is much more difficult to solve than "making the panic an > error return". I'm interested in more information about this issue. Do you have a reference to an old discussion about this topic or do you like to explain it a little bit further for me (and probably others)? Thanks in advance Björn :-) From owner-freebsd-fs@FreeBSD.ORG Wed May 3 07:20:16 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D48D816A401 for ; Wed, 3 May 2006 07:20:16 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7791443D48 for ; Wed, 3 May 2006 07:20:16 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 834031A3C2C; Wed, 3 May 2006 00:20:15 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 92C855164F; Wed, 3 May 2006 03:20:14 -0400 (EDT) Date: Wed, 3 May 2006 03:20:14 -0400 From: Kris Kennaway To: Bj?rn K?nig Message-ID: <20060503072013.GA2926@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ReaqsoxgOBHFXBhH" Content-Disposition: inline In-Reply-To: <44584421.3000807@cs.tu-berlin.de> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, Kris Kennaway Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 07:20:16 -0000 --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 07:48:17AM +0200, Bj?rn K?nig wrote: > Kris Kennaway schrieb: > >On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: > >>Of course I think we could do patches to overcome corrupting panics, > >>but the core FreeBSD team would not accept this, as they are happy > >>with panics and corruptions they make to other filesystems. > > > >Of course not, don't make silly accusations :-) > > > >The problem is much more difficult to solve than "making the panic an > >error return". >=20 > I'm interested in more information about this issue. Do you have a=20 > reference to an old discussion about this topic or do you like to=20 > explain it a little bit further for me (and probably others)? See the URL that Peter provided in his original post. The issue that he is testing is how well the filesystem behaves when you arbitrarily damage it and then run fsck (ideally, fsck should detect all of the damage and repair it). He seems to have found cases where fsck does not detect and repair the damage, leading to panics at runtime. You can ignore Pavel's reply since he didn't have anything to add to the discussion :-) Kris --ReaqsoxgOBHFXBhH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEWFmtWry0BWjoQKURAjj6AJ9+7UFC/IGSMy4kPmpxa1QL9+81gQCfVH62 EdYIhTuEK9KjgyqYgT/R+TI= =riDE -----END PGP SIGNATURE----- --ReaqsoxgOBHFXBhH-- From owner-freebsd-fs@FreeBSD.ORG Wed May 3 07:54:53 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08CFB16A417 for ; Wed, 3 May 2006 07:54:53 +0000 (UTC) (envelope-from peter@holm.cc) Received: from wbm7.pair.net (wbm7.pair.net [209.68.4.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id A30F343D4C for ; Wed, 3 May 2006 07:54:52 +0000 (GMT) (envelope-from peter@holm.cc) Received: by wbm7.pair.net (Postfix, from userid 65534) id 3972710574; Wed, 3 May 2006 03:54:50 -0400 (EDT) Received: from 193.3.141.124 ([193.3.141.124]) (SquirrelMail authenticated user holm@aedde.pair.com) by webmail7.pair.com with HTTP; Wed, 3 May 2006 09:54:50 +0200 (CEST) Message-ID: <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> In-Reply-To: <20060503072013.GA2926@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> Date: Wed, 3 May 2006 09:54:50 +0200 (CEST) From: "Peter Holm" To: "Kris Kennaway" User-Agent: SquirrelMail/1.4.5 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-fs@freebsd.org, Kris Kennaway Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 07:54:53 -0000 > On Wed, May 03, 2006 at 07:48:17AM +0200, Bj?rn K?nig wrote: >> Kris Kennaway schrieb: >> >On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: >> >>Of course I think we could do patches to overcome corrupting panics, >> >>but the core FreeBSD team would not accept this, as they are happy >> >>with panics and corruptions they make to other filesystems. >> > >> >Of course not, don't make silly accusations :-) >> > >> >The problem is much more difficult to solve than "making the panic an >> >error return". >> >> I'm interested in more information about this issue. Do you have a >> reference to an old discussion about this topic or do you like to >> explain it a little bit further for me (and probably others)? > > See the URL that Peter provided in his original post. > > The issue that he is testing is how well the filesystem behaves when > you arbitrarily damage it and then run fsck (ideally, fsck should > detect all of the damage and repair it). He seems to have found cases > where fsck does not detect and repair the damage, leading to panics at > runtime. > Actually the filesystems mounts without any problems if fsck is run first. The objective of this exercise was to show that background fsck may lead to panics. This was a problem I saw a lot a year ago when I did some testing of patches and in the cause of a working day saw two or three panics. With background fsck I would from time to time get a secondary panic, which typically zapped the original crash dump. - Peter - Peter > You can ignore Pavel's reply since he didn't have anything to add to > the discussion :-) > > Kris > - Peter Holm From owner-freebsd-fs@FreeBSD.ORG Wed May 3 10:04:03 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73EAA16A403 for ; Wed, 3 May 2006 10:04:03 +0000 (UTC) (envelope-from freebsd-fs@merdin.com) Received: from smtp.mailix.net (smtp.mailix.net [216.148.213.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1092443D46 for ; Wed, 3 May 2006 10:04:02 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from [62.148.144.242] (helo=[62.148.144.242]) by smtp.mailix.net with asmtp (Exim 4.24-H) id 1FbEDI-0000ed-Ov for freebsd-fs@freebsd.org; Wed, 03 May 2006 03:04:09 -0700 Date: Wed, 3 May 2006 14:03:55 +0400 From: Pavel Merdine X-Mailer: The Bat! (v3.71.04) Professional X-Priority: 3 (Normal) Message-ID: <213965528.20060503140355@merdin.com> To: Kris Kennaway In-Reply-To: <20060502221306.GD95348@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> MIME-Version: 1.0 X-SA-Exim-Mail-From: freebsd-fs@merdin.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-SA-Exim-Version: 3.1 (built Thu Oct 23 13:26:47 PDT 2003) X-SA-Exim-Scanned: No; Unknown failure X-uvscan-result: clean (1FbEDI-0000ed-Ov) Subject: Re[2]: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 10:04:03 -0000 Hello , Wednesday, May 3, 2006, 2:13:07 AM, you wrote: > On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: >> Hello , >> >> Thank you for raising this problem again. I already tried to do that >> in that list, but received an answer that kernel is intended to do >> that. For example, you have a faulty disk. And you have a faulty >> sector which happened to occur on the directory place. So each time >> kernel reads this sector it panics. So it's initially hard to even >> understand what happens. And also it leads to corruption and lost >> files on other file system (each time). Imagine if you have 15 disks. >> In this case you have many files lost just because of a small (and not >> significant) fault. It's just a nonsense. >> Personally, I just replaced bad_dir with error return. >> By the way, there was some bug in fs in kernel that could lead to >> panic even on clean filesystem (bad_dir as far as I remember). It is >> very rare and it was fixed on DragonFly. As far as I remember a fix >> for this was also commited to current recently. >> >> I think that Linux is usually much smarter on this. By default it >> remounts a file system as read-only in case it detects a filesystem >> corruption. I would be very happy if FreeBSD could do the same, >> because fs panics really hurt when you have many systems with disks. >> >> Of course I think we could do patches to overcome corrupting panics, >> but the core FreeBSD team would not accept this, as they are happy >> with panics and corruptions they make to other filesystems. > Of course not, don't make silly accusations :-) > The problem is much more difficult to solve than "making the panic an > error return". I think that is you who call me silly. I did not mean what you wrote. Where did I say that? I just said I made an error return for myself. And it was a quick patch to stop infinite series of panics (leaded to hundred files lost). What would you do in that case, very smart and genius man? I wrote I would be very happy if FreeBSD could do read-only remounts. > Kris -- / Pavel Merdine From owner-freebsd-fs@FreeBSD.ORG Wed May 3 10:19:18 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA46516A400 for ; Wed, 3 May 2006 10:19:18 +0000 (UTC) (envelope-from freebsd-fs@merdin.com) Received: from smtp.mailix.net (smtp.mailix.net [216.148.213.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E10043D46 for ; Wed, 3 May 2006 10:19:18 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from [62.148.144.242] (helo=[62.148.144.242]) by smtp.mailix.net with asmtp (Exim 4.24-CA) id 1FbERt-0004YO-Ta for freebsd-fs@freebsd.org; Wed, 03 May 2006 03:19:14 -0700 Date: Wed, 3 May 2006 14:19:11 +0400 From: Pavel Merdin X-Mailer: The Bat! (v3.71.04) Professional X-Priority: 3 (Normal) Message-ID: <1545943738.20060503141911@merdin.com> To: Scott Long In-Reply-To: <4457F7A0.9050502@samsco.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <4457F7A0.9050502@samsco.org> MIME-Version: 1.0 X-SA-Exim-Mail-From: freebsd-fs@merdin.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-SA-Exim-Version: 3.1 (built Thu Oct 23 13:26:47 PDT 2003) X-SA-Exim-Scanned: No; Unknown failure X-uvscan-result: clean (1FbERt-0004YO-Ta) Subject: Re[2]: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 10:19:18 -0000 Hello , > You were so close to having an interesting email, and then you decided > to expose yourself for the troll that you are. Thanks for the input, > it will be ignored appropriately in the future. So nothing has changed. -- / Pavel Merdine From owner-freebsd-fs@FreeBSD.ORG Wed May 3 10:57:18 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8763816A403 for ; Wed, 3 May 2006 10:57:18 +0000 (UTC) (envelope-from freebsd-fs@merdin.com) Received: from smtp.mailix.net (smtp.mailix.net [216.148.213.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 890FF43D55 for ; Wed, 3 May 2006 10:57:17 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from [62.148.144.242] (helo=[62.148.144.242]) by smtp.mailix.net with asmtp (Exim 4.24-H) id 1FbF2p-00025P-Nh for freebsd-fs@freebsd.org; Wed, 03 May 2006 03:57:24 -0700 Date: Wed, 3 May 2006 14:57:10 +0400 From: Pavel Merdin X-Mailer: The Bat! (v3.71.04) Professional X-Priority: 3 (Normal) Message-ID: <1408111762.20060503145710@merdin.com> To: Peter Holm In-Reply-To: <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> MIME-Version: 1.0 X-SA-Exim-Mail-From: freebsd-fs@merdin.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-SA-Exim-Version: 3.1 (built Thu Oct 23 13:26:47 PDT 2003) X-SA-Exim-Scanned: No; Unknown failure X-uvscan-result: clean (1FbF2p-00025P-Nh) Subject: Re[2]: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 10:57:18 -0000 Hello , Wednesday, May 3, 2006, 11:54:50 AM, you wrote: >> On Wed, May 03, 2006 at 07:48:17AM +0200, Bj?rn K?nig wrote: >>> Kris Kennaway schrieb: >>> >On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: >>> >>Of course I think we could do patches to overcome corrupting panics, >>> >>but the core FreeBSD team would not accept this, as they are happy >>> >>with panics and corruptions they make to other filesystems. >>> > >>> >Of course not, don't make silly accusations :-) >>> > >>> >The problem is much more difficult to solve than "making the panic an >>> >error return". >>> >>> I'm interested in more information about this issue. Do you have a >>> reference to an old discussion about this topic or do you like to >>> explain it a little bit further for me (and probably others)? >> >> See the URL that Peter provided in his original post. >> >> The issue that he is testing is how well the filesystem behaves when >> you arbitrarily damage it and then run fsck (ideally, fsck should >> detect all of the damage and repair it). He seems to have found cases >> where fsck does not detect and repair the damage, leading to panics at >> runtime. >> > Actually the filesystems mounts without any problems if fsck is run first That's not a bug in this case. It's a feature. And there is nothing new in this. Background fsck helps saving start time, but it's risky as kernel can panic if system accesses problematic sector before fsck. So background fsck should be turned off if one needs reliability (e.g. on servers). -- / Pavel Merdin From owner-freebsd-fs@FreeBSD.ORG Wed May 3 14:04:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69ABD16A406 for ; Wed, 3 May 2006 14:04:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id C06C243D49 for ; Wed, 3 May 2006 14:04:29 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 0A90A46C33; Wed, 3 May 2006 10:04:23 -0400 (EDT) Date: Wed, 3 May 2006 15:04:22 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Pavel Merdin In-Reply-To: <1408111762.20060503145710@merdin.com> Message-ID: <20060503150126.K21389@fledge.watson.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> <1408111762.20060503145710@merdin.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re[2]: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 14:04:31 -0000 On Wed, 3 May 2006, Pavel Merdin wrote: >> Actually the filesystems mounts without any problems if fsck is run first > > That's not a bug in this case. It's a feature. And there is nothing new in > this. Background fsck helps saving start time, but it's risky as kernel can > panic if system accesses problematic sector before fsck. So background fsck > should be turned off if one needs reliability (e.g. on servers). Well, the feature and problem are that bgfsck relies on invariants holding true for data written to the disk, so corrects only a narrow set of expected failure modes. I.e., that soft updates really does sequence changes out to the disk such that certain invariants regarding meta-data hold. This allows access to the file system before the checks are complete, but places assumptions on what will be found on disk. Those invariants might fail to hold for a few reasons -- bugs in UFS, hardware failure, and design breakage in the hardware are among the most common. One known problem is that the guarantees provided by recent ATA disks are really very weak with respect to the expectations of currently file systems. Robert N M Watson From owner-freebsd-fs@FreeBSD.ORG Wed May 3 18:44:13 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 283AB16A413 for ; Wed, 3 May 2006 18:44:13 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id D747043D5C for ; Wed, 3 May 2006 18:44:09 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id A230F1A3C2F; Wed, 3 May 2006 11:44:09 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 939FA518DD; Wed, 3 May 2006 14:44:08 -0400 (EDT) Date: Wed, 3 May 2006 14:44:08 -0400 From: Kris Kennaway To: Peter Holm Message-ID: <20060503184408.GA31172@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gBBFr7Ir9EOA20Yy" Content-Disposition: inline In-Reply-To: <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, Kris Kennaway Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 18:44:13 -0000 --gBBFr7Ir9EOA20Yy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 09:54:50AM +0200, Peter Holm wrote: >=20 > > On Wed, May 03, 2006 at 07:48:17AM +0200, Bj?rn K?nig wrote: > >> Kris Kennaway schrieb: > >> >On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: > >> >>Of course I think we could do patches to overcome corrupting panics, > >> >>but the core FreeBSD team would not accept this, as they are happy > >> >>with panics and corruptions they make to other filesystems. > >> > > >> >Of course not, don't make silly accusations :-) > >> > > >> >The problem is much more difficult to solve than "making the panic an > >> >error return". > >> > >> I'm interested in more information about this issue. Do you have a > >> reference to an old discussion about this topic or do you like to > >> explain it a little bit further for me (and probably others)? > > > > See the URL that Peter provided in his original post. > > > > The issue that he is testing is how well the filesystem behaves when > > you arbitrarily damage it and then run fsck (ideally, fsck should > > detect all of the damage and repair it). He seems to have found cases > > where fsck does not detect and repair the damage, leading to panics at > > runtime. > > >=20 > Actually the filesystems mounts without any problems if fsck is run first. >=20 > The objective of this exercise was to show that background fsck may lead > to panics. This was a problem I saw a lot a year ago when I did some > testing of patches and in the cause of a working day saw two or three > panics. With background fsck I would from time to time get a secondary > panic, which typically zapped the original crash dump. Oh, I misunderstood then :( Yes, this is pretty much to be expected: bg fsck depends on the (fairly strong) assumption that the only kinds of filesystem damage that are present at startup on filesystems with soft updates enabled are a) survivable (i.e. will not cause runtime problems before they are repaired), and b) may be repaired online. Modulo modern disk hardware violating these assumptions anyway, bg fsck more or less works as long as you only have "power failure" shutdowns. When your kernel panics instead (especially if it's a filesystem-related panic), all bets are off. With its dying breath, your kernel may decide to scribble all over your filesystem, causing arbitrary damage to it. Many of these types of damage are not "survivable", as you have demonstrated -- in fact the very existence of fsck is proof that the kernel is not designed to handle arbitrary damage at runtime. So the moral is that if your kernel is panicking a lot, turn off bg fsck or you'll probably hit other filesystem panics at runtime. I don't think you can completely prevent this problem, but one thing that may help would be for the kernel to attempt to write a marker to the dump device when it panics, and if this marker is present at boot time a fg fsck is forced. Of course the kernel will not always be able to do this, but it should work most of the time (since crashdumps usually work for most panics). Kris --gBBFr7Ir9EOA20Yy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEWPn4Wry0BWjoQKURAiiVAKCBDzQFis6Ett3TWstAMB2K63wu0QCg+VTm 9FQcMU9KZgpdm5yd+IfwJBM= =yycR -----END PGP SIGNATURE----- --gBBFr7Ir9EOA20Yy-- From owner-freebsd-fs@FreeBSD.ORG Wed May 3 18:45:52 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5A6216A456 for ; Wed, 3 May 2006 18:45:52 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0FA143D76 for ; Wed, 3 May 2006 18:45:45 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id C6FE31A4E04; Wed, 3 May 2006 11:45:45 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 00C91518DD; Wed, 3 May 2006 14:45:44 -0400 (EDT) Date: Wed, 3 May 2006 14:45:44 -0400 From: Kris Kennaway To: Pavel Merdine Message-ID: <20060503184544.GB31172@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <213965528.20060503140355@merdin.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WhfpMioaduB5tiZL" Content-Disposition: inline In-Reply-To: <213965528.20060503140355@merdin.com> User-Agent: Mutt/1.4.2.1i Cc: Kris Kennaway Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 18:45:52 -0000 --WhfpMioaduB5tiZL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 02:03:55PM +0400, Pavel Merdine wrote: > Hello , >=20 >=20 >=20 > Wednesday, May 3, 2006, 2:13:07 AM, you wrote: >=20 > > On Wed, May 03, 2006 at 12:32:29AM +0400, Pavel Merdine wrote: > >> Hello , > >>=20 > >> Thank you for raising this problem again. I already tried to do that > >> in that list, but received an answer that kernel is intended to do > >> that. For example, you have a faulty disk. And you have a faulty > >> sector which happened to occur on the directory place. So each time > >> kernel reads this sector it panics. So it's initially hard to even > >> understand what happens. And also it leads to corruption and lost > >> files on other file system (each time). Imagine if you have 15 disks. > >> In this case you have many files lost just because of a small (and not > >> significant) fault. It's just a nonsense. > >> Personally, I just replaced bad_dir with error return. > >> By the way, there was some bug in fs in kernel that could lead to > >> panic even on clean filesystem (bad_dir as far as I remember). It is > >> very rare and it was fixed on DragonFly. As far as I remember a fix > >> for this was also commited to current recently. > >>=20 > >> I think that Linux is usually much smarter on this. By default it > >> remounts a file system as read-only in case it detects a filesystem > >> corruption. I would be very happy if FreeBSD could do the same, > >> because fs panics really hurt when you have many systems with disks. > >>=20 > >> Of course I think we could do patches to overcome corrupting panics, > >> but the core FreeBSD team would not accept this, as they are happy > >> with panics and corruptions they make to other filesystems. >=20 > > Of course not, don't make silly accusations :-) > > The problem is much more difficult to solve than "making the panic an > > error return". >=20 > I think that is you who call me silly. I did not mean what you wrote. > Where did I say that? I just said I made an error return for myself. And you accused the "core FreeBSD team" of being happy with panics and filesystem corruption. That's a ridiculous accusation, so you are treated with ridicule for saying it. Kris --WhfpMioaduB5tiZL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEWPpXWry0BWjoQKURAhZ7AJ9na9iet+7DsbhOB0SuhpmAbduaTgCZAdKu f7nn37Pf0Ssz2TpkzbBXeUM= =dSxH -----END PGP SIGNATURE----- --WhfpMioaduB5tiZL-- From owner-freebsd-fs@FreeBSD.ORG Wed May 3 20:05:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D72C16A5D3 for ; Wed, 3 May 2006 20:05:58 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D27143D6D for ; Wed, 3 May 2006 20:05:56 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 6730E50C1F; Wed, 3 May 2006 22:05:55 +0200 (CEST) Received: from localhost (dlf3.neoplus.adsl.tpnet.pl [83.24.35.3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 7A41350B0D; Wed, 3 May 2006 22:05:47 +0200 (CEST) Date: Wed, 3 May 2006 22:04:06 +0200 From: Pawel Jakub Dawidek To: Kris Kennaway Message-ID: <20060503200406.GF3335@garage.freebsd.pl> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> <20060503184408.GA31172@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xs+9IvWevLaxKUtW" Content-Disposition: inline In-Reply-To: <20060503184408.GA31172@xor.obsecurity.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r535 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 20:05:59 -0000 --xs+9IvWevLaxKUtW Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 02:44:08PM -0400, Kris Kennaway wrote: +> Modulo modern disk hardware violating these assumptions anyway, bg +> fsck more or less works as long as you only have "power failure" +> shutdowns. +>=20 +> When your kernel panics instead (especially if it's a +> filesystem-related panic), all bets are off. With its dying breath, +> your kernel may decide to scribble all over your filesystem, causing +> arbitrary damage to it. Many of these types of damage are not +> "survivable", as you have demonstrated -- in fact the very existence +> of fsck is proof that the kernel is not designed to handle arbitrary +> damage at runtime. +>=20 +> So the moral is that if your kernel is panicking a lot, turn off bg +> fsck or you'll probably hit other filesystem panics at runtime. +>=20 +> I don't think you can completely prevent this problem, but one thing +> that may help would be for the kernel to attempt to write a marker to +> the dump device when it panics, and if this marker is present at boot +> time a fg fsck is forced. Of course the kernel will not always be +> able to do this, but it should work most of the time (since crashdumps +> usually work for most panics). Actually my feelings are exactly opposite. When you have a panic (ok, maybe not file system related), your system just stops, but disks write cache is flushed (at least from what I tested). On a power failure disk's write cache won't be flushed, which may left your file systems in totally unexpected state and I wouldn't be surprised if chances are that such a file system can't be repaired at all (also by fgfsck). In my opinion, bgfsck is only safe when write cache is turned off or there are no power failure problems. Of course this assumes there are no bugs in soft-updates code, which, given the size of SU code, is quite risky assumption. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --xs+9IvWevLaxKUtW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEWQy2ForvXbEpPzQRAv7HAKDyp9xiBc+t3Fsy2U34/eAexbw9dQCg5z+j GbS36QArjsgm880eq+rWw94= =QGQu -----END PGP SIGNATURE----- --xs+9IvWevLaxKUtW-- From owner-freebsd-fs@FreeBSD.ORG Wed May 3 20:15:47 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A012F16A43C; Wed, 3 May 2006 20:15:47 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19F0543D70; Wed, 3 May 2006 20:15:47 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id F14FB1A3C28; Wed, 3 May 2006 13:15:46 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 0449451F27; Wed, 3 May 2006 16:15:46 -0400 (EDT) Date: Wed, 3 May 2006 16:15:45 -0400 From: Kris Kennaway To: Pawel Jakub Dawidek Message-ID: <20060503201545.GA33457@xor.obsecurity.org> References: <20060502193900.GA94069@peter.osted.lan> <1541458526.20060503003229@merdin.com> <20060502221306.GD95348@xor.obsecurity.org> <44584421.3000807@cs.tu-berlin.de> <20060503072013.GA2926@xor.obsecurity.org> <18034.193.3.141.124.1146642890.squirrel@webmail7.pair.com> <20060503184408.GA31172@xor.obsecurity.org> <20060503200406.GF3335@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="a8Wt8u1KmwUX3Y2C" Content-Disposition: inline In-Reply-To: <20060503200406.GF3335@garage.freebsd.pl> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, Kris Kennaway Subject: Re: Stress testing the UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2006 20:15:49 -0000 --a8Wt8u1KmwUX3Y2C Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 03, 2006 at 10:04:06PM +0200, Pawel Jakub Dawidek wrote: > On Wed, May 03, 2006 at 02:44:08PM -0400, Kris Kennaway wrote: > +> Modulo modern disk hardware violating these assumptions anyway, bg > +> fsck more or less works as long as you only have "power failure" > +> shutdowns. > +>=20 > +> When your kernel panics instead (especially if it's a > +> filesystem-related panic), all bets are off. With its dying breath, > +> your kernel may decide to scribble all over your filesystem, causing > +> arbitrary damage to it. Many of these types of damage are not > +> "survivable", as you have demonstrated -- in fact the very existence > +> of fsck is proof that the kernel is not designed to handle arbitrary > +> damage at runtime. > +>=20 > +> So the moral is that if your kernel is panicking a lot, turn off bg > +> fsck or you'll probably hit other filesystem panics at runtime. > +>=20 > +> I don't think you can completely prevent this problem, but one thing > +> that may help would be for the kernel to attempt to write a marker to > +> the dump device when it panics, and if this marker is present at boot > +> time a fg fsck is forced. Of course the kernel will not always be > +> able to do this, but it should work most of the time (since crashdumps > +> usually work for most panics). >=20 > Actually my feelings are exactly opposite. When you have a panic (ok, > maybe not file system related), your system just stops, but disks write > cache is flushed (at least from what I tested). Per Peter's webpage (also my own experiences and those of others), further filesystem panics are very common after your system previously panics and you are running bg fsck. The panics stop after you force a fg fsck. Personally I usually disable bg fsck on all my machines because this problem hits me too often :-) Kris --a8Wt8u1KmwUX3Y2C Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEWQ9xWry0BWjoQKURAt5UAKDLAwfrclXacYQwvH/Lj/Vlf1aPbQCgwJx/ dIeAt6DrGe+dBN15bH+Z6vE= =1zvp -----END PGP SIGNATURE----- --a8Wt8u1KmwUX3Y2C-- From owner-freebsd-fs@FreeBSD.ORG Fri May 5 07:50:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B73A016A404 for ; Fri, 5 May 2006 07:50:25 +0000 (UTC) (envelope-from matthewk@adam.com.au) Received: from levanto.mail.adnap.net.au (levanto.mail.adnap.net.au [203.6.132.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F93743D4C for ; Fri, 5 May 2006 07:50:25 +0000 (GMT) (envelope-from matthewk@adam.com.au) Received: from [219.90.134.207] (219-90-134-207.ip.adam.com.au [219.90.134.207]) by levanto.mail.adnap.net.au (Postfix) with ESMTP id AE02F58A4 for ; Fri, 5 May 2006 17:20:20 +0930 (CST) Message-ID: <445B03CE.5070603@adam.com.au> Date: Fri, 05 May 2006 17:20:38 +0930 From: Matthew Keen User-Agent: Thunderbird 1.5 (X11/20051201) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: 3Ware Escalade 7450 - Cannot Create Filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 May 2006 07:50:25 -0000 I've been trying this for atleast the last 12 hours now... i've followed the handbook to a tee, i've tried googling etc and I am not getting anywhere... a solution will be greatly appreciated if not some help. Got a 3Ware Escalade 7450, Latest 7.7.1 Firmware from their site, FreeBSD 4.8.. everything is detecting fine in the dmesg area including both arrays.. 1x 300Gig Seagate in JBOD 3x 300Gig Seagate in RAID0 (Stripe) mammoth# disklabel -B -w -r twed0s1 auto mammoth# disklabel -e twed0s1e ----------- # /dev/twed0s1e: type: ESDI disk: amnesiac label: twed0s1e flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 36480 sectors/unit: 586067202 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 586067202 0 unused 0 0 # (Cyl. 0 - 36480*) --------------- mammoth# newfs /dev/twed0s1e newfs: /dev/twed0s1e: `e' partition is unavailable Here is my fdisk twed0 output.. mammoth# fdisk twed0 ******* Working on device /dev/twed0 ******* parameters extracted from in-core disklabel are: cylinders=36481 heads=255 sectors/track=63 (16065 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=36481 heads=255 sectors/track=63 (16065 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165,(FreeBSD/NetBSD/386BSD) start 63, size 586067202 (286165 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 640/ head 254/ sector 63 The data for partition 2 is: The data for partition 3 is: The data for partition 4 is: -- Kind Regards Matthew Keen -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Adam Internet 282 Gilbert St Adelaide SA 5000 Phone: 8423 4000 Fax : 8231 0223 Post : G.P.O Box 121, Adelaide 5001 Office Hours:- Monday - Friday: 9:00 AM - 10:00 PM Saturday - Sunday: 9:00 AM - 5:00 PM -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- From owner-freebsd-fs@FreeBSD.ORG Fri May 5 08:24:05 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5812116A400 for ; Fri, 5 May 2006 08:24:05 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30302.mail.mud.yahoo.com (web30302.mail.mud.yahoo.com [68.142.200.95]) by mx1.FreeBSD.org (Postfix) with SMTP id D381643D46 for ; Fri, 5 May 2006 08:24:04 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 58723 invoked by uid 60001); 5 May 2006 08:24:04 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kB7aiqkkcQH/4Q8+XFK4LuvXfTWea05WBWHEKQl6ftYBZhdksEEmE6r+mf4Y70fdMKMXqf3J7DKLhp5Wj3kKS5QSa1PQNFwtMnwa5ZEiQHHK/gQO16jhKa5Da4rCjQogPEHaWStCVEHLt0Z2ZNVId9SnGXlwBYAI117gDcehL3c= ; Message-ID: <20060505082404.58721.qmail@web30302.mail.mud.yahoo.com> Received: from [213.54.80.61] by web30302.mail.mud.yahoo.com via HTTP; Fri, 05 May 2006 01:24:04 PDT Date: Fri, 5 May 2006 01:24:04 -0700 (PDT) From: "R. B. Riddick" To: Matthew Keen , freebsd-fs@freebsd.org In-Reply-To: <445B03CE.5070603@adam.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: 3Ware Escalade 7450 - Cannot Create Filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 May 2006 08:24:05 -0000 --- Matthew Keen wrote: > 8 partitions: > # size offset fstype [fsize bsize bps/cpg] > c: 586067202 0 unused 0 0 # (Cyl. 0 - > 36480*) > --------------- > > mammoth# newfs /dev/twed0s1e > newfs: /dev/twed0s1e: `e' partition is unavailable > I say, r u sure, u have a partition 'e'? The bsdlabel/disklabel output does not look like... Maybe u should add the 'e' partition with bsdlabel -e twed0s1 by just adding the line e: 586067186 16 4.2BSD 0 0 0 in the end (I think the first 16 sectors of every slice should be reserved for something I dont know...). Then u may try successfully the newfs /dev/twed0s1e command... Just my 2 pennies... :-)) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com