From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 01:51:29 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8561616A469; Sun, 9 Sep 2007 01:51:29 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx1.freebsd.org (Postfix) with ESMTP id 680D913C467; Sun, 9 Sep 2007 01:51:28 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <46E3519E.3040001@FreeBSD.org> Date: Sun, 09 Sep 2007 03:51:26 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Peter Schuller References: <855622.76488.qm@web36601.mail.mud.yahoo.com> <20070908215102.GA8291@hyperion.scode.org> In-Reply-To: <20070908215102.GA8291@hyperion.scode.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Simun Mikecin , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: Swapping on ZFS - stability issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 01:51:29 -0000 Peter Schuller wrote: >> Try one of this (depends wheter you need encrypted swap or not): >> >> 1. create zvol with 4k blocksize instead of default 8k: >> zfs create -b 4k -V 4g tank/swap >> swapon /dev/zvol/tank/swap > > Using a 4k blocksize did not seem to have an effect on the problem. > > The test I am doing is with top in one virtual console and the test > program on another. I keep refreshing top seeing memory use climb, > until it starts hitting swap. After a few seconds, top hangs and the > only indications of life are ping and the console. > Please follow the directions in the developers handbook chapter on kernel debugging to obtain the relevant debugging information. Kris From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 10:00:45 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4583116A47C; Sun, 9 Sep 2007 10:00:45 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0015213C483; Sun, 9 Sep 2007 10:00:43 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 8F4DB23C474; Sun, 9 Sep 2007 12:00:42 +0200 (CEST) Date: Sun, 9 Sep 2007 12:00:42 +0200 From: Peter Schuller To: Kris Kennaway Message-ID: <20070909100041.GA60318@hyperion.scode.org> References: <855622.76488.qm@web36601.mail.mud.yahoo.com> <20070908215102.GA8291@hyperion.scode.org> <46E3519E.3040001@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LQksG6bCIzRHxTLp" Content-Disposition: inline In-Reply-To: <46E3519E.3040001@FreeBSD.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: Simun Mikecin , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: Swapping on ZFS - stability issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 10:00:45 -0000 --LQksG6bCIzRHxTLp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [kris] > Please follow the directions in the developers handbook chapter on kernel= =20 > debugging to obtain the relevant debugging information. Apologies. Aside from the first instance, where the machine had sat there for 12+ hours, I never got any panic message or anything like that which gives an opportunity for a stacktrace, etc. [pjd] > This is a known issue also in OpenSolaris. ZFS needs to allocate memory > to send I/O request, and when there is no memory, it can't allocate it > thus can't swap a process out and free it. Ok, so a known issue. Unless someone tells me otherwise I'll update the ZFS-on-FreeBSD Wiki pages to warn about this issue (assuming it's editable by me). Thank you, --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --LQksG6bCIzRHxTLp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG48RJDNor2+l1i30RAu0YAKC7zHGvcHL5iwVpma0xhrraqwl8mACfTuTB oeB8gVcSyZEUlqAXxe8poYI= =jI+K -----END PGP SIGNATURE----- --LQksG6bCIzRHxTLp-- From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 11:12:32 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE42316A418; Sun, 9 Sep 2007 11:12:32 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx1.freebsd.org (Postfix) with ESMTP id BC1F613C45E; Sun, 9 Sep 2007 11:12:31 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <46E3D51E.3020504@FreeBSD.org> Date: Sun, 09 Sep 2007 13:12:30 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Peter Schuller References: <855622.76488.qm@web36601.mail.mud.yahoo.com> <20070908215102.GA8291@hyperion.scode.org> <46E3519E.3040001@FreeBSD.org> <20070909100041.GA60318@hyperion.scode.org> In-Reply-To: <20070909100041.GA60318@hyperion.scode.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Simun Mikecin , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: Swapping on ZFS - stability issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 11:12:32 -0000 Peter Schuller wrote: > [kris] >> Please follow the directions in the developers handbook chapter on kernel >> debugging to obtain the relevant debugging information. > > Apologies. Aside from the first instance, where the machine had sat > there for 12+ hours, I never got any panic message or anything like > that which gives an opportunity for a stacktrace, etc. Doesn't matter, you can debug deadlocks too :) > [pjd] >> This is a known issue also in OpenSolaris. ZFS needs to allocate memory >> to send I/O request, and when there is no memory, it can't allocate it >> thus can't swap a process out and free it. > > Ok, so a known issue. Unless someone tells me otherwise I'll update > the ZFS-on-FreeBSD Wiki pages to warn about this issue (assuming it's > editable by me). Thanks. Kris From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 17:07:35 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B86B016A419 for ; Sun, 9 Sep 2007 17:07:35 +0000 (UTC) (envelope-from jo_t@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 2283C13C465 for ; Sun, 9 Sep 2007 17:07:34 +0000 (UTC) (envelope-from jo_t@gmx.net) Received: (qmail invoked by alias); 09 Sep 2007 16:40:53 -0000 Received: from wh58-703.st.Uni-Magdeburg.DE (EHLO [192.168.73.100]) [141.44.198.73] by mail.gmx.net (mp045) with SMTP; 09 Sep 2007 18:40:53 +0200 X-Authenticated: #2964489 X-Provags-ID: V01U2FsdGVkX1+Y4zwsuKyYcx38LqxE0y8ECndXaHzqs6kxCSZdQ9 GKddYfUrK4ZHe1 Message-ID: <46E4225F.1020806@gmx.net> Date: Sun, 09 Sep 2007 18:42:07 +0200 From: Johannes Totz User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070728 Thunderbird/2.0.0.6 Mnenhy/0.7.5.666 MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 0.95.3 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Subject: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 17:07:35 -0000 Hi! Seems like UFS does not handle disk/write errors properly, causes silent corruptions and which causes a panic later during snapshot creation. > #uname -a > FreeBSD alfred 6.2-STABLE FreeBSD 6.2-STABLE #0: Thu Jul 12 20:40:55 CEST 2007 root@alfred:/usr/obj/usr/src/sys/ALFRED i386 One day a write error on one of my disks happened: > Aug 22 05:24:39 alfred kernel: ad0: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=469004995 > Aug 22 05:24:40 alfred kernel: ad0: FAILURE - READ_DMA48 status=51 error=10 LBA=469004995 > Aug 22 05:24:40 alfred kernel: g_vfs_done():ufs/home[READ(offset=240130525184, length=2048)]error = 5 > Aug 22 05:25:08 alfred kernel: ad0: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=490974155 > Aug 22 05:25:08 alfred kernel: ad0: FAILURE - READ_DMA48 status=51 error=10 LBA=490974155 > Aug 22 05:25:08 alfred kernel: g_vfs_done():ufs/home[READ(offset=251378735104, length=2048)]error = 5 This has never happened before and did not happen again (yet). A long test with smartctl reports "all fine". So lets attribute that to a cosmic ray (or neutrino, pick your favorite) hitting the controller. The system continued to run fine afterwards. But: next morning during automatic snapshot creation it panic'ed with: > Aug 23 06:38:14 alfred kernel: ffs_snapshot_mount: old format snapshot inode 8 > Aug 23 06:38:14 alfred savecore: reboot after panic: snapacct_ufs2: bad block So of course it restarted. And tried to do a background fsck. And failed again... and again... and again... > Aug 23 07:08:15 alfred kernel: ffs_snapshot_mount: old format snapshot inode 4 > Aug 23 07:08:15 alfred savecore: reboot after panic: snapacct_ufs2: bad block The report inode varies but "bad block" is always the same. So this went on for about 10x until I had a chance to interrupt it (i.e. woke from slumber) and boot into single user mode. Multiple runs of fsck fixed the problem. Deleted all old snapshot files and system is fine. No further problems. Maybe some files got lost; can't tell, there are a few million on it. Also see: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/114676 Unfortunately I don't have time to dig into this. But I wanted to report it. Maybe someone already fixed it... Cheers, Johannes From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 17:27:50 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24B1B16A420 for ; Sun, 9 Sep 2007 17:27:50 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx1.freebsd.org (Postfix) with ESMTP id 6867713C442; Sun, 9 Sep 2007 17:27:49 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <46E42D14.5060605@FreeBSD.org> Date: Sun, 09 Sep 2007 19:27:48 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Johannes Totz References: <46E4225F.1020806@gmx.net> In-Reply-To: <46E4225F.1020806@gmx.net> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 17:27:50 -0000 Johannes Totz wrote: > Hi! > > Seems like UFS does not handle disk/write errors properly, causes silent > corruptions and which causes a panic later during snapshot creation. > >> #uname -a >> FreeBSD alfred 6.2-STABLE FreeBSD 6.2-STABLE #0: Thu Jul 12 20:40:55 CEST 2007 root@alfred:/usr/obj/usr/src/sys/ALFRED i386 > > One day a write error on one of my disks happened: > >> Aug 22 05:24:39 alfred kernel: ad0: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=469004995 >> Aug 22 05:24:40 alfred kernel: ad0: FAILURE - READ_DMA48 status=51 error=10 LBA=469004995 >> Aug 22 05:24:40 alfred kernel: g_vfs_done():ufs/home[READ(offset=240130525184, length=2048)]error = 5 >> Aug 22 05:25:08 alfred kernel: ad0: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=490974155 >> Aug 22 05:25:08 alfred kernel: ad0: FAILURE - READ_DMA48 status=51 error=10 LBA=490974155 >> Aug 22 05:25:08 alfred kernel: g_vfs_done():ufs/home[READ(offset=251378735104, length=2048)]error = 5 > > This has never happened before and did not happen again (yet). A long > test with smartctl reports "all fine". So lets attribute that to a > cosmic ray (or neutrino, pick your favorite) hitting the controller. > > The system continued to run fine afterwards. > But: next morning during automatic snapshot creation it panic'ed with: > >> Aug 23 06:38:14 alfred kernel: ffs_snapshot_mount: old format snapshot inode 8 >> Aug 23 06:38:14 alfred savecore: reboot after panic: snapacct_ufs2: bad block > > So of course it restarted. And tried to do a background fsck. And failed > again... and again... and again... > >> Aug 23 07:08:15 alfred kernel: ffs_snapshot_mount: old format snapshot inode 4 >> Aug 23 07:08:15 alfred savecore: reboot after panic: snapacct_ufs2: bad block > > The report inode varies but "bad block" is always the same. > So this went on for about 10x until I had a chance to interrupt it (i.e. > woke from slumber) and boot into single user mode. > Multiple runs of fsck fixed the problem. Deleted all old snapshot files > and system is fine. No further problems. Maybe some files got lost; > can't tell, there are a few million on it. > > Also see: > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/114676 > > Unfortunately I don't have time to dig into this. But I wanted to report > it. Maybe someone already fixed it... bg fsck cannot fix arbitrary filesystem corruption. Nor is it intended to. Kris From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 19:57:29 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A25C116A419; Sun, 9 Sep 2007 19:57:29 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F3DC13C45B; Sun, 9 Sep 2007 19:57:29 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 0552223C4AB; Sun, 9 Sep 2007 21:57:27 +0200 (CEST) Date: Sun, 9 Sep 2007 21:57:27 +0200 From: Peter Schuller To: Kris Kennaway Message-ID: <20070909195727.GA97233@hyperion.scode.org> References: <855622.76488.qm@web36601.mail.mud.yahoo.com> <20070908215102.GA8291@hyperion.scode.org> <46E3519E.3040001@FreeBSD.org> <20070909100041.GA60318@hyperion.scode.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="AqsLC8rIMeq19msA" Content-Disposition: inline In-Reply-To: <20070909100041.GA60318@hyperion.scode.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: Simun Mikecin , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: Swapping on ZFS - stability issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 19:57:29 -0000 --AqsLC8rIMeq19msA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > Ok, so a known issue. Unless someone tells me otherwise I'll update > the ZFS-on-FreeBSD Wiki pages to warn about this issue (assuming it's > editable by me). It is not editable by me. So if anyone with edit privileges wants to, what I am referring to is the quickstart guide which has swap-on-zfs as an explicit example, thus implying it will actually work ;) http://wiki.freebsd.org/ZFSQuickStartGuide For convenience, here is the URL to pjd's post that may be worth linking to: http://lists.freebsd.org/pipermail/freebsd-current/2007-September/076831.ht= ml --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --AqsLC8rIMeq19msA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG5FAmDNor2+l1i30RAmgfAKCLiLmyEYR+ZWPQKWg0ipfRHfkmMACgsJAF U6VoEu/5dFcybt7prEywAXg= =34U3 -----END PGP SIGNATURE----- --AqsLC8rIMeq19msA-- From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 20:09:35 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EE0716A419; Sun, 9 Sep 2007 20:09:35 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id E23EC13C481; Sun, 9 Sep 2007 20:09:34 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 254BF23C4AB; Sun, 9 Sep 2007 22:09:34 +0200 (CEST) Date: Sun, 9 Sep 2007 22:09:34 +0200 From: Peter Schuller To: Kris Kennaway Message-ID: <20070909200933.GA98161@hyperion.scode.org> References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UugvWAfsgieZRqgk" Content-Disposition: inline In-Reply-To: <46E42D14.5060605@FreeBSD.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 20:09:35 -0000 --UugvWAfsgieZRqgk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > bg fsck cannot fix arbitrary filesystem corruption. Nor is it intended t= o. But is not the proposed problem that UFS caused the corruption to begin with? Given that updates are already done in such a way as to cause predictable inconsistency in the event of a crash, should not appropriate course of action in a case such as this be to panic, unmount the fs, rollback and error out, or otherwise abort the operation in a way the filesystem can be fsck:ed and re-mounted (assuming the device is alive), rather than cause corruption? --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --UugvWAfsgieZRqgk Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG5FL9DNor2+l1i30RAj49AJ0beAgk+Vua8EHZvXM2OIFu3tOkcACfW+4i VBTspk6BIGd6DU6Jt9xeohs= =DZqC -----END PGP SIGNATURE----- --UugvWAfsgieZRqgk-- From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 20:57:58 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 332E616A41B for ; Sun, 9 Sep 2007 20:57:58 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx1.freebsd.org (Postfix) with ESMTP id 4312713C465; Sun, 9 Sep 2007 20:57:57 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <46E45E54.6040207@FreeBSD.org> Date: Sun, 09 Sep 2007 22:57:56 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Peter Schuller References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> In-Reply-To: <20070909200933.GA98161@hyperion.scode.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 20:57:58 -0000 Peter Schuller wrote: >> bg fsck cannot fix arbitrary filesystem corruption. Nor is it intended to. > > But is not the proposed problem that UFS caused the corruption to > begin with? No, "something unexpected happened". > Given that updates are already done in such a way as to cause > predictable inconsistency in the event of a crash, should not > appropriate course of action in a case such as this be to panic, > unmount the fs, rollback and error out, or otherwise abort the > operation in a way the filesystem can be fsck:ed and re-mounted > (assuming the device is alive), rather than cause corruption? Soft updates isn't journalling, so you can't "roll back" an error. It works by maintaining knowledge of the on-disk state of data and ensuring that it only writes to disk in a suitable order so that the on-disk state is supposed to remain consistent. Unfortunately there are many ways in which this can fail, mostly involving external factors violating the assumptions upon which soft updates relies. For example, the data written on disk may not correspond to the data dispatched by soft updates, due to things like write caching in the hardware, write reordering, data corruption, unpredictable disk behaviour during power loss, hardware failure, etc. Similarly, background fsck assumes that the only filesystem errors it will encounter are those permitted by the soft updates model, which are "benign", i.e. non-fatal and correctable at runtime. When the state of your disk departs from the realm of these assumptions, bg fsck may not be able to repair the damage. Kris From owner-freebsd-fs@FreeBSD.ORG Sun Sep 9 22:11:46 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2103316A420; Sun, 9 Sep 2007 22:11:46 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5538913C4A3; Sun, 9 Sep 2007 22:11:44 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 0932323C490; Mon, 10 Sep 2007 00:11:42 +0200 (CEST) Date: Mon, 10 Sep 2007 00:11:42 +0200 From: Peter Schuller To: Kris Kennaway Message-ID: <20070909221142.GA6435@hyperion.scode.org> References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> <46E45E54.6040207@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline In-Reply-To: <46E45E54.6040207@FreeBSD.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2007 22:11:46 -0000 --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > Soft updates isn't journalling, so you can't "roll back" an error. It=20 > works by maintaining knowledge of the on-disk state of data and ensuring = =20 > that it only writes to disk in a suitable order so that the on-disk state= =20 > is supposed to remain consistent. I am aware of this, I was speaking generally. The least "committal" solution being to just panic. The point I was trying to make was that as long as errors are traditional and simple, as in not being able to read a particular sector, or a write to a sector failed, aborting all operations should not lead to corruption since that is exactly what the filesystem has been designed to prevent (essentially panicing the machine from the perspective of the on-disk filesystem even if the system is not actually paniced, such as if the filesystem is unmounted instead). > Unfortunately there are many ways in which this can fail, mostly involvin= g=20 > external factors violating the assumptions upon which soft updates relies= =2E =20 > For example, the data written on disk may not correspond to the data=20 > dispatched by soft updates, due to things like write caching in the=20 > hardware, write reordering, data corruption, unpredictable disk behaviour= =20 > during power loss, hardware failure, etc. I am aware of this too (and paranoid about it). > Similarly, background fsck assumes that the only filesystem errors it wil= l=20 > encounter are those permitted by the soft updates model, which are=20 > "benign", i.e. non-fatal and correctable at runtime. When the state of= =20 > your disk departs from the realm of these assumptions, bg fsck may not be= =20 > able to repair the damage. My thinking was that in simple cases (e.g., say you put UFS on a geom provider that simulates failure, or the disk has a transient write failure on some particular sector, etc), unmounting the filesystem (or remounting read-only) would lead to a filesystem with only expected (and designed for) inconsistencies - assuming of course that there is no other issues going on, such as random corruption on the drive or in the I/O path. In any case, I was not really looking to get into a debate. I only commented because my reading of the original post was that of a potential bug in UFS, rather than lack of understanding that fsck cannot fix arbitrary errors. As with most such bug reports coming from a real-life situation, one can never prove that there was not random corruption along the I/O path or whatever else. Since I know from personal experience, and my understanding from previous ML traffic is that it is a known issue, the I/O failure handling in UFS is not rock solid in terms of system stability; so taking that a bit further and causing corruption did not seem like a huge leap (e.g., perhaps continuing with a dependent write even though the preveious write failed - not unthinkable without being familiar with the code). --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --xHFwDpU9dbj6ez1V Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFG5G+eDNor2+l1i30RArmDAJ9dyRW7dTVopYFAczdAa0ydBEOZBQCfREWq EzVSVUGfzCCFo3tMEUYlgW8= =ZB6b -----END PGP SIGNATURE----- --xHFwDpU9dbj6ez1V-- From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 07:14:50 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BF8316A419 for ; Mon, 10 Sep 2007 07:14:50 +0000 (UTC) (envelope-from nikhil.success@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.189]) by mx1.freebsd.org (Postfix) with ESMTP id 05F0F13C46A for ; Mon, 10 Sep 2007 07:14:49 +0000 (UTC) (envelope-from nikhil.success@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so918763rvb for ; Mon, 10 Sep 2007 00:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=JOuo8G8FRz7GBUIJt72bBc7zaplU57NzGcNJhh+kGaE=; b=BbDZ85O4OtHIyQafBQET0vhe2UxzmywpMOVixA0HCfgNq/yuJpXfWVTj00dhlWh6rbGW0XOXmdBQAJ2J5O4WrE6jfwQ577BYTVAExM4u3tuGYcVvbTgvoIWN/joZh+1x8rBS5Eth19uyDDTbxFI5aDouFFuw0xqClcvWsuMRaWg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=YKoq4Z0SnYJiPqOXCk4bwfRfioHlo7jx1xB4hi0UmInzQ43Qes3pNoH1w/3GUyPd4gs6XN+nBOuKDdGhS3hQnA4cr1cQYP7JGPKjIqubwdvCCiNsQk/yk6ufwQ1jGtSlYMRUR20fqFJfZYy+5eBuNslQA5Y3l1wKqXoGiKN5rhw= Received: by 10.141.3.10 with SMTP id f10mr1752823rvi.1189408489108; Mon, 10 Sep 2007 00:14:49 -0700 (PDT) Received: by 10.141.82.6 with HTTP; Mon, 10 Sep 2007 00:14:49 -0700 (PDT) Message-ID: Date: Mon, 10 Sep 2007 12:44:49 +0530 From: "Nikhil Gupta" To: freebsd-fs@freebsd.org In-Reply-To: <20070908120007.977FA16A503@hub.freebsd.org> MIME-Version: 1.0 References: <20070908120007.977FA16A503@hub.freebsd.org> Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: freebsd-fs Digest, Vol 220, Issue 5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 07:14:50 -0000 Thanks for the replies. One more small query I have. What output do you get when you give uname command on freebsd. I am asking this as I donot have access to any freebsd system Thanks On 9/8/07, freebsd-fs-request@freebsd.org wrote: > > Send freebsd-fs mailing list submissions to > freebsd-fs@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > or, via email, send a message with subject or body 'help' to > freebsd-fs-request@freebsd.org > > You can reach the person managing the list at > freebsd-fs-owner@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-fs digest..." > > > Today's Topics: > > 1. fetching atime information (Nikhil Gupta) > 2. Re: fetching atime information (Andrew Pantyukhin) > 3. Re: fetching atime information (Bill Vermillion) > 4. noatime on / and /var too ? (Gore Jarold) > 5. net/samba3 faults while try to export fusefs or msdosfs > volumes (Vladimir Grebenschikov) > 6. On-disk indexing for "Project Ideas" page (Nikolay Pavlov) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 7 Sep 2007 19:07:51 +0530 > From: "Nikhil Gupta" > Subject: fetching atime information > To: freebsd-fs@freebsd.org > Message-ID: > > Content-Type: text/plain; charset=3DISO-8859-1 > > Hi all, > > I am trying to find for all partitions whether atime update is enabled or > disabled. > What command can display the atime status (whether enabled or disabled) > information for all partitions on free bsd. > > for eg. in Linux, if I give mount command, it displays properties of each > partitions along with noatime (if atime update is disabled). > > Nikhil > > > ------------------------------ > > Message: 2 > Date: Fri, 7 Sep 2007 18:13:55 +0400 > From: Andrew Pantyukhin > Subject: Re: fetching atime information > To: Nikhil Gupta > Cc: freebsd-fs@freebsd.org > Message-ID: <20070907141354.GB56443@amilo.cenkes.org> > Content-Type: text/plain; charset=3Dus-ascii > > On Fri, Sep 07, 2007 at 07:07:51PM +0530, Nikhil Gupta wrote: > > Hi all, > > > > I am trying to find for all partitions whether atime update is enabled > or > > disabled. > > What command can display the atime status (whether enabled or disabled) > > information for all partitions on free bsd. > > > > for eg. in Linux, if I give mount command, it displays properties of > each > > partitions along with noatime (if atime update is disabled). > > Same here. > % mount|grep noatime > /dev/ad0s1f on /usr (ufs, NFS exported, local, noatime) > > > ------------------------------ > > Message: 3 > Date: Fri, 7 Sep 2007 10:33:59 -0400 > From: Bill Vermillion > Subject: Re: fetching atime information > To: Nikhil Gupta > Cc: freebsd-fs@freebsd.org > Message-ID: <20070907143359.GA94581@wjv.com> > Content-Type: text/plain; charset=3Dus-ascii > > On Fri, Sep 07, 2007 at 19:07 , while impersonating an expert on > the internet, Nikhil Gupta sent this to stdout: > > > Hi all, > > > > I am trying to find for all partitions whether atime update is enabled > or > > disabled. > > What command can display the atime status (whether enabled or disabled) > > information for all partitions on free bsd. > > > > for eg. in Linux, if I give mount command, it displays properties of > each > > partitions along with noatime (if atime update is disabled). > > Another poster showed the output of mount. > > You can also check the /etc/fstab to see and/or change which > file systems you set for 'noatime'. > > Bill > -- > Bill Vermillion - bv @ wjv . com > > > ------------------------------ > > Message: 4 > Date: Fri, 7 Sep 2007 15:17:09 -0700 (PDT) > From: Gore Jarold > Subject: noatime on / and /var too ? > To: freebsd-fs@freebsd.org > Message-ID: <704329.73647.qm@web63015.mail.re1.yahoo.com> > Content-Type: text/plain; charset=3Diso-8859-1 > > For performance, I have always set 'noatime' on my > large data partitions. > > For some odd reason I never bothered to set it on / > and /var ... is there any reason not to do that ? > > I know it won't change much since they are not busy > filesystems, but if there is no risk and no "best > practices" reason _not_ to do it, I might as well... > > right ? > > > > > _________________________________________________________________________= ___________ > Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, > news, photos & more. > http://mobile.yahoo.com/go?refer=3D1GNXIC > > > ------------------------------ > > Message: 5 > Date: Sat, 08 Sep 2007 01:45:09 +0400 > From: Vladimir Grebenschikov > Subject: net/samba3 faults while try to export fusefs or msdosfs > volumes > To: freebsd-fs@freebsd.org > Message-ID: <1189201509.1628.10.camel@localhost> > Content-Type: text/plain > > Hi > > Anybody already found that problem ? Probably some clues ? > > from /var/log/samba.log: > katis (172.22.2.11) connect to service C initially as user vova > (uid=3D207, gid=3D207) (pid 40339) > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(41) > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(42) > INTERNAL ERROR: Signal 6 in pid 40339 (3.0.25a) > Please read the Trouble-Shooting section of the Samba3-HOWTO > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(44) > > > -- > Vladimir B. Grebenschikov > vova@fbsd.ru > > > ------------------------------ > > Message: 6 > Date: Sat, 8 Sep 2007 12:43:44 +0300 > From: Nikolay Pavlov > Subject: On-disk indexing for "Project Ideas" page > To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org > Message-ID: <200709081243.48890.qpadla@gmail.com> > Content-Type: text/plain; charset=3D"utf-8" > > Recently while reading "Design and Implementation of FreeBSD operation > system" by Marshall Kirk McKusick and gnn i have found a very intresting > paragraph regarding UFS2 implementation, indexing and B-trees. According > to it on-disk indexes was not implemented, but some structures was > reserved for future development. May be some SOC students could implement > this in future. How about to adding this into Project Ideas page? > Let me quote from the paragraph "8.3 Naming": > > Finding of Names in Directories > > A common request to the filesystem is to lookup a specific name in a > directory. The kernel usually does the lookup by starting at the beginnin= g > of the directory and going through, comparing each entry in turn. First, > the length of the sought-after name is compared with the length of the > name being checked. If the lengths are identical, a string comparison of > the name being sought and the directory entry is made. If they match, the > search is complete; if they fail, either in the length or in the string > comparison, the search continues with the next entry. Whenever a name is > found, its name and containing directory are entered into the systemwide > name cache described in Section 6.6. Whenever a search is unsuccessful, a= n > entry is made in the cache showing that the name does not exist in the > particular directory. Before starting a directory scan, the kernel looks > for the name in the cache. If either a positive or negative entry is > found, the directory scan can be avoided. > > Another common operation is to lookup all the entries in a directory. For > example, many programs do a stat system call on each name in a directory > in the order that the names appear in the directory. To improve > performance for these programs, the kernel maintains the directory offset > of the last successful lookup for each directory. Each time that a lookup > is done in that directory, the search is started from the offset at which > the previous name was found (instead of from the beginning of the > directory). For programs that step sequentially through a directory with = n > files, search time decreases from Order(n2) to Order(n). > > One quick benchmark that demonstrates the maximum effectiveness of the > cache is running the ls -l command on a directory containing 600 files. O= n > a system that retains the most recent directory offset, the amount of > system time for this test is reduced by 85 percent. Unfortunately, the > maximum effectiveness is much greater than the average effectiveness. > Although the cache is 90 percent effective when hit, it is applicable to > only about 25 percent of the names being looked up. Despite the amount of > time spent in the lookup routine itself decreasing substantially, the > improvement is diminished because more time is spent in the routines that > that routine calls. Each cache miss causes a directory to be accessed > twice=97once to search from the middle to the end and once to search from > the beginning to the middle. > > These caches provide good directory lookup performance but are ineffectiv= e > for large directories that have a high rate of entry creation and > deletion. Each time a new directory entry is created, the kernel must sca= n > the entire directory to ensure that the entry does not already exist. Whe= n > an existing entry is deleted, the kernel must scan the directory to find > the entry to be removed. For directories with many entries these linear > scans are time-consuming. > > The approach to solving this problem in FreeBSD 5.2 is to introduce > dynamic > directory hashing that retrofits a directory indexing system to UFS [Dows= e > & Malone, 2002]. To avoid repeated linear searches of large directories, > the dynamic directory hashing builds a hash table of directory entries on > the fly when the directory is first accessed. This table avoids directory > scans on later lookups, creates, and deletes. Unlike filesystems > originally designed with large directories in mind, these indices are not > saved on disk and so the system is backward compatible. The drawback is > that the indices need to be built the first time that a large directory i= s > encountered after each system reboot. The effect of the dynamic directory > hashing is that large directories in UFS cause minimal performance > problems. > > When we built UFS2, we contemplated solving the large directory update > problem by changing to a more complex directory structure such as one tha= t > uses B-trees. This technique is used in many modern filesystems such as > XFS [Sweeney et al., 1996], JFS [Best & Kleikamp, 2003], ReiserFS [Reiser= , > 2001], and in later versions of Ext2 [Phillips, 2001]. We decided not to > make the change at the time that UFS2 was first implemented for several > reasons. First, we had limited time and resources, and we wanted to get > something working and stable that could be used in the time frame of > FreeBSD 5.2. By keeping the same directory format, we were able to reuse > all the directory code from UFS1, did not have to change numerous > filesystem utilities to understand and maintain a new directory format, > and were able to produce a stable and reliable filesystem in the time > frame available to us. The other reason that we felt that we could retain > the existing directory structure is because of the dynamic directory > hashing that was added to FreeBSD. > > Borrowing the technique used by the Ext2 filesystem a flag was also added > to show that an on-disk indexing structure is supported for directories > [Phillips, 2001]. This flag is unconditionally turned off by the existing > implementation of UFS. In the future, if an implementation of an on-disk > directory-indexing structure is added, the implementations that support i= t > will not turn the flag off. Index-supporting kernels will maintain the > indices and leave the flag on. If an old non-index-supporting kernel is > run, it will turn off the flag so that when the filesystem is once again > run under a new kernel, the new kernel will discover that the indexing > flag has been turned off and will know that the indices may be out date > and have to be rebuilt before being used. The only constraint on an > implementation of the indices is that they have to be an auxiliary data > structure that references the old linear directory format. > > > -- > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Best regards, Nikolay Pavlov. <<<----------------------------------- > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: not available > Type: application/pgp-signature > Size: 189 bytes > Desc: This is a digitally signed message part. > Url : > http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20070908/00697f= f9/attachment-0001.pgp > > ------------------------------ > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > End of freebsd-fs Digest, Vol 220, Issue 5 > ****************************************** > From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 07:16:03 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B19016A419 for ; Mon, 10 Sep 2007 07:16:03 +0000 (UTC) (envelope-from nikhil.success@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.186]) by mx1.freebsd.org (Postfix) with ESMTP id B5E6113C478 for ; Mon, 10 Sep 2007 07:16:02 +0000 (UTC) (envelope-from nikhil.success@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so918968rvb for ; Mon, 10 Sep 2007 00:16:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=ZiNJAeq40LE1zFbOSzJTb3vKt6HM65vF/GcvvIIeAsw=; b=BJFHzFIPrLbkeB+SU13eo83G9JqTLS9+QYSPVcoW6ZW4VkW+qCIhPT0iU02VulKNxS52RMG42YiOCHeje14x1irD8R64Dt7xDdl7HMVdXoioDn9qUbZMlgbfv5QAuTelSBWDme+qg0fWwaPZ0hGCry1E0s1tdWlBaUc370fRDZM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=SGoNDJGdxe3GDxijBfavcYsuiTEvAoPlpD5OYfcjEIqkSO6qGVqJEkWhaWg8/qp7GunB4OLn5vwXxgHENkjBql4eB997527dapKV9pmNvshgN/e0CfnvYhDJ2Y33gzJWqNMHLsvWkJoZiPqWZ0GtTkIBKlqBkng/nNdqc1LMrCc= Received: by 10.141.74.17 with SMTP id b17mr1757550rvl.1189408562204; Mon, 10 Sep 2007 00:16:02 -0700 (PDT) Received: by 10.141.82.6 with HTTP; Mon, 10 Sep 2007 00:16:02 -0700 (PDT) Message-ID: Date: Mon, 10 Sep 2007 12:46:02 +0530 From: "Nikhil Gupta" To: freebsd-fs@freebsd.org In-Reply-To: <20070908120007.977FA16A503@hub.freebsd.org> MIME-Version: 1.0 References: <20070908120007.977FA16A503@hub.freebsd.org> Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: freebsd-fs Digest, Vol 220, Issue 5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 07:16:03 -0000 Can anybody help me for sun solaris for the same. On sun solaris, how do we get the similar information of atime as we get o= n linux and bsd using mount command. On 9/8/07, freebsd-fs-request@freebsd.org wrote: > > Send freebsd-fs mailing list submissions to > freebsd-fs@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > or, via email, send a message with subject or body 'help' to > freebsd-fs-request@freebsd.org > > You can reach the person managing the list at > freebsd-fs-owner@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-fs digest..." > > > Today's Topics: > > 1. fetching atime information (Nikhil Gupta) > 2. Re: fetching atime information (Andrew Pantyukhin) > 3. Re: fetching atime information (Bill Vermillion) > 4. noatime on / and /var too ? (Gore Jarold) > 5. net/samba3 faults while try to export fusefs or msdosfs > volumes (Vladimir Grebenschikov) > 6. On-disk indexing for "Project Ideas" page (Nikolay Pavlov) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 7 Sep 2007 19:07:51 +0530 > From: "Nikhil Gupta" > Subject: fetching atime information > To: freebsd-fs@freebsd.org > Message-ID: > > Content-Type: text/plain; charset=3DISO-8859-1 > > Hi all, > > I am trying to find for all partitions whether atime update is enabled or > disabled. > What command can display the atime status (whether enabled or disabled) > information for all partitions on free bsd. > > for eg. in Linux, if I give mount command, it displays properties of each > partitions along with noatime (if atime update is disabled). > > Nikhil > > > ------------------------------ > > Message: 2 > Date: Fri, 7 Sep 2007 18:13:55 +0400 > From: Andrew Pantyukhin > Subject: Re: fetching atime information > To: Nikhil Gupta > Cc: freebsd-fs@freebsd.org > Message-ID: <20070907141354.GB56443@amilo.cenkes.org> > Content-Type: text/plain; charset=3Dus-ascii > > On Fri, Sep 07, 2007 at 07:07:51PM +0530, Nikhil Gupta wrote: > > Hi all, > > > > I am trying to find for all partitions whether atime update is enabled > or > > disabled. > > What command can display the atime status (whether enabled or disabled) > > information for all partitions on free bsd. > > > > for eg. in Linux, if I give mount command, it displays properties of > each > > partitions along with noatime (if atime update is disabled). > > Same here. > % mount|grep noatime > /dev/ad0s1f on /usr (ufs, NFS exported, local, noatime) > > > ------------------------------ > > Message: 3 > Date: Fri, 7 Sep 2007 10:33:59 -0400 > From: Bill Vermillion > Subject: Re: fetching atime information > To: Nikhil Gupta > Cc: freebsd-fs@freebsd.org > Message-ID: <20070907143359.GA94581@wjv.com> > Content-Type: text/plain; charset=3Dus-ascii > > On Fri, Sep 07, 2007 at 19:07 , while impersonating an expert on > the internet, Nikhil Gupta sent this to stdout: > > > Hi all, > > > > I am trying to find for all partitions whether atime update is enabled > or > > disabled. > > What command can display the atime status (whether enabled or disabled) > > information for all partitions on free bsd. > > > > for eg. in Linux, if I give mount command, it displays properties of > each > > partitions along with noatime (if atime update is disabled). > > Another poster showed the output of mount. > > You can also check the /etc/fstab to see and/or change which > file systems you set for 'noatime'. > > Bill > -- > Bill Vermillion - bv @ wjv . com > > > ------------------------------ > > Message: 4 > Date: Fri, 7 Sep 2007 15:17:09 -0700 (PDT) > From: Gore Jarold > Subject: noatime on / and /var too ? > To: freebsd-fs@freebsd.org > Message-ID: <704329.73647.qm@web63015.mail.re1.yahoo.com> > Content-Type: text/plain; charset=3Diso-8859-1 > > For performance, I have always set 'noatime' on my > large data partitions. > > For some odd reason I never bothered to set it on / > and /var ... is there any reason not to do that ? > > I know it won't change much since they are not busy > filesystems, but if there is no risk and no "best > practices" reason _not_ to do it, I might as well... > > right ? > > > > > _________________________________________________________________________= ___________ > Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, > news, photos & more. > http://mobile.yahoo.com/go?refer=3D1GNXIC > > > ------------------------------ > > Message: 5 > Date: Sat, 08 Sep 2007 01:45:09 +0400 > From: Vladimir Grebenschikov > Subject: net/samba3 faults while try to export fusefs or msdosfs > volumes > To: freebsd-fs@freebsd.org > Message-ID: <1189201509.1628.10.camel@localhost> > Content-Type: text/plain > > Hi > > Anybody already found that problem ? Probably some clues ? > > from /var/log/samba.log: > katis (172.22.2.11) connect to service C initially as user vova > (uid=3D207, gid=3D207) (pid 40339) > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(41) > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(42) > INTERNAL ERROR: Signal 6 in pid 40339 (3.0.25a) > Please read the Trouble-Shooting section of the Samba3-HOWTO > [2007/09/08 01:34:40, 0] lib/fault.c:fault_report(44) > > > -- > Vladimir B. Grebenschikov > vova@fbsd.ru > > > ------------------------------ > > Message: 6 > Date: Sat, 8 Sep 2007 12:43:44 +0300 > From: Nikolay Pavlov > Subject: On-disk indexing for "Project Ideas" page > To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org > Message-ID: <200709081243.48890.qpadla@gmail.com> > Content-Type: text/plain; charset=3D"utf-8" > > Recently while reading "Design and Implementation of FreeBSD operation > system" by Marshall Kirk McKusick and gnn i have found a very intresting > paragraph regarding UFS2 implementation, indexing and B-trees. According > to it on-disk indexes was not implemented, but some structures was > reserved for future development. May be some SOC students could implement > this in future. How about to adding this into Project Ideas page? > Let me quote from the paragraph "8.3 Naming": > > Finding of Names in Directories > > A common request to the filesystem is to lookup a specific name in a > directory. The kernel usually does the lookup by starting at the beginnin= g > of the directory and going through, comparing each entry in turn. First, > the length of the sought-after name is compared with the length of the > name being checked. If the lengths are identical, a string comparison of > the name being sought and the directory entry is made. If they match, the > search is complete; if they fail, either in the length or in the string > comparison, the search continues with the next entry. Whenever a name is > found, its name and containing directory are entered into the systemwide > name cache described in Section 6.6. Whenever a search is unsuccessful, a= n > entry is made in the cache showing that the name does not exist in the > particular directory. Before starting a directory scan, the kernel looks > for the name in the cache. If either a positive or negative entry is > found, the directory scan can be avoided. > > Another common operation is to lookup all the entries in a directory. For > example, many programs do a stat system call on each name in a directory > in the order that the names appear in the directory. To improve > performance for these programs, the kernel maintains the directory offset > of the last successful lookup for each directory. Each time that a lookup > is done in that directory, the search is started from the offset at which > the previous name was found (instead of from the beginning of the > directory). For programs that step sequentially through a directory with = n > files, search time decreases from Order(n2) to Order(n). > > One quick benchmark that demonstrates the maximum effectiveness of the > cache is running the ls -l command on a directory containing 600 files. O= n > a system that retains the most recent directory offset, the amount of > system time for this test is reduced by 85 percent. Unfortunately, the > maximum effectiveness is much greater than the average effectiveness. > Although the cache is 90 percent effective when hit, it is applicable to > only about 25 percent of the names being looked up. Despite the amount of > time spent in the lookup routine itself decreasing substantially, the > improvement is diminished because more time is spent in the routines that > that routine calls. Each cache miss causes a directory to be accessed > twice=97once to search from the middle to the end and once to search from > the beginning to the middle. > > These caches provide good directory lookup performance but are ineffectiv= e > for large directories that have a high rate of entry creation and > deletion. Each time a new directory entry is created, the kernel must sca= n > the entire directory to ensure that the entry does not already exist. Whe= n > an existing entry is deleted, the kernel must scan the directory to find > the entry to be removed. For directories with many entries these linear > scans are time-consuming. > > The approach to solving this problem in FreeBSD 5.2 is to introduce > dynamic > directory hashing that retrofits a directory indexing system to UFS [Dows= e > & Malone, 2002]. To avoid repeated linear searches of large directories, > the dynamic directory hashing builds a hash table of directory entries on > the fly when the directory is first accessed. This table avoids directory > scans on later lookups, creates, and deletes. Unlike filesystems > originally designed with large directories in mind, these indices are not > saved on disk and so the system is backward compatible. The drawback is > that the indices need to be built the first time that a large directory i= s > encountered after each system reboot. The effect of the dynamic directory > hashing is that large directories in UFS cause minimal performance > problems. > > When we built UFS2, we contemplated solving the large directory update > problem by changing to a more complex directory structure such as one tha= t > uses B-trees. This technique is used in many modern filesystems such as > XFS [Sweeney et al., 1996], JFS [Best & Kleikamp, 2003], ReiserFS [Reiser= , > 2001], and in later versions of Ext2 [Phillips, 2001]. We decided not to > make the change at the time that UFS2 was first implemented for several > reasons. First, we had limited time and resources, and we wanted to get > something working and stable that could be used in the time frame of > FreeBSD 5.2. By keeping the same directory format, we were able to reuse > all the directory code from UFS1, did not have to change numerous > filesystem utilities to understand and maintain a new directory format, > and were able to produce a stable and reliable filesystem in the time > frame available to us. The other reason that we felt that we could retain > the existing directory structure is because of the dynamic directory > hashing that was added to FreeBSD. > > Borrowing the technique used by the Ext2 filesystem a flag was also added > to show that an on-disk indexing structure is supported for directories > [Phillips, 2001]. This flag is unconditionally turned off by the existing > implementation of UFS. In the future, if an implementation of an on-disk > directory-indexing structure is added, the implementations that support i= t > will not turn the flag off. Index-supporting kernels will maintain the > indices and leave the flag on. If an old non-index-supporting kernel is > run, it will turn off the flag so that when the filesystem is once again > run under a new kernel, the new kernel will discover that the indexing > flag has been turned off and will know that the indices may be out date > and have to be rebuilt before being used. The only constraint on an > implementation of the indices is that they have to be an auxiliary data > structure that references the old linear directory format. > > > -- > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Best regards, Nikolay Pavlov. <<<----------------------------------- > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: not available > Type: application/pgp-signature > Size: 189 bytes > Desc: This is a digitally signed message part. > Url : > http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20070908/00697f= f9/attachment-0001.pgp > > ------------------------------ > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > End of freebsd-fs Digest, Vol 220, Issue 5 > ****************************************** > From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 11:08:03 2007 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70F6416A4CA for ; Mon, 10 Sep 2007 11:08:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4218913C48D for ; Mon, 10 Sep 2007 11:08:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l8AB83hT017231 for ; Mon, 10 Sep 2007 11:08:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l8AB82On017227 for freebsd-fs@FreeBSD.org; Mon, 10 Sep 2007 11:08:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Sep 2007 11:08:02 GMT Message-Id: <200709101108.l8AB82On017227@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 11:08:03 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/114856 fs [ntfs] [patch] Bug in NTFS allows bogus file modes. o bin/115165 fs [PATCH] amd(8): add functionality of mount_nfs' -L -a o kern/116170 fs Kernel panic when mounting /tmp 5 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] dirmask support for NTFS ala MSDOSFS 1 problem total. From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 21:58:13 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 639D316A418 for ; Mon, 10 Sep 2007 21:58:13 +0000 (UTC) (envelope-from cb@severious.net) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.freebsd.org (Postfix) with ESMTP id 4EAF113C474 for ; Mon, 10 Sep 2007 21:58:13 +0000 (UTC) (envelope-from cb@severious.net) Received: by ion.gank.org (Postfix, from userid 1001) id 0434811A7B; Mon, 10 Sep 2007 16:58:13 -0500 (CDT) Date: Mon, 10 Sep 2007 16:58:12 -0500 From: Craig Boston To: Gore Jarold Message-ID: <20070910215812.GB10142@nowhere> Mail-Followup-To: Craig Boston , Gore Jarold , freebsd-fs@freebsd.org References: <704329.73647.qm@web63015.mail.re1.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <704329.73647.qm@web63015.mail.re1.yahoo.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org Subject: Re: noatime on / and /var too ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 21:58:13 -0000 On Fri, Sep 07, 2007 at 03:17:09PM -0700, Gore Jarold wrote: > I know it won't change much since they are not busy > filesystems, but if there is no risk and no "best > practices" reason _not_ to do it, I might as well... I always set noatime on everything for years now and have never run into any problems with it. Unless you're specifically using atime for something (I think some news server software may use it), I can't think of a good reason to leave it enabled. Craig From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 22:07:52 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBC4F16A418 for ; Mon, 10 Sep 2007 22:07:52 +0000 (UTC) (envelope-from cb@severious.net) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.freebsd.org (Postfix) with ESMTP id AB77013C442 for ; Mon, 10 Sep 2007 22:07:52 +0000 (UTC) (envelope-from cb@severious.net) Received: by ion.gank.org (Postfix, from userid 1001) id 6974411A7B; Mon, 10 Sep 2007 17:07:52 -0500 (CDT) Date: Mon, 10 Sep 2007 17:07:51 -0500 From: Craig Boston To: Peter Schuller Message-ID: <20070910220751.GC10142@nowhere> Mail-Followup-To: Craig Boston , Peter Schuller , freebsd-fs@freebsd.org References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> <46E45E54.6040207@FreeBSD.org> <20070909221142.GA6435@hyperion.scode.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070909221142.GA6435@hyperion.scode.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 22:07:52 -0000 On Mon, Sep 10, 2007 at 12:11:42AM +0200, Peter Schuller wrote: > Kris Kenneway said: > > Unfortunately there are many ways in which this can fail, mostly involving > > external factors violating the assumptions upon which soft updates relies. > > For example, the data written on disk may not correspond to the data > > dispatched by soft updates, due to things like write caching in the > > hardware, write reordering, data corruption, unpredictable disk behaviour > > during power loss, hardware failure, etc. > > I am aware of this too (and paranoid about it). Although it's still branded experimental for now, you may want to look at ZFS after the 7.0 release. There's a whole host of things to consider (different performance characteristics, possible patent problems, etc), but it's one of the most paranoid filesystems I've seen. It doesn't really trust that the disk actually works correctly and goes to great lengths to recover from read failure or random data corruption. It still sometimes panics on write failure, but that may be considered a feature. Craig From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 22:13:49 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E5B916A419 for ; Mon, 10 Sep 2007 22:13:49 +0000 (UTC) (envelope-from cb@severious.net) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.freebsd.org (Postfix) with ESMTP id 782E313C442 for ; Mon, 10 Sep 2007 22:13:49 +0000 (UTC) (envelope-from cb@severious.net) Received: by ion.gank.org (Postfix, from userid 1001) id F1742117A8; Mon, 10 Sep 2007 17:13:48 -0500 (CDT) Date: Mon, 10 Sep 2007 17:13:48 -0500 From: Craig Boston To: freebsd-fs@freebsd.org Message-ID: <20070910221347.GD10142@nowhere> Mail-Followup-To: Craig Boston , freebsd-fs@freebsd.org References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> <46E45E54.6040207@FreeBSD.org> <20070909221142.GA6435@hyperion.scode.org> <20070910220751.GC10142@nowhere> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070910220751.GC10142@nowhere> User-Agent: Mutt/1.4.2.3i Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 22:13:49 -0000 On Mon, Sep 10, 2007 at 05:07:51PM -0500, Craig Boston wrote: > On Mon, Sep 10, 2007 at 12:11:42AM +0200, Peter Schuller wrote: > > Kris Kenneway said: s/Kenneway/Kennaway/ Sorry, there was no quote attribution so I typed it by hand to keep context. Must have had a brain lapse. From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 22:16:29 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A760016A418 for ; Mon, 10 Sep 2007 22:16:29 +0000 (UTC) (envelope-from jo_t@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 0263A13C457 for ; Mon, 10 Sep 2007 22:16:28 +0000 (UTC) (envelope-from jo_t@gmx.net) Received: (qmail invoked by alias); 10 Sep 2007 22:16:27 -0000 Received: from wh58-703.st.Uni-Magdeburg.DE (EHLO [192.168.73.100]) [141.44.198.73] by mail.gmx.net (mp056) with SMTP; 11 Sep 2007 00:16:27 +0200 X-Authenticated: #2964489 X-Provags-ID: V01U2FsdGVkX1/oGDlals/o2Eeonplx5P8aRDgzNEIB3W/1B1X0jg oD/1vDEmz5NHU7 Message-ID: <46E5C286.8010102@gmx.net> Date: Tue, 11 Sep 2007 00:17:42 +0200 From: Johannes Totz User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070728 Thunderbird/2.0.0.6 Mnenhy/0.7.5.666 MIME-Version: 1.0 To: Peter Schuller References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> In-Reply-To: <20070909200933.GA98161@hyperion.scode.org> X-Enigmail-Version: 0.95.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: freebsd-fs@freebsd.org, Kris Kennaway Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 22:16:29 -0000 Peter Schuller wrote: >> bg fsck cannot fix arbitrary filesystem corruption. Nor is it >> intended to. > > But is not the proposed problem that UFS caused the corruption to > begin with? That is want I wanted to report. This corruption should not have happened in the first place. Either some part in geom did not properly report the error back to the upper layer (e.g. the file system); or UFS ignored it silently (as there is nothing in the logs). In any case, the behavior is not correct. But I must also admit that I have no idea what the correct behavior would be like: the write error was most likely a metadata flush (or softupdate checkpoint, not sure of the terminology here) because the (file-)system was idle at that time. Cheers Johannes From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 23:00:54 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9AAE16A41A for ; Mon, 10 Sep 2007 23:00:54 +0000 (UTC) (envelope-from bv@bilver.wjv.com) Received: from wjv.com (fl-65-40-24-38.sta.embarqhsd.net [65.40.24.38]) by mx1.freebsd.org (Postfix) with ESMTP id 921F213C459 for ; Mon, 10 Sep 2007 23:00:53 +0000 (UTC) (envelope-from bv@bilver.wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by wjv.com (8.14.1/8.13.1) with ESMTP id l8AN0UDw092308; Mon, 10 Sep 2007 19:00:30 -0400 (EDT) (envelope-from bv@bilver.wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.14.1/8.13.1/Submit) id l8AN0O9u092307; Mon, 10 Sep 2007 19:00:24 -0400 (EDT) (envelope-from bv) Date: Mon, 10 Sep 2007 19:00:24 -0400 From: Bill Vermillion To: Craig Boston , Gore Jarold , freebsd-fs@freebsd.org Message-ID: <20070910230024.GA92246@wjv.com> References: <704329.73647.qm@web63015.mail.re1.yahoo.com> <20070910215812.GB10142@nowhere> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070910215812.GB10142@nowhere> User-Agent: Mutt/1.4.2.2i Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com Cc: Subject: Re: noatime on / and /var too ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bv@wjv.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2007 23:00:54 -0000 On Mon, Sep 10, 2007 at 16:58 Craig Boston saw "Error reading FAT table? Try SKINNY table?" And promptly said: > On Fri, Sep 07, 2007 at 03:17:09PM -0700, Gore Jarold wrote: > > I know it won't change much since they are not busy > > filesystems, but if there is no risk and no "best > > practices" reason _not_ to do it, I might as well... > I always set noatime on everything for years now and have never > run into any problems with it. > Unless you're specifically using atime for something (I think > some news server software may use it), I can't think of a good > reason to leave it enabled. > Craig I've not seen news software use that by default, but I do definately run no atime on my news server. Even though it's small adding 100s to 1000s each day and expiring each night make it really un-neccesary to retain the atimes on those. I'd say that anything that gets a lot of access to many files, and there is no modification to those - archival files, web files, etc - that would be a good reason to turn on atime IMO. I only use it on user file systems and not on / or /var. Bill -- Bill Vermillion - bv @ wjv . com From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 04:07:25 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85EE716A417 for ; Tue, 11 Sep 2007 04:07:25 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 3D5BD13C457 for ; Tue, 11 Sep 2007 04:07:25 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1IUpOV-0002i1-2N for freebsd-fs@freebsd.org; Mon, 10 Sep 2007 21:58:50 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 10 Sep 2007 21:58:03 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 10 Sep 2007 21:58:03 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Mon, 10 Sep 2007 11:07:23 +0200 Lines: 32 Message-ID: References: <855622.76488.qm@web36601.mail.mud.yahoo.com> <20070908215102.GA8291@hyperion.scode.org> <46E3519E.3040001@FreeBSD.org> <20070909100041.GA60318@hyperion.scode.org> <20070909195727.GA97233@hyperion.scode.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-ripemd160; protocol="application/pgp-signature"; boundary="------------enig5F5455D324CF031B5B6F1B65" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 1.5.0.12 (X11/20060911) In-Reply-To: <20070909195727.GA97233@hyperion.scode.org> X-Enigmail-Version: 0.94.4.0 Sender: news X-UiO-SPF-Received: Received-SPF: pass (mail-mx6.uio.no: domain of sea.gmane.org designates 80.91.229.5 as permitted sender) client-ip=80.91.229.5; envelope-from=news@sea.gmane.org; helo=sea.gmane.org; X-UiO-Spam-info: not spam, SpamAssassin (score=-3.0, required=12.0, autolearn=disabled, UIO_RECEIVED_FROM_NORWAY=-3) X-UiO-Scanned: BA5659D7436362B3A697436972C4C8293005877C X-UiO-SPAM-Test: remote_host: 80.91.229.5 spam_score: -29 maxlevel 200 minaction 2 bait 0 mail/h: 5 total 262 max/h 18 blacklist 0 greylist 0 ratelimit 0 Subject: Re: Swapping on ZFS - stability issue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2007 04:07:25 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5F5455D324CF031B5B6F1B65 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Peter Schuller wrote: >> Ok, so a known issue. Unless someone tells me otherwise I'll update >> the ZFS-on-FreeBSD Wiki pages to warn about this issue (assuming it's >> editable by me). >=20 > It is not editable by me. So if anyone with edit privileges wants to, > what I am referring to is the quickstart guide which has swap-on-zfs > as an explicit example, thus implying it will actually work ;) Done. --------------enig5F5455D324CF031B5B6F1B65 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFG5QlMldnAQVacBcgRA5z7AKCaGap9d7SZXDFJNuIeskA1K2FDMACg0Ula +KNqq8tfyDmWHKfOzvw6LvE= =W2rw -----END PGP SIGNATURE----- --------------enig5F5455D324CF031B5B6F1B65-- From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 08:40:14 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAEAB16A41A for ; Tue, 11 Sep 2007 08:40:14 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id A810313C457 for ; Tue, 11 Sep 2007 08:40:14 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1IV1I2-0000ox-HS for freebsd-fs@freebsd.org; Tue, 11 Sep 2007 10:40:10 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 11 Sep 2007 10:40:10 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 11 Sep 2007 10:40:10 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Tue, 11 Sep 2007 10:40:02 +0200 Lines: 35 Message-ID: References: <46E4225F.1020806@gmx.net> <46E42D14.5060605@FreeBSD.org> <20070909200933.GA98161@hyperion.scode.org> <46E5C286.8010102@gmx.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-ripemd160; protocol="application/pgp-signature"; boundary="------------enigFE4FFBF22490163F316D9C2A" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 1.5.0.12 (X11/20060911) In-Reply-To: <46E5C286.8010102@gmx.net> X-Enigmail-Version: 0.94.4.0 Sender: news Subject: Re: UFS not handling errors correctly X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2007 08:40:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigFE4FFBF22490163F316D9C2A Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Johannes Totz wrote: > Either some part in geom did not properly report the error back to the > upper layer (e.g. the file system); or UFS ignored it silently (as ther= e > is nothing in the logs). > In any case, the behavior is not correct. (Maybe the drive didn't report the error?) UFS (at least in [Free]BSD) is notorious for not handling lower-level=20 errors and "unusual events" properly, so it's a likely culprit. --------------enigFE4FFBF22490163F316D9C2A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFG5lRildnAQVacBcgRAzk6AJ4wwoHn5JFh7ojVNEAU3yfVd5GZ5QCgkwS/ a1Vd5JuKKwqspgBRrHuJzw4= =Gv4a -----END PGP SIGNATURE----- --------------enigFE4FFBF22490163F316D9C2A-- From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 09:41:00 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6384916A41B for ; Tue, 11 Sep 2007 09:41:00 +0000 (UTC) (envelope-from alexandr.kovalenko@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.235]) by mx1.freebsd.org (Postfix) with ESMTP id 3467913C467 for ; Tue, 11 Sep 2007 09:41:00 +0000 (UTC) (envelope-from alexandr.kovalenko@gmail.com) Received: by wx-out-0506.google.com with SMTP id i29so1281964wxd for ; Tue, 11 Sep 2007 02:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=vG8JSLtftxvAbqed8FwAI7bYe5jnAeIrplwvYSRmB5c=; b=FvQqJjFbHsh8H8K2s30L2sEEmK0mPdJc4lV6qF+IN4gFziR+xkPj1KJEjCg8gmV121kpJFi/E3OvRUbcGvYTngUDY2EK8Ukioln+ZONoJA6PDIfu6cWGpPgUZTXhlcmgPm6f6u6RXBtDkyGoPyG3xiAhBY7Jh/WLtvXnCQp7xEc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=BSN+c026DEMln1OUP71/jKsJo/4nkzBUZqIvVwivzU+FrIFlNGm+b5BASgJ7ElPiwJUxK15FYbZJR/P+6u6uc6NoNIr4v9kiT7ENlpr6CyggJaKqb9RLxNqrNffNv7MAivtgaY/5e3MZX749ad6kXozmLYtxprwwWhuBA1xkG+0= Received: by 10.90.63.16 with SMTP id l16mr12471623aga.1189501939099; Tue, 11 Sep 2007 02:12:19 -0700 (PDT) Received: by 10.35.33.14 with HTTP; Tue, 11 Sep 2007 02:12:19 -0700 (PDT) Message-ID: <1d2641260709110212h73a0f304ie8ca4f5cac448d99@mail.gmail.com> Date: Tue, 11 Sep 2007 12:12:19 +0300 From: "Alexandr Kovalenko" To: bv@wjv.com In-Reply-To: <20070910230024.GA92246@wjv.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <704329.73647.qm@web63015.mail.re1.yahoo.com> <20070910215812.GB10142@nowhere> <20070910230024.GA92246@wjv.com> Cc: freebsd-fs@freebsd.org, Gore Jarold , Craig Boston Subject: Re: noatime on / and /var too ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2007 09:41:00 -0000 2007/9/11, Bill Vermillion : > On Mon, Sep 10, 2007 at 16:58 Craig Boston saw "Error reading FAT > table? Try SKINNY table?" And promptly said: > > > On Fri, Sep 07, 2007 at 03:17:09PM -0700, Gore Jarold wrote: > > > I know it won't change much since they are not busy > > > filesystems, but if there is no risk and no "best > > > practices" reason _not_ to do it, I might as well... > > > I always set noatime on everything for years now and have never > > run into any problems with it. > > > Unless you're specifically using atime for something (I think > > some news server software may use it), I can't think of a good > > reason to leave it enabled. > > > Craig > > I've not seen news software use that by default, but I do > definately run no atime on my news server. Even though it's small > adding 100s to 1000s each day and expiring each night make it > really un-neccesary to retain the atimes on those. > > I'd say that anything that gets a lot of access to many files, > and there is no modification to those - archival files, web files, > etc - that would be a good reason to turn on atime IMO. > > I only use it on user file systems and not on / or /var. setting noatime to /var may confuse some mail clients, ie mutt -- Alexandr Kovalenko http://uafug.org.ua/