From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 04:21:36 2008 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E25816A468; Sun, 27 Jan 2008 04:21:36 +0000 (UTC) (envelope-from mi+kde@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 4858913C46A; Sun, 27 Jan 2008 04:21:35 +0000 (UTC) (envelope-from mi+kde@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.1/8.14.1) with ESMTP id m0R3r3K9094543 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Jan 2008 22:53:03 -0500 (EST) (envelope-from mi+kde@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by aldan.algebra.com (8.14.1/8.14.1/Submit) id m0R3r3ii094542; Sat, 26 Jan 2008 22:53:03 -0500 (EST) (envelope-from mi+kde@aldan.algebra.com) From: Mikhail Teterin To: freebsd-questions@FreeBSD.org Date: Sat, 26 Jan 2008 22:53:02 -0500 User-Agent: KMail/1.9.7 X-Face: %UW#n0|w>ydeGt/b@1-.UFP=K^~-:0f#O:D7whJ5G_<5143Bb3kOIs9XpX+"V+~$adGP:J|SLieM31VIhqXeLBli" Cc: hardware@FreeBSD.org, fs@FreeBSD.org Subject: mounting/reading a DVD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 04:21:37 -0000 Hello! I finally got to opening a DVD I received for New Year and wanted to back it up before watching. I mounted the disk: /dev/acd0 on /cdrom (cd9660, local, read-only) and I can list the contents: env LANG=C ls -l /cdrom/ total 8 dr-xr-xr-x 1 root wheel 2048 Oct 6 2005 audio_ts dr-xr-xr-x 1 root wheel 2048 Oct 6 2005 jacket_p dr-xr-xr-x 1 root wheel 4096 Oct 6 2005 video_ts But when I try to copy all that to a hard-drive, I get a ton of read-errors -- most of the many files on the disk are unreadable: ... g_vfs_done():acd0[READ(offset=4623824896, length=65536)]error = 5 acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 g_vfs_done():acd0[READ(offset=4623828992, length=65536)]error = 5 acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 ... ``dd'' refuses to read from /dev/acd0: dd: /dev/acd0: Invalid argument cat tries to, but fails: cat: stdin: Input/output error Is there a step I'm missing? I strongly doubt, the disk is damaged, as I just unwrapped it myself... I'm on FreeBSD/amd64 running 6.3 as of Dec 30th. The DVD-drive is: acd0: DVDR at ata1-master UDMA66 Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 04:55:29 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FA6816A421 for ; Sun, 27 Jan 2008 04:55:29 +0000 (UTC) (envelope-from cryptwizard@gmail.com) Received: from po-out-1718.google.com (po-out-1718.google.com [72.14.252.154]) by mx1.freebsd.org (Postfix) with ESMTP id 457D013C461 for ; Sun, 27 Jan 2008 04:55:29 +0000 (UTC) (envelope-from cryptwizard@gmail.com) Received: by po-out-1718.google.com with SMTP id a23so1132514poh.3 for ; Sat, 26 Jan 2008 20:55:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=AQUeBJvfGc7K7Rj8twNO0fB1GUyB1yN+dokUzW2ISxw=; b=Z8emNJdR23Ov8fvk3Uap+hcVGWDb/Xp7xcJTfsHX9/rkmNzpXvGm6r/JCvHUy6NDX23vhCn5c04L0+ATp8TvBr4Nh/rl6ZYrfbIolCq2JWsS/wBPZx0oVtfAmo7yDCf0izwl/oFfNfjuCBA5mjesgD38+lgOX80rAKdA7iB7B8M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=P6g1wN6h0GUUGGSagV//ZA1U3D0+SO3Rw6vyMXprUH8WgSe7b5siKG2LqkJ8XoAC4OMD3d+RokRvAI2lBD9Vbpl1H1ev3g/U+zZbYry4Fs1e20UTb2NkFjWFaS9SRIdpFXxDnTk3cqo9T5uNX62001JfXJKcHqfl+AwXAe3BfQs= Received: by 10.141.28.12 with SMTP id f12mr2572072rvj.1.1201408741756; Sat, 26 Jan 2008 20:39:01 -0800 (PST) Received: by 10.140.143.6 with HTTP; Sat, 26 Jan 2008 20:39:01 -0800 (PST) Message-ID: Date: Sun, 27 Jan 2008 15:39:01 +1100 From: CryptWizard To: "Mikhail Teterin" In-Reply-To: <200801262253.03019@aldan> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200801262253.03019@aldan> Cc: freebsd-questions@freebsd.org, fs@freebsd.org, hardware@freebsd.org Subject: Re: mounting/reading a DVD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 04:55:29 -0000 It's because the DVD is copy protected. On Jan 27, 2008 2:53 PM, Mikhail Teterin wrote: > Hello! > > I finally got to opening a DVD I received for New Year and wanted to back it > up before watching. > > I mounted the disk: > > /dev/acd0 on /cdrom (cd9660, local, read-only) > > and I can list the contents: > > env LANG=C ls -l /cdrom/ > total 8 > dr-xr-xr-x 1 root wheel 2048 Oct 6 2005 audio_ts > dr-xr-xr-x 1 root wheel 2048 Oct 6 2005 jacket_p > dr-xr-xr-x 1 root wheel 4096 Oct 6 2005 video_ts > > > But when I try to copy all that to a hard-drive, I get a ton of read-errors -- > most of the many files on the disk are unreadable: > > ... > g_vfs_done():acd0[READ(offset=4623824896, length=65536)]error = 5 > acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 > g_vfs_done():acd0[READ(offset=4623828992, length=65536)]error = 5 > acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 > ... > > ``dd'' refuses to read from /dev/acd0: > > dd: /dev/acd0: Invalid argument > > cat tries to, but fails: > > cat: stdin: Input/output error > > Is there a step I'm missing? I strongly doubt, the disk is damaged, as I just > unwrapped it myself... > > I'm on FreeBSD/amd64 running 6.3 as of Dec 30th. The DVD-drive is: > > acd0: DVDR at ata1-master UDMA66 > > Thanks! > > -mi > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 06:06:43 2008 Return-Path: Delivered-To: fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E38816A419; Sun, 27 Jan 2008 06:06:43 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.freebsd.org (Postfix) with ESMTP id 1BED313C458; Sun, 27 Jan 2008 06:06:42 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.14.2/8.14.1) with ESMTP id m0R5awHI068127; Sun, 27 Jan 2008 00:36:58 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.14.2/8.14.1/Submit) id m0R5aw5V068126; Sun, 27 Jan 2008 00:36:58 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Sun, 27 Jan 2008 00:36:58 -0500 From: David Schultz To: Mikhail Teterin Message-ID: <20080127053658.GA68060@VARK.MIT.EDU> Mail-Followup-To: Mikhail Teterin , freebsd-questions@FreeBSD.ORG, hardware@FreeBSD.ORG, fs@FreeBSD.ORG References: <200801262253.03019@aldan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200801262253.03019@aldan> Cc: freebsd-questions@FreeBSD.ORG, fs@FreeBSD.ORG, hardware@FreeBSD.ORG Subject: Re: mounting/reading a DVD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 06:06:43 -0000 On Sat, Jan 26, 2008, Mikhail Teterin wrote: > g_vfs_done():acd0[READ(offset=4623824896, length=65536)]error = 5 > acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 > g_vfs_done():acd0[READ(offset=4623828992, length=65536)]error = 5 > acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x6f ascq=0x04 > ... > > ``dd'' refuses to read from /dev/acd0: > > dd: /dev/acd0: Invalid argument > > cat tries to, but fails: > > cat: stdin: Input/output error > > Is there a step I'm missing? I strongly doubt, the disk is damaged, as I just > unwrapped it myself... libdvdcss can probably help, and you can use mplayer or ogle as a front end to it. mplayer has an option to copy the tracks to a file as well. It may be necessary to set the drive's region code to match the disc's before it allows you to read the track keys. There's an ioctl for that... From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 11:13:57 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 240AF16A473 for ; Sun, 27 Jan 2008 11:13:57 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id CF87B13C459 for ; Sun, 27 Jan 2008 11:13:56 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JJ5SQ-0006VQ-OU for freebsd-fs@freebsd.org; Sun, 27 Jan 2008 11:13:50 +0000 Received: from host71-40-static.74-81-b.business.telecomitalia.it ([81.74.40.71]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 27 Jan 2008 11:13:50 +0000 Received: from lapo by host71-40-static.74-81-b.business.telecomitalia.it with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 27 Jan 2008 11:13:50 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Lapo Luchini Date: Sun, 27 Jan 2008 12:13:46 +0100 Lines: 14 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: host71-40-static.74-81-b.business.telecomitalia.it User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.9) Gecko/20071031 Thunderbird/2.0.0.9 Mnenhy/0.7.5.0 X-Enigmail-Version: 0.95.6 OpenPGP: id=C8F252FB; url=http://www.lapo.it/pgpkey.txt Sender: news Subject: dumpdev on encrypted swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 11:13:57 -0000 Is it possible to use an automatically-encrypted GELI swap (such as putting ad0s1b.eli in the fstab) as a dump device? Of course ad0s1b.eli is not useful, as dump would be one-time encrypted and unreadable at next boot. dumpdev="ad0s1b" should work (I guess at dump time swap is used no more, so it wouldn't overwrite the crashdump), but of course ad0s1b is a provider consumed by ad0s1b.eli... so kern.geom.debugflags shuold probably be used to allow that, but I don't feel very at home with the idea that then *every* device is protected no more by accidental overwrite. Is there a way to have that kind of configuration "automatically work"? (other than, I guess, hack the dump code to set the debugflags itself just before attempting the dump, or something like that) Lapo From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 19:53:41 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD0E416A417 for ; Sun, 27 Jan 2008 19:53:41 +0000 (UTC) (envelope-from joe@skyrush.com) Received: from shadow.wildlava.net (shadow.wildlava.net [67.40.138.81]) by mx1.freebsd.org (Postfix) with ESMTP id 8062413C45B for ; Sun, 27 Jan 2008 19:53:41 +0000 (UTC) (envelope-from joe@skyrush.com) Received: from crater.wildlava.net (crater.wildlava.net [67.40.138.82]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shadow.wildlava.net (Postfix) with ESMTP id 3F3FC8F165; Sun, 27 Jan 2008 12:33:52 -0700 (MST) Message-ID: <479CDC9E.8040604@skyrush.com> Date: Sun, 27 Jan 2008 12:33:50 -0700 From: Joe Peterson User-Agent: Thunderbird 2.0.0.9 (X11/20071208) MIME-Version: 1.0 To: pjd@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Unexpected "resilver" after reboot (after scrub found CKSUM problems) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 19:53:41 -0000 Hi Pawel (or anyone else who might know), I had a strange thing happen on ZFS the other day, and I cannot find any info about it on the web - thought you might have some ideas. I am using 7.0-RC1 at the moment. I found a checksum error in ZFS during a scrub. This is strange in itself, since I believe the disk is OK (see below): pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 ad0s1d ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: /home/joe/music/jukebox/christmas/Esquivel/Merry_XMas_from_the_SpaceAge_Bachelor_Pad/07-Snowfall.mp3 This is how it appears after a recent reboot, however. After a scrub, I see varying number of non-zero counts under CKSUM. Not sure why it is zero after reboot (maybe that's normal). However, the strange this is that after my first reboot after the scrub found the issue, zpool status told me that "resilver completed with 0 errors", and there were no known errors. Only trying to read the file and/or rescrubbing returned the status to the error state and made the CKSUM column non-zero. Since I do not have a mirror or raid config, I'm not sure why it would resilver at all, and I did nothing explicit to cause a resilver (as far as I know)... Any ideas? As an aside, I, along with some others on freebsd-stable@freebsd.org, have been seeing what "look" like disk errors in the system logs. I have a suspicion that there could be some other cause (lots of discussion on that list, if you are interested). Strangely, this disk checks out fine on both short and long tests in Seatools, and smartctl shows it as OK. Also, using Linux to do lots of reads from it does not show any issue or error logs. At this point, I am not sure if the CKSUM issue is a real HW flaw or something else... Thanks, Joe From owner-freebsd-fs@FreeBSD.ORG Sun Jan 27 21:09:09 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D9A216A418; Sun, 27 Jan 2008 21:09:09 +0000 (UTC) (envelope-from mi+kde@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id B696E13C45A; Sun, 27 Jan 2008 21:09:08 +0000 (UTC) (envelope-from mi+kde@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.1/8.14.1) with ESMTP id m0RL97OU002312 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 27 Jan 2008 16:09:07 -0500 (EST) (envelope-from mi+kde@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by aldan.algebra.com (8.14.1/8.14.1/Submit) id m0RL97uF002311; Sun, 27 Jan 2008 16:09:07 -0500 (EST) (envelope-from mi+kde@aldan.algebra.com) From: Mikhail Teterin To: CryptWizard Date: Sun, 27 Jan 2008 16:09:06 -0500 User-Agent: KMail/1.9.7 References: <200801262253.03019@aldan> In-Reply-To: X-Face: %UW#n0|w>ydeGt/b@1-.UFP=K^~-:0f#O:D7whJ5G_<5143Bb3kOIs9XpX+"V+~$adGP:J|SLieM31VIhqXeLBli" Cc: freebsd-questions@freebsd.org, fs@freebsd.org, hardware@freebsd.org Subject: Re: mounting/reading a DVD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 21:09:09 -0000 On =D3=D5=C2=CF=D4=C1 26 =D3=A6=DE=C5=CE=D8 2008, CryptWizard wrote: =3D It's because the DVD is copy protected. Yes, I guess so... Using ddrescue, as described in=20 http://gentoo-wiki.com/HOWTO_Backup_a_DVD#ARccOS_.26_Other_intentional_sect= or_corruption seems to have extracted an ISO-image... -mi From owner-freebsd-fs@FreeBSD.ORG Mon Jan 28 11:07:00 2008 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F3B916A4AB for ; Mon, 28 Jan 2008 11:07:00 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7E56413C4D9 for ; Mon, 28 Jan 2008 11:07:00 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m0SB702R016293 for ; Mon, 28 Jan 2008 11:07:00 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m0SB6xWS016289 for freebsd-fs@FreeBSD.org; Mon, 28 Jan 2008 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 28 Jan 2008 11:06:59 GMT Message-Id: <200801281106.m0SB6xWS016289@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2008 11:07:00 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/116170 fs [panic] Kernel panic when mounting /tmp 3 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o bin/118249 fs mv(1): moving a directory changes its mtime 2 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Jan 28 14:40:02 2008 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5577116A417 for ; Mon, 28 Jan 2008 14:40:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4560413C467 for ; Mon, 28 Jan 2008 14:40:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m0SEe2d6038675 for ; Mon, 28 Jan 2008 14:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m0SEe2Nb038674; Mon, 28 Jan 2008 14:40:02 GMT (envelope-from gnats) Date: Mon, 28 Jan 2008 14:40:02 GMT Message-Id: <200801281440.m0SEe2Nb038674@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Joe Peterson Cc: Subject: Re: bin/118249: mv(1): moving a directory changes its mtime X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joe Peterson List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2008 14:40:02 -0000 The following reply was made to PR bin/118249; it has been noted by GNATS. From: Joe Peterson To: bug-followup@FreeBSD.org Cc: Subject: Re: bin/118249: mv(1): moving a directory changes its mtime Date: Mon, 28 Jan 2008 07:14:09 -0700 A couple of notes on this... * It does not happen with, e.g., ZFS, so it appears to be UFS-specific. * When doing cp -pR, it does not happen. * Along these lines, I tried moving files and dirs *across* file systems, and it does not happen. So it's the case of moving a dir to another dir in the *same* fs that invokes the issue. Since moving across filesystems (according to the man page) uses "rm -f dest && cp -pRP source dest && rm -rf source" rather than rename, this makes sense. P.S. I used the web-form to submit the bug, so the text box must treat the text as one long line (hence the 417 character line). From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 17:06:03 2008 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD26516A421 for ; Wed, 30 Jan 2008 17:06:03 +0000 (UTC) (envelope-from mi+mill@aldan.algebra.com) Received: from mail1.sea5.speakeasy.net (mail1.sea5.speakeasy.net [69.17.117.3]) by mx1.freebsd.org (Postfix) with ESMTP id B6E2013C467 for ; Wed, 30 Jan 2008 17:06:03 +0000 (UTC) (envelope-from mi+mill@aldan.algebra.com) Received: (qmail 15326 invoked from network); 30 Jan 2008 16:39:23 -0000 Received: from aldan.algebra.com (HELO aldan-mlp) ([216.254.65.224]) (envelope-sender ) by mail1.sea5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 30 Jan 2008 16:39:23 -0000 From: Mikhail Teterin To: questions@freebsd.org Date: Wed, 30 Jan 2008 11:39:21 -0500 User-Agent: KMail/1.7.1 Organization: Virtual Estates, Inc. MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200801301139.21989.mi+mill@aldan.algebra.com> Cc: fs@freebsd.org Subject: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 17:06:03 -0000 Hello! I'd like the entire contents of each user's .mozilla/firefox/*/Cache directory to be excluded from the regular filesystem dumps. Running ``chflags -R nodump /home/*/.mozilla/firefox/*/Cache'' does the trick, but this needs to be redone daily -- prior to running the backup -- because new entries appear in the caches, obviously... The new entries don't have the nodump flag set. Is there a way, the flag can be set automatically? For example, inherited from the directory? Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 18:13:58 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09E3316A60E; Wed, 30 Jan 2008 18:13:58 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id C070A13C4CC; Wed, 30 Jan 2008 18:13:57 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id DC22F208F; Wed, 30 Jan 2008 18:54:02 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.2/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id CDFE6208E; Wed, 30 Jan 2008 18:54:02 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id B3C2784487; Wed, 30 Jan 2008 18:54:02 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mikhail Teterin References: <200801301139.21989.mi+mill@aldan.algebra.com> Date: Wed, 30 Jan 2008 18:54:02 +0100 In-Reply-To: <200801301139.21989.mi+mill@aldan.algebra.com> (Mikhail Teterin's message of "Wed\, 30 Jan 2008 11\:39\:21 -0500") Message-ID: <86tzkvurlx.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 18:13:58 -0000 Mikhail Teterin writes: > I'd like the entire contents of each user's .mozilla/firefox/*/Cache > directory to be excluded from the regular filesystem dumps. > > Running ``chflags -R nodump /home/*/.mozilla/firefox/*/Cache'' does > the trick, but this needs to be redone daily -- prior to running the > backup -- because new entries appear in the caches, obviously... The > new entries don't have the nodump flag set. > > Is there a way, the flag can be set automatically? For example, > inherited from the directory? Thanks! Hmm, shoudn't whichever backup tool you're using stop when it encounters a nodump directory? You shouldn't need to set nodump on the files themselves. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 18:50:05 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9510E16A41B; Wed, 30 Jan 2008 18:50:05 +0000 (UTC) (envelope-from bp@barryp.org) Received: from eden.barryp.org (host-42-60-230-24.midco.net [24.230.60.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7303713C4D1; Wed, 30 Jan 2008 18:50:05 +0000 (UTC) (envelope-from bp@barryp.org) Received: from geo.med.und.nodak.edu ([134.129.166.11]) by eden.barryp.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67 (FreeBSD)) (envelope-from ) id 1JKHQK-000DuA-CS; Wed, 30 Jan 2008 12:12:36 -0600 Message-ID: <47A0BE14.3040804@barryp.org> Date: Wed, 30 Jan 2008 12:12:36 -0600 From: Barry Pederson User-Agent: Thunderbird 2.0.0.9 (X11/20071031) MIME-Version: 1.0 To: Mikhail Teterin References: <200801301139.21989.mi+mill@aldan.algebra.com> In-Reply-To: <200801301139.21989.mi+mill@aldan.algebra.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 18:50:05 -0000 Mikhail Teterin wrote: > Hello! > > I'd like the entire contents of each user's .mozilla/firefox/*/Cache directory > to be excluded from the regular filesystem dumps. > > Running ``chflags -R nodump /home/*/.mozilla/firefox/*/Cache'' does the trick, > but this needs to be redone daily -- prior to running the backup -- because > new entries appear in the caches, obviously... The new entries don't have the > nodump flag set. > > Is there a way, the flag can be set automatically? For example, inherited from > the directory? Thanks! The dump man page for 6.2 says: Directories and regular files which have their ``nodump'' flag (UF_NODUMP) set will be omitted along with everything under such directories, subject to the -h option. So if the "Cache" directories themselves are flagged, I think you'd be OK. Maybe chflags nodump /home/*/.mozilla/firefox/*/Cache (without the -R) would be a good thing to quickly make sure the directories are flagged in case the users delete/recreate the Cache folders, but wouldn't take as long as flagging each and every cache file. You'd probably also want to add "-h 0" to the dump args, otherwise the cache files would be included in level 0 dumps. Barry From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 21:12:19 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B398516A420; Wed, 30 Jan 2008 21:12:19 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id 6D3FF13C457; Wed, 30 Jan 2008 21:12:19 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost.mckusick.com [127.0.0.1]) by chez.mckusick.com (8.13.8/8.13.6) with ESMTP id m0UKlV0m004741; Wed, 30 Jan 2008 12:47:39 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <200801302047.m0UKlV0m004741@chez.mckusick.com> To: Mikhail Teterin Date: Wed, 30 Jan 2008 12:47:31 -0800 From: Kirk McKusick Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 21:12:19 -0000 > From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= > To: Mikhail Teterin > Date: Wed, 30 Jan 2008 18:54:02 +0100 > Cc: questions@freebsd.org, fs@freebsd.org > Subject: Re: Automatic `nodump' flag? > > Mikhail Teterin writes: > > I'd like the entire contents of each user's .mozilla/firefox/*/Cache > > directory to be excluded from the regular filesystem dumps. > > > > Running ``chflags -R nodump /home/*/.mozilla/firefox/*/Cache'' does > > the trick, but this needs to be redone daily -- prior to running the > > backup -- because new entries appear in the caches, obviously... The > > new entries don't have the nodump flag set. > > > > Is there a way, the flag can be set automatically? For example, > > inherited from the directory? Thanks! > > Hmm, shoudn't whichever backup tool you're using stop when it encounters > a nodump directory? You shouldn't need to set nodump on the files > themselves. > > DES > -- > Dag-Erling Smørgrav - des@des.no The dump program runs on the raw disk partition dumping sequentially by inode number. So, it has no idea of the file-tree hierarchy. Thus any propagation of the "nodump" flag would have to be done by the filesystem (or by using a different archiving program). It seems to me that possible changes that could be made would be to update the semantics of the existing "nodump" flag to say that if it is set on a directory, then any new files or directories created within that directory would also have the "nodump" flag set. Or a new "nodumpall" flag could be added that when set on a directory would propagate to any new files or directories created within that directory. Both of these would be easy to implement. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 22:13:58 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE06316A418; Wed, 30 Jan 2008 22:13:58 +0000 (UTC) (envelope-from bp@barryp.org) Received: from eden.barryp.org (host-42-60-230-24.midco.net [24.230.60.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8AF6113C4F4; Wed, 30 Jan 2008 22:13:58 +0000 (UTC) (envelope-from bp@barryp.org) Received: from geo.med.und.nodak.edu ([134.129.166.11]) by eden.barryp.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67 (FreeBSD)) (envelope-from ) id 1JKLBq-000FH2-Bz; Wed, 30 Jan 2008 16:13:54 -0600 Message-ID: <47A0F6A2.9060600@barryp.org> Date: Wed, 30 Jan 2008 16:13:54 -0600 From: Barry Pederson User-Agent: Thunderbird 2.0.0.9 (X11/20071031) MIME-Version: 1.0 To: Kirk McKusick References: <200801302047.m0UKlV0m004741@chez.mckusick.com> In-Reply-To: <200801302047.m0UKlV0m004741@chez.mckusick.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 22:13:58 -0000 Kirk McKusick wrote: > The dump program runs on the raw disk partition dumping sequentially > by inode number. So, it has no idea of the file-tree hierarchy. I was just looking at the source to dump, specifically traverse.c and from what I can see, doesn't dump pass #2 scan through all directories and then in the searchdir() function remove a directory's children from the list of inodes to backup if the directory has the nodump flag? --------- 414 if (nodump) { 415 ip = getino(dp->d_ino, &mode); 416 if (TSTINO(dp->d_ino, dumpinomap)) { 417 CLRINO(dp->d_ino, dumpinomap); 418 *tapesize -= blockest(ip); --------- Barry From owner-freebsd-fs@FreeBSD.ORG Wed Jan 30 22:37:44 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAC2516A41A; Wed, 30 Jan 2008 22:37:44 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [IPv6:2001:4070:101:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 52E2E13C467; Wed, 30 Jan 2008 22:37:32 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (localhost [IPv6:::1]) by wojtek.tensor.gdynia.pl (8.13.8/8.13.8) with ESMTP id m0UMXeJV007389; Wed, 30 Jan 2008 23:33:40 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.13.8/8.13.8/Submit) with ESMTP id m0UMXVIi007386; Wed, 30 Jan 2008 23:33:34 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 30 Jan 2008 23:33:31 +0100 (CET) From: Wojciech Puchar To: Kirk McKusick In-Reply-To: <200801302047.m0UKlV0m004741@chez.mckusick.com> Message-ID: <20080130233244.L7385@wojtek.tensor.gdynia.pl> References: <200801302047.m0UKlV0m004741@chez.mckusick.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Mikhail Teterin , questions@freebsd.org, fs@freebsd.org, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 22:37:44 -0000 > The dump program runs on the raw disk partition dumping sequentially > by inode number. So, it has no idea of the file-tree hierarchy. Thus > any propagation of the "nodump" flag would have to be done by the Yes, dump DO descend to check for nodump flags - while doing backup for raw filesystem. there is enough to set nodump flag to directory, and no files in it will be backed up From owner-freebsd-fs@FreeBSD.ORG Thu Jan 31 01:01:18 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74B1916A419; Thu, 31 Jan 2008 01:01:18 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id 5202E13C44B; Thu, 31 Jan 2008 01:01:18 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost.mckusick.com [127.0.0.1]) by chez.mckusick.com (8.13.8/8.13.6) with ESMTP id m0V10vjX009384; Wed, 30 Jan 2008 17:01:05 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <200801310101.m0V10vjX009384@chez.mckusick.com> To: Barry Pederson Date: Wed, 30 Jan 2008 17:00:57 -0800 From: Kirk McKusick Cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2008 01:01:18 -0000 > Date: Wed, 30 Jan 2008 16:13:54 -0600 > From: Barry Pederson > To: Kirk McKusick > CC: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , questions@freebsd.org, > fs@freebsd.org > Subject: Re: Automatic `nodump' flag? > X-ASK-Info: Message Queued (2008/01/30 14:14:17) > X-ASK-Info: Confirmed by User (2008/01/30 16:37:01) > > Kirk McKusick wrote: > > > The dump program runs on the raw disk partition dumping sequentially > > by inode number. So, it has no idea of the file-tree hierarchy. > > I was just looking at the source to dump, specifically traverse.c and > from what I can see, doesn't dump pass #2 scan through all directories > and then in the searchdir() function remove a directory's children from > the list of inodes to backup if the directory has the nodump flag? > > --------- > 414 if (nodump) { > 415 ip = getino(dp->d_ino, &mode); > 416 if (TSTINO(dp->d_ino, dumpinomap)) { > 417 CLRINO(dp->d_ino, dumpinomap); > 418 *tapesize -= blockest(ip); > --------- > > Barry You are completely correct. This does prune out everything below a directory marked `nodump' even if those files are not also marked `nodump'. Note that by default, level 0 dumps will ignore the `nodump' flag. You have to use `-h 0' if you want a level 0 dump to honor the `nodump' flag. You would think I would remember code that I wrote (though in my defense it was written over 20 years ago :-) Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Thu Jan 31 10:35:47 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDDD616A417; Thu, 31 Jan 2008 10:35:47 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 8252C13C448; Thu, 31 Jan 2008 10:35:47 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 8793B2082; Thu, 31 Jan 2008 11:35:39 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.2/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 767D6207F; Thu, 31 Jan 2008 11:35:39 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id 4D1BF8449D; Thu, 31 Jan 2008 11:35:39 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Kirk McKusick References: <200801310101.m0V10vjX009384@chez.mckusick.com> Date: Thu, 31 Jan 2008 11:35:39 +0100 In-Reply-To: <200801310101.m0V10vjX009384@chez.mckusick.com> (Kirk McKusick's message of "Wed\, 30 Jan 2008 17\:00\:57 -0800") Message-ID: <86wspqz3ic.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2008 10:35:47 -0000 Kirk McKusick writes: > You would think I would remember code that I wrote (though in my > defense it was written over 20 years ago :-) hey, I can't even remember code that I wrote 20 days ago, much less 20 years, so I won't be the one to cast the first stone :) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu Jan 31 10:57:56 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93CE016A41B; Thu, 31 Jan 2008 10:57:56 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [IPv6:2001:4070:101:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 6B01213C4D3; Thu, 31 Jan 2008 10:57:50 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (localhost [IPv6:::1]) by wojtek.tensor.gdynia.pl (8.13.8/8.13.8) with ESMTP id m0VArwZl002167; Thu, 31 Jan 2008 11:53:58 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.13.8/8.13.8/Submit) with ESMTP id m0VArpj4002164; Thu, 31 Jan 2008 11:53:53 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 31 Jan 2008 11:53:51 +0100 (CET) From: Wojciech Puchar To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= In-Reply-To: <86wspqz3ic.fsf@ds4.des.no> Message-ID: <20080131115218.X2154@wojtek.tensor.gdynia.pl> References: <200801310101.m0V10vjX009384@chez.mckusick.com> <86wspqz3ic.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kirk McKusick , questions@freebsd.org, fs@freebsd.org Subject: Re: Automatic `nodump' flag? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2008 10:57:56 -0000 > Kirk McKusick writes: >> You would think I would remember code that I wrote (though in my >> defense it was written over 20 years ago :-) > > hey, I can't even remember code that I wrote 20 days ago, much less 20 > years, so I won't be the one to cast the first stone :) > it's not that bad as you say but after a year it's very difficult to me, not mentioning 12-year old program i wrote for DOS and now it's user requested me to make changes ;) quite difficult From owner-freebsd-fs@FreeBSD.ORG Fri Feb 1 10:23:47 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8547216A419; Fri, 1 Feb 2008 10:23:47 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id E300C13C4CE; Fri, 1 Feb 2008 10:23:46 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 5B63F43E4C0; Fri, 1 Feb 2008 12:00:18 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hFjIPVg1AKeF; Fri, 1 Feb 2008 12:00:18 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id BAAB043E448; Fri, 1 Feb 2008 12:00:17 +0200 (EET) Message-ID: <47A2EDB0.8000801@icyb.net.ua> Date: Fri, 01 Feb 2008 12:00:16 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Remko Lodder , scottl@FreeBSD.org, freebsd-fs@freebsd.org References: <200612221824.kBMIOhfM049471@freefall.freebsd.org> In-Reply-To: <200612221824.kBMIOhfM049471@freefall.freebsd.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Bruce Evans Subject: Re: kern/84983: [udf] [patch] udf filesystem: stat-ting files could randomly fail X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 10:23:47 -0000 on 22/12/2006 20:24 Pav Lucistnik said the following: > Synopsis: [udf] [patch] udf filesystem: stat-ting files could randomly fail > > State-Changed-From-To: open->closed > State-Changed-By: pav > State-Changed-When: Fri Dec 22 18:24:14 UTC 2006 > State-Changed-Why: > Fixed in 6.1 and up > > http://www.freebsd.org/cgi/query-pr.cgi?pr=84983 I found a bug in the patch. I got a panic in real situation when testing a UDF fs with a directory with a huge number of files (~10^4), but this can be easily shown in the code too: static int udf_readatoffset(struct udf_node *node, int *size, off_t offset, struct buf **bp, uint8_t **data) { ... *size = min(*size, MAXBSIZE); if ((error = udf_readlblks(udfmp, sector, *size + (offset & udfmp->bmask), bp))) { If it so happens that *size gets MAXBSIZ value and (offset & udfmp->bmask) is not zero, then a value > MAXBSIZ would be passed to udf_readlblks->bread->breadn->getblk and the latter will panic because it has an explicit assert for size <= MAXBSIZ. I think there should be something in the code like the following: *size = min(*size, MAXBSIZE - (offset & udfmp->bmask)); ---- a different, under-debugged problem ----- BTW, on some smaller directories (but still large ones) I get some very strange problems with reading a directory too. It seems like some bad interaction between udf and buffer cache system. I added a lot of debugging prints and the problems looks like the following: read starting at physical sector (2048-byte one) N, size is ~20K, N%4=0 bread(4 * N, some_big_size) correct data is read repeat the above couple dozen times read starting at physical sector (2048-byte one) N+1, size is ~20K bread(4 * (N+1), some_big_size) data is read from physical sector N+4 (instead of N+1) I remember that Bruce Evance warned me that something like this could happen but I couldn't understand him, because I don't understand VM/buffer subsystem. I'll try to dig up the email. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri Feb 1 10:23:47 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DBF316A46D; Fri, 1 Feb 2008 10:23:47 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id E843C13C4D3; Fri, 1 Feb 2008 10:23:46 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 852E743E542; Fri, 1 Feb 2008 12:05:06 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2lnDZozSuo4P; Fri, 1 Feb 2008 12:05:06 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id CF65A43C749; Fri, 1 Feb 2008 12:05:05 +0200 (EET) Message-ID: <47A2EED0.3010209@icyb.net.ua> Date: Fri, 01 Feb 2008 12:05:04 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Remko Lodder , scottl@FreeBSD.org, freebsd-fs@freebsd.org References: <200612221824.kBMIOhfM049471@freefall.freebsd.org> <47A2EDB0.8000801@icyb.net.ua> In-Reply-To: <47A2EDB0.8000801@icyb.net.ua> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Bruce Evans Subject: Re: kern/84983: [udf] [patch] udf filesystem: stat-ting files could randomly fail X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 10:23:47 -0000 Sorry - I sent this a reply to a wrong email. Correct PR is kern/77234. http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 on 01/02/2008 12:00 Andriy Gapon said the following: > on 22/12/2006 20:24 Pav Lucistnik said the following: >> Synopsis: [udf] [patch] udf filesystem: stat-ting files could randomly fail >> >> State-Changed-From-To: open->closed >> State-Changed-By: pav >> State-Changed-When: Fri Dec 22 18:24:14 UTC 2006 >> State-Changed-Why: >> Fixed in 6.1 and up >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=84983 > > I found a bug in the patch. I got a panic in real situation when testing > a UDF fs with a directory with a huge number of files (~10^4), but this > can be easily shown in the code too: > > static int > udf_readatoffset(struct udf_node *node, int *size, off_t offset, > struct buf **bp, uint8_t **data) > { > ... > *size = min(*size, MAXBSIZE); > > if ((error = udf_readlblks(udfmp, sector, *size + (offset & > udfmp->bmask), bp))) { > > If it so happens that *size gets MAXBSIZ value and (offset & > udfmp->bmask) is not zero, then a value > MAXBSIZ would be passed to > udf_readlblks->bread->breadn->getblk and the latter will panic because > it has an explicit assert for size <= MAXBSIZ. > > I think there should be something in the code like the following: > *size = min(*size, MAXBSIZE - (offset & udfmp->bmask)); > > ---- a different, under-debugged problem ----- > BTW, on some smaller directories (but still large ones) I get some very > strange problems with reading a directory too. It seems like some bad > interaction between udf and buffer cache system. I added a lot of > debugging prints and the problems looks like the following: > > read starting at physical sector (2048-byte one) N, size is ~20K, N%4=0 > bread(4 * N, some_big_size) > correct data is read > repeat the above couple dozen times > read starting at physical sector (2048-byte one) N+1, size is ~20K > bread(4 * (N+1), some_big_size) > data is read from physical sector N+4 (instead of N+1) > > I remember that Bruce Evance warned me that something like this could > happen but I couldn't understand him, because I don't understand > VM/buffer subsystem. I'll try to dig up the email. > > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri Feb 1 10:27:19 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 521A916A419; Fri, 1 Feb 2008 10:27:19 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 049C413C461; Fri, 1 Feb 2008 10:27:18 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 252E443E519; Fri, 1 Feb 2008 12:27:18 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b5m+0X1hyfDH; Fri, 1 Feb 2008 12:27:18 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 76D8A43E50E; Fri, 1 Feb 2008 12:27:17 +0200 (EET) Message-ID: <47A2F404.7010208@icyb.net.ua> Date: Fri, 01 Feb 2008 12:27:16 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Remko Lodder , scottl@FreeBSD.org, freebsd-fs@freebsd.org References: <200612221824.kBMIOhfM049471@freefall.freebsd.org> <47A2EDB0.8000801@icyb.net.ua> In-Reply-To: <47A2EDB0.8000801@icyb.net.ua> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Bruce Evans Subject: Re: kern/84983: [udf] [patch] udf filesystem: stat-ting files could randomly fail X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 10:27:19 -0000 on 01/02/2008 12:00 Andriy Gapon said the following: > ---- a different, under-debugged problem ----- > BTW, on some smaller directories (but still large ones) I get some very > strange problems with reading a directory too. It seems like some bad > interaction between udf and buffer cache system. I added a lot of > debugging prints and the problems looks like the following: > > read starting at physical sector (2048-byte one) N, size is ~20K, N%4=0 > bread(4 * N, some_big_size) > correct data is read > repeat the above couple dozen times > read starting at physical sector (2048-byte one) N+1, size is ~20K > bread(4 * (N+1), some_big_size) > data is read from physical sector N+4 (instead of N+1) > > I remember that Bruce Evance warned me that something like this could > happen but I couldn't understand him, because I don't understand > VM/buffer subsystem. I'll try to dig up the email. > Sorry for the flood - additional info: if I limit max read size in udf_readatoffset() to 2048 (instead of MAXBSIZE), then large directories can be read OK. Seems like something with overlapping buffers, maybe? BTW, here's how I created test environment for the described issues (in tcsh): mkdir /tmp/bigdir cd /tmp/bigdir set i=1 while ($i < NNNNN) touch file.$i set i=`expr $i + 1` end cd /tmp mkisofs -udf -o test.iso bigdir mdconfig -a -t vnode -f test.iso -S 2048 -u 0 mount_udf /dev/md0 /mnt ls -l /mnt -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri Feb 1 14:31:31 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FE1116A417 for ; Fri, 1 Feb 2008 14:31:31 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 5CC8D13C447 for ; Fri, 1 Feb 2008 14:31:31 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.storspeed.com (209-163-168-124.static.tenantsolutions.net [209.163.168.124] (may be forged)) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id m11EVL7X025088; Fri, 1 Feb 2008 08:31:22 -0600 (CST) (envelope-from anderson@freebsd.org) Message-ID: <47A32D39.5090105@freebsd.org> Date: Fri, 01 Feb 2008 08:31:21 -0600 From: Eric Anderson User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Matt Emmerton References: <000801c85b94$f3a58ea0$1200a8c0@hermes> <1200856930.9818.2.camel@jill.exit.com> <002201c85bb6$3d2a7fb0$1200a8c0@hermes> In-Reply-To: <002201c85bb6$3d2a7fb0$1200a8c0@hermes> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: freebsd-fs@freebsd.org Subject: Re: Looking for help to reconstruct a corrupted UFS2 filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 14:31:31 -0000 Matt Emmerton wrote: >> On Sun, 2008-01-20 at 13:47 -0500, Matt Emmerton wrote: >>> What are my options at this point? Since all the superblocks are >>> identical, >>> fsck always behaves the same. I suspect that one of the key blocks >>> that the >>> superblock points to is corrupted. Is any of this data replicated on >>> disk? >>> Can I troll the disk looking for intermediate blocks and easily chain >>> together portions of directory trees? >> >> This kind of thing is why I put ports/sysutils/ffs2recov together. You >> won't be able to recover everything but you should be able to get a lot >> of it back. > > Thanks Frank. I'm playing around with this tool now. Something must be > really hosed since I'm getting a lot of segfaults. > > For example, ffs2recov -s /dev/ad1s1 segfaults after finding 3 > superblocks, and these superblocks aren't close to anything that newfs > -N dumps out (except the one at offset 160). It also attempts to read > blk 18445195961337644512, which is clearly wrong. (I'm 99% sure that I > used the newfs defaults when I created this filesystem, so why would > ffs2recov be looking for superblocks in different locations?) > > ffs2rrecov -p also segfaults after dumping part of cg 3, and ffs2recov > -d segfaults after hitting inode 8331. > > ffs2recov -a and ffs2recov -r do a lot of complaining regarding > failure to allocate large amount (or negative) memory. > > I'm guessing that it's starting off with bad data, and that's not > helping. I'm doing some brute force work with ffs2recov -i to sniff out > good inodes and will start from there. You probably have a corrupt cylinder group, which is a common problem when something like this corrupts the cg blocks badly. You might be able to go in with fsdb and find the badness, or (eek!) a hex editor to the right spot on disk and change the values. I would highly recommend dd'ing the disk to a file on another disk prior to doing either of those things, and work on the image file. Eric From owner-freebsd-fs@FreeBSD.ORG Fri Feb 1 17:41:55 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1769F16A41A for ; Fri, 1 Feb 2008 17:41:55 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.freebsd.org (Postfix) with ESMTP id 9096D13C461 for ; Fri, 1 Feb 2008 17:41:54 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by koef.zs64.net (8.14.2/8.14.2) with ESMTP id m11HMERq057338 for ; Fri, 1 Feb 2008 18:22:14 +0100 (CET) (envelope-from cracauer@koef.zs64.net) Received: (from cracauer@localhost) by koef.zs64.net (8.14.2/8.14.2/Submit) id m11HMEtU057337 for freebsd-fs@freebsd.org; Fri, 1 Feb 2008 12:22:14 -0500 (EST) (envelope-from cracauer) Date: Fri, 1 Feb 2008 12:22:14 -0500 From: Martin Cracauer To: freebsd-fs@freebsd.org Message-ID: <20080201172214.GA55957@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: fsck and mount disagree on whether superblocks are usable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 17:41:55 -0000 This is not an emergency but I find it odd. Mount and fsck agree on whether superblocks are usable. Mount can mount readonly, but fsck can use neither the primary superblock nor the alternatives. Here the long story: Just for kicks I replicated a harddrive, while dd'ing off /dev/ad0 with live (read/write mounts) filesystems and all partition tables and disklabels. Restored to a different drive I have everything, all partitions are there and I can mount readonly. The filesystem is of course dirty, hence the readonly mount. However, fsck refuses to run. BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE [who did that all-uppercase, anyway?] Allowing it to look for alternate superblocks I get: BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE LOOK FOR ALTERNATE SUPERBLOCKS? [yn] y 32 is not a file system superblock 28756320 is not a file system superblock 57512608 is not a file system superblock 86268896 is not a file system superblock 115025184 is not a file system superblock 143781472 is not a file system superblock 172537760 is not a file system superblock 201294048 is not a file system superblock 230050336 is not a file system superblock 258806624 is not a file system superblock SEARCH FOR ALTERNATE SUPER-BLOCK FAILED. YOU MUST USE THE -b OPTION TO FSCK TO SPECIFY THE LOCATION OF AN ALTERNATE SUPER-BLOCK TO SUPPLY NEEDED INFORMATION; SEE fsck(8). Why would the superblocks be entirely unrecognizable by fsck when mount can mount the filesystem just fine (readonly)? Here are the fields for the primary superblock as printed on the original machine, printed from a modified fsck: .fs_sblkno 40 .fs_cblkno 48 .fs_iblkno 56 .fs_dblkno 3000 .fs_ncg 831 .fs_bsize 16384 .fs_fsize 2048 .fs_frag 8 .fs_bmask -16384 .fs_fmask -2048 .fs_bshift 14 .fs_fshift 11 .fs_fragshift 3 .fs_fsbtodb 2 .fs_sbsize 2048 .fs_nindir 2048 .fs_inopb 64 .fs_cssize 14336 .fs_ipg 23552 .fs_fpg 94064 .fs_magic 424935705 Here is the same print for the fsck that refuses to run on the other machine: .fs_sblkno 40 .fs_cblkno 48 .fs_iblkno 56 .fs_dblkno 3000 .fs_ncg 753 .fs_bsize 16384 .fs_fsize 2048 .fs_frag 8 .fs_bmask -16384 .fs_fmask -2048 .fs_bshift 14 .fs_fshift 11 .fs_fragshift 3 .fs_fsbtodb 2 .fs_sbsize 2048 .fs_nindir 2048 .fs_inopb 64 .fs_cssize 12288 .fs_ipg 23552 .fs_fpg 94064 .fs_magic 424935705 Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/ From owner-freebsd-fs@FreeBSD.ORG Sat Feb 2 10:07:04 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7F4F16A418 for ; Sat, 2 Feb 2008 10:07:04 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.187]) by mx1.freebsd.org (Postfix) with ESMTP id 147B713C45A for ; Sat, 2 Feb 2008 10:07:04 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by rv-out-0910.google.com with SMTP id g13so1126645rvb.43 for ; Sat, 02 Feb 2008 02:07:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=wNQCqbE3t4mhYu/e823FuIzQ6NNveTt8b5cogVNzTYA=; b=KI20/GVZCbEmtspc7Oito7U9exyAin858eORSPUkaZgh5nMd8N7vNTDgvoZmcWHdMz3QV3xVZBnCmAHfZ7KighzkpN8oWx2tTItjlcE6e6vJCvA0NFKDCyKHKACK/sHL+TOV1xoyhGnEVNZlNboclSKbcb5getmzAd8UwXM99ng= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=FzK130uHTM49ud71jLJAtd1SJMr/bCSPSzdatyEUK/KLoeu35F96xhfl3+xLUFGFPbaj11tzHC2U62iz5So8tYRJ+HusUowsc1uhOdaxbmuglgOjIar83Dptac6P6nTP1QiOuM31aNveXIcxi+AK3xDRz9NFE3GYHkaR2RlFuTs= Received: by 10.141.27.16 with SMTP id e16mr2025527rvj.97.1201946386071; Sat, 02 Feb 2008 01:59:46 -0800 (PST) Received: by 10.141.170.18 with HTTP; Sat, 2 Feb 2008 01:59:46 -0800 (PST) Message-ID: <2e77fc10802020159s7a8de40dlc938a0535def20d6@mail.gmail.com> Date: Sat, 2 Feb 2008 11:59:46 +0200 From: "Niki Denev" Sender: ndenev@gmail.com To: freebsd-fs@freebsd.org In-Reply-To: <2e77fc10802020152k2f5385c5w5938d91b1183f8e0@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <2e77fc10802020152k2f5385c5w5938d91b1183f8e0@mail.gmail.com> X-Google-Sender-Auth: c74940b030ba7932 Subject: Re: ZFS panics X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Feb 2008 10:07:04 -0000 On Feb 2, 2008 11:52 AM, Niki Denev wrote: [snip] > I've tried to use the "list" command in kdb as shown in the developers handbook > but it keeps saying "No source file for address XXX" [snip] sorry, i forgot to remove those lines after i managed to get "add-debug-symbols" load zfs.ko.symbols From owner-freebsd-fs@FreeBSD.ORG Sat Feb 2 10:19:09 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD80C16A41A for ; Sat, 2 Feb 2008 10:19:09 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.190]) by mx1.freebsd.org (Postfix) with ESMTP id 7F15313C457 for ; Sat, 2 Feb 2008 10:19:09 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by rv-out-0910.google.com with SMTP id g13so1128506rvb.43 for ; Sat, 02 Feb 2008 02:19:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; bh=7veNi0ZujplnapSKp9psDCCXlFjZgV4idB9i13N1Ckk=; b=aMJYY3+xMOhmqLT873OpAQDOCMCmbcyOZFsU7T86R+XOF6RgAPNjzNX8GnWCJHlg+OpwjysOFBdmofUkxsdqR+yExq/8/f+Mf3UOSk2BHSbDG29GDFmci8jyLx0e8DfMytSpGBQZY0y0nc83CgkYcaFImXxREbjLNqtO5YhL7eA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; b=M+GeD9hVUTRjxxqm+XU+oLWLRQeBdJ1qXSnfzquQVmN2u9wXZFxmvy2/FCg60j9cpuhp4siv7ToYmmoQRCXMd3yIHib1/4ZSAELisVditvvctBLu6cDR89/RnSOajS9mNVeL6hju6wt8aeKShU+vUPV3OADrlEOd3aZYp7hA7lk= Received: by 10.140.187.10 with SMTP id k10mr3144359rvf.95.1201945929650; Sat, 02 Feb 2008 01:52:09 -0800 (PST) Received: by 10.141.170.18 with HTTP; Sat, 2 Feb 2008 01:52:09 -0800 (PST) Message-ID: <2e77fc10802020152k2f5385c5w5938d91b1183f8e0@mail.gmail.com> Date: Sat, 2 Feb 2008 11:52:09 +0200 From: "Niki Denev" Sender: ndenev@gmail.com To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Google-Sender-Auth: bf8b2badd3e91c8e Subject: ZFS panics X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Feb 2008 10:19:09 -0000 Hi, I'm doing some stress testing on one server using ZFS and i have experienced two kernel panics in the last days. The machine runs AMD64 7.0-PRERELEASE on dual quad-core (8 cores total) Intel Xeon 2.0Ghz, with 8Gigs of Ram. The disk subsystem consists of eight hitachi SATA drives on a Areca 1231ML with 1G of cache memory and a battery backup. I'm using GUID partitions only. One 10G for the system on UFS2 with geom_journal, 10G swap/dump partition, and the rest 2.7TB is a ZFS pool. I also have this in loader.conf : vm.kmem_size="1G" vm.kmem_size_max="1G" I was running multiple bonnie++ instances in parallel writing and reading from the ZFS pool. The first time i ran 80 bonnie++ instances and the machine rebooted after about 3 hours. The second time i ran 16 bonnie++ instances and the machine survived good 11 hours. I've tried to use the "list" command in kdb as shown in the developers handbook but it keeps saying "No source file for address XXX" Here it is the first panic that i experienced. The second one looks identical : (i'm not entirely sure that i load the zfs symbols properly?) sm-srv221# kldstat |grep zfs 2 1 0xffffffff80bfc000 f5a40 zfs.ko sm-srv221# kgdb -q /boot/kernel/kernel.symbols /var/crash/vmcore.0 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 04 fault virtual address = 0x18 fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff80c19d16 stack pointer = 0x10:0xffffffffd996a8f0 frame pointer = 0x10:0xffffffffd996a920 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 321 (txg_thread_enter) trap number = 12 panic: page fault cpuid = 4 Uptime: 3h20m21s Physical memory: 8177 MB Dumping 522 MB: 507 491 475 459 443 427 411 395 379 363 347 331 315 299 283 267 251 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11 #0 doadump () at pcpu.h:194 194 pcpu.h: No such file or directory. in pcpu.h (kgdb) add-debug-symbols /boot/kernel/zfs.ko.symbols 0xffffffff80bfc000 (kgdb) list *0xffffffff80c19d16 0xffffffff80c19d16 is in dmu_objset_sync_dnodes (/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:707). 702 ASSERT(dn->dn_dbuf->db_data_pending); 703 /* 704 * Initialize dn_zio outside dnode_sync() 705 * to accomodate meta-dnode 706 */ 707 dn->dn_zio = dn->dn_dbuf->db_data_pending->dr_zio; 708 ASSERT(dn->dn_zio); 709 710 ASSERT3U(dn->dn_nlevels, <=, DN_MAX_LEVELS); 711 list_remove(list, dn); (kgdb) bt #0 doadump () at pcpu.h:194 #1 0x0000000000000004 in avl_balance2child () #2 0xffffffff80478619 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #3 0xffffffff80478a1d in panic (fmt=0x104
) at /usr/src/sys/kern/kern_shutdown.c:563 #4 0xffffffff8074f174 in trap_fatal (frame=0xffffff0003377000, eva=18446742974251873384) at /usr/src/sys/amd64/amd64/trap.c:724 #5 0xffffffff8074f545 in trap_pfault (frame=0xffffffffd996a840, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:641 #6 0xffffffff8074fe88 in trap (frame=0xffffffffd996a840) at /usr/src/sys/amd64/amd64/trap.c:410 #7 0xffffffff80735aee in calltrap () at /usr/src/sys/amd64/amd64/exception.S:169 #8 0xffffffff80c19d16 in dmu_objset_sync_dnodes (list=0xffffff0003730d20, tx=0xffffff0137f9e800) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:707 #9 0xffffffff80c19e7d in dmu_objset_sync (os=0xffffff0003730c00, pio=0xffffff0131a4fac0, tx=0xffffff0137f9e800) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:809 #10 0xffffffff80c27372 in dsl_pool_sync (dp=0xffffff00032b2800, txg=15331) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:188 #11 0xffffffff80c31da0 in spa_sync (spa=0xffffff00032be000, txg=15331) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/spa.c:2989 #12 0xffffffff80c37abf in txg_sync_thread (arg=Variable "arg" is not available. ) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/txg.c:331 #13 0xffffffff80459d33 in fork_exit (callout=0xffffffff80c37990 , arg=0xffffff00032b2800, frame=0xffffffffd996ac80) at /usr/src/sys/kern/kern_fork.c:781 #14 0xffffffff80735ebe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:415 #15 0x0000000000000000 in ?? () #16 0x0000000000000000 in ?? () #17 0x0000000000000001 in avl_balance2child () #18 0x0000000000000000 in ?? () #19 0x0000000000000000 in ?? () #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () #22 0x0000000000000000 in ?? () #23 0x0000000000000000 in ?? () #24 0x0000000000000000 in ?? () #25 0x0000000000000000 in ?? () #26 0x0000000000000000 in ?? () #27 0x0000000000000000 in ?? () #28 0x0000000000000000 in ?? () #29 0x0000000000000000 in ?? () #30 0x0000000000000000 in ?? () #31 0x0000000000000000 in ?? () #32 0x0000000000000000 in ?? () #33 0x0000000000000000 in ?? () #34 0x0000000000000000 in ?? () #35 0x0000000000000000 in ?? () #36 0x0000000000000000 in ?? () #37 0x0000000000000000 in ?? () #38 0x0000000000000000 in ?? () #39 0x0000000000e06000 in ?? () #40 0xffffffff80a7a740 in tdq_cpu () #41 0xffffffff80a83f40 in tdq_groups () #42 0xffffffff80a83d40 in tdq_cpu () #43 0xffffff0003377000 in ?? () #44 0xffffffff80a77540 in tdg_maxid () #45 0xffffffffd996a4b8 in ?? () #46 0xffffff0003377000 in ?? () #47 0xffffffff80496bc8 in sched_switch (td=0xffffffff80c37990, newtd=0x0, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/sched_ule.c:1898 #48 0x0000000000000000 in ?? () #49 0x0000000000000000 in ?? () #50 0x0000000000000000 in ?? () #51 0x0000000000000000 in ?? () #52 0x0000000000000000 in ?? () #53 0x0000000000000000 in ?? () #54 0x0000000000000000 in ?? () From owner-freebsd-fs@FreeBSD.ORG Sat Feb 2 19:31:00 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C61616A419 for ; Sat, 2 Feb 2008 19:31:00 +0000 (UTC) (envelope-from jhs@berklix.org) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 82BC913C45B for ; Sat, 2 Feb 2008 19:30:59 +0000 (UTC) (envelope-from jhs@berklix.org) Received: from js.berklix.net (p549A456B.dip.t-dialin.net [84.154.69.107]) (authenticated bits=0) by tower.berklix.org (8.13.6/8.13.6) with ESMTP id m12JEnIB060054; Sat, 2 Feb 2008 19:14:50 GMT (envelope-from jhs@berklix.org) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by js.berklix.net (8.13.8/8.13.8) with ESMTP id m12JGf28003191; Sat, 2 Feb 2008 20:16:41 +0100 (CET) (envelope-from jhs@berklix.org) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.13.8/8.13.8) with ESMTP id m12JGUjN049706; Sat, 2 Feb 2008 20:16:36 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <200802021916.m12JGUjN049706@fire.js.berklix.net> To: Martin Cracauer In-reply-to: <20080201172214.GA55957@cons.org> References: <20080201172214.GA55957@cons.org> Comments: In-reply-to Martin Cracauer message dated "Fri, 01 Feb 2008 12:22:14 -0500." Date: Sat, 02 Feb 2008 20:16:30 +0100 From: "Julian H. Stacey" Cc: freebsd-fs@freebsd.org Subject: Re: fsck and mount disagree on whether superblocks are usable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Feb 2008 19:31:00 -0000 Martin Cracauer wrote: > This is not an emergency but I find it odd. Mount and fsck agree on > whether superblocks are usable. Mount can mount readonly, but fsck > can use neither the primary superblock nor the alternatives. > > 32 is not a file system superblock Just in case, You know secondary block on newer FSs moved from 32 ? Ref man fsck_ufs -b Use the block specified immediately after the flag as the super block for the file system. An alternate super block is usually located at block 32 for UFS1, and block 160 for UFS2. -- Julian Stacey. BSD Unix Linux Net Consultant, Munich. http://berklix.com