From owner-freebsd-fs@FreeBSD.ORG Sun Aug 2 08:50:05 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04057106566C for ; Sun, 2 Aug 2009 08:50:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E57168FC0A for ; Sun, 2 Aug 2009 08:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n728o49S095431 for ; Sun, 2 Aug 2009 08:50:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n728o4dC095430; Sun, 2 Aug 2009 08:50:04 GMT (envelope-from gnats) Date: Sun, 2 Aug 2009 08:50:04 GMT Message-Id: <200908020850.n728o4dC095430@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Aldis Berjoza Cc: Subject: Re: kern/137037: [zfs] [hang] zfs rollback on root causes FreeBSD to freeze in few seconds X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Aldis Berjoza List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Aug 2009 08:50:05 -0000 The following reply was made to PR kern/137037; it has been noted by GNATS. From: Aldis Berjoza To: bug-followup@FreeBSD.org, killasmurf86@gmail.com Cc: Subject: Re: kern/137037: [zfs] [hang] zfs rollback on root causes FreeBSD to freeze in few seconds Date: Sun, 02 Aug 2009 11:44:53 +0300 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I don't know if this helps, but: Today I did zfs rollback on root again... But this time I logged out from X, and that logged in.... For the first time, after root rollback my system didn't hang.... uname -a: FreeBSD 192.168.128.100 8.0-BETA2 FreeBSD 8.0-BETA2 #0: Mon Jul 20 21:43:13 EEST 2009 root@192.168.128.100:/usr/obj/usr/src/sys/ANTIGENERIC i386 - -- Aldis Berjoza My public PGP key: http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xA81349A77ED573D3 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkp1UgUACgkQqBNJp37Vc9NjbwCffjydqiAgAiUSICQtLttHe/F/ D5gAoJ3XEZCmwZH4BAQZcCjf8YTqoutd =9OET -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Sun Aug 2 09:27:17 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAA6A106566B; Sun, 2 Aug 2009 09:27:17 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by mx1.freebsd.org (Postfix) with ESMTP id E3EBF8FC0C; Sun, 2 Aug 2009 09:27:16 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by jpru.ffm.jpru.de (8.13.8/8.13.8) with ESMTP id n729RF5k085356; Sun, 2 Aug 2009 11:27:15 +0200 (CEST) (envelope-from lists@jpru.de) Received: (from unger@localhost) by jpru.ffm.jpru.de (8.13.8/8.13.8/Submit) id n729REd7085355; Sun, 2 Aug 2009 11:27:14 +0200 (CEST) (envelope-from lists@jpru.de) X-Authentication-Warning: jpru.ffm.jpru.de: unger set sender to lists@jpru.de using -f Date: Sun, 2 Aug 2009 11:27:14 +0200 From: Juergen Unger To: Pawel Jakub Dawidek Message-ID: <20090802092714.GA5813@jpru.ffm.jpru.de> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090729084723.GD1586@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Aug 2009 09:27:18 -0000 Hi Pawel, On Wed, Jul 29, 2009 at 10:47:23AM +0200, Pawel Jakub Dawidek wrote: > On Tue, Jul 28, 2009 at 12:50:26PM +0300, Andriy Gapon wrote: > > on 27/07/2009 22:58 O. Hartmann said the following: > > > Juergen Unger wrote: > > [snip] > > >>> _sx_xlock(3c,0,874aa28d,70f,8ae9a9f8,...) at _sx_xlock+0x43 > > >>> dmu_buf_update_user(0,8ae9a9f8,0,0,0,...) at dmu_buf_update_user+0x35 > > >>> zfs_znode_dmu_fini(8ae9a9f8,874b312d,1114,110b,879ab000,...) at zfs_znode_dmu_f3 > > >>> zfs_freebsd_reclaim(fcd29c3c,1,0,8ec63754,fcd29c60,...) at zfs_freebsd_reclaim+0 > > >>> VOP_RECLAIM_APV(874b65a0,fcd29c3c,0,0,8ec637c8,...) at VOP_RECLAIM_APV+0xa5 > > >>> vgonel(8ec637c8,0,80c77037,386,0,...) at vgonel+0x1a4 > > >>> vnlru_free(80f2a0f0,0,80c77037,300,3e8,...) at vnlru_free+0x2d5 > > >>> vnlru_proc(0,fcd29d38,80c652bc,33e,871932a8,...) at vnlru_proc+0x80 > > >>> fork_exit(8090d960,0,fcd29d38) at fork_exit+0xb8 > > >>> fork_trampoline() at fork_trampoline+0x8 >[snip] > > P.S. I see that zfs_inactive checks for z_dbuf being NULL and there is the > > following comment: > > /* > > * The fs has been unmounted, or we did a > > * suspend/resume and this file no longer exists. > > */ > > Maybe zfs_freebsd_reclaim should do the same? > > Yes, you might be right. > > Could you guys, who can reproduce it, try this patch: > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch I tried the patch, restarted the whole thing yesterday morning and after less then 24 hours and approximately 3215 zfs-receive jobs it do not crashes anymore, but the last started zfs-receive jobs is hanging, cannot be killed, even not with -9. Even other zfs commands are hanging and cannot be killed, while zpool commands seems to be not affected. root 86397 0.0 0.0 3920 1308 ?? D 3:18AM 0:00.29 zfs receive -Fv zzzz/203 root 5001 0.0 0.0 3920 1208 0 D+ 10:45AM 0:00.00 zfs list -t snapshot root 5477 0.0 0.0 3920 1240 3 D+ 11:08AM 0:00.00 zfs list also the sync command I tried to execute hangs forever: root 5457 0.0 0.0 1528 492 2- D+ 11:05AM 0:00.04 sync Other parts of the system which do not have something todo with zfs are still working well. I will leave the machine running in this state, is there something I can do to retrieve other usefull information for you? thnx in advance, Juergen -- ENOSIG From owner-freebsd-fs@FreeBSD.ORG Sun Aug 2 09:30:00 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C90E41065670; Sun, 2 Aug 2009 09:30:00 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 29E978FC17; Sun, 2 Aug 2009 09:29:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id BFEE645CAC; Sun, 2 Aug 2009 11:29:57 +0200 (CEST) Received: from localhost (abib242.neoplus.adsl.tpnet.pl [83.7.117.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 25DD145C98; Sun, 2 Aug 2009 11:29:52 +0200 (CEST) Date: Sun, 2 Aug 2009 11:30:16 +0200 From: Pawel Jakub Dawidek To: Juergen Unger Message-ID: <20090802093016.GB3071@garage.freebsd.pl> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3MwIy2ne0vdjdPXF" Content-Disposition: inline In-Reply-To: <20090802092714.GA5813@jpru.ffm.jpru.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL, RCVD_IN_XBL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Aug 2009 09:30:01 -0000 --3MwIy2ne0vdjdPXF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 02, 2009 at 11:27:14AM +0200, Juergen Unger wrote: > Hi Pawel, >=20 > On Wed, Jul 29, 2009 at 10:47:23AM +0200, Pawel Jakub Dawidek wrote: > > On Tue, Jul 28, 2009 at 12:50:26PM +0300, Andriy Gapon wrote: > > > on 27/07/2009 22:58 O. Hartmann said the following: > > > > Juergen Unger wrote: > > > [snip] > > > >>> _sx_xlock(3c,0,874aa28d,70f,8ae9a9f8,...) at _sx_xlock+0x43 > > > >>> dmu_buf_update_user(0,8ae9a9f8,0,0,0,...) at dmu_buf_update_user+= 0x35 > > > >>> zfs_znode_dmu_fini(8ae9a9f8,874b312d,1114,110b,879ab000,...) at z= fs_znode_dmu_f3 > > > >>> zfs_freebsd_reclaim(fcd29c3c,1,0,8ec63754,fcd29c60,...) at zfs_fr= eebsd_reclaim+0 > > > >>> VOP_RECLAIM_APV(874b65a0,fcd29c3c,0,0,8ec637c8,...) at VOP_RECLAI= M_APV+0xa5 > > > >>> vgonel(8ec637c8,0,80c77037,386,0,...) at vgonel+0x1a4 > > > >>> vnlru_free(80f2a0f0,0,80c77037,300,3e8,...) at vnlru_free+0x2d5 > > > >>> vnlru_proc(0,fcd29d38,80c652bc,33e,871932a8,...) at vnlru_proc+0x= 80 > > > >>> fork_exit(8090d960,0,fcd29d38) at fork_exit+0xb8 > > > >>> fork_trampoline() at fork_trampoline+0x8 > >[snip] > > > P.S. I see that zfs_inactive checks for z_dbuf being NULL and there i= s the > > > following comment: > > > /* > > > * The fs has been unmounted, or we did a > > > * suspend/resume and this file no longer exists. > > > */ > > > Maybe zfs_freebsd_reclaim should do the same? > >=20 > > Yes, you might be right. > >=20 > > Could you guys, who can reproduce it, try this patch: > >=20 > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch >=20 > I tried the patch, restarted the whole thing yesterday morning > and after less then 24 hours and approximately 3215 zfs-receive > jobs it do not crashes anymore, but the last started zfs-receive > jobs is hanging, cannot be killed, even not with -9. Even other > zfs commands are hanging and cannot be killed, while zpool commands > seems to be not affected. >=20 > root 86397 0.0 0.0 3920 1308 ?? D 3:18AM 0:00.29 zfs receive= -Fv zzzz/203 > root 5001 0.0 0.0 3920 1208 0 D+ 10:45AM 0:00.00 zfs list -t= snapshot > root 5477 0.0 0.0 3920 1240 3 D+ 11:08AM 0:00.00 zfs list >=20 > also the sync command I tried to execute hangs forever: >=20 > root 5457 0.0 0.0 1528 492 2- D+ 11:05AM 0:00.04 sync >=20 > Other parts of the system which do not have something todo with zfs > are still working well. I will leave the machine running in this > state, is there something I can do to retrieve other usefull information > for you? If you can break into debugger and send me 'show alltrace' for starters. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --3MwIy2ne0vdjdPXF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKdVyoForvXbEpPzQRAi8dAKCEpGhqHjJvGVa3Bw0iyvNiElA7MQCg2HvH CIaaGbnB7hW+rlFIBoNdCgQ= =IyTt -----END PGP SIGNATURE----- --3MwIy2ne0vdjdPXF-- From owner-freebsd-fs@FreeBSD.ORG Mon Aug 3 10:08:56 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B33B7106564A for ; Mon, 3 Aug 2009 10:08:56 +0000 (UTC) (envelope-from stark@mapper.nl) Received: from smtp-out0.tiscali.nl (smtp-out0.tiscali.nl [195.241.79.175]) by mx1.freebsd.org (Postfix) with ESMTP id 7437E8FC17 for ; Mon, 3 Aug 2009 10:08:56 +0000 (UTC) (envelope-from stark@mapper.nl) Received: from [82.170.17.27] (helo=mapper.nl) by smtp-out0.tiscali.nl with esmtp (Exim) (envelope-from ) id 1MXuHZ-0001vz-Cm for freebsd-fs@freebsd.org; Mon, 03 Aug 2009 11:56:42 +0200 Received: from bowser ([192.168.0.1] helo=[0.0.0.0]) by mapper.nl with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1MXuHS-000MJL-PC for freebsd-fs@freebsd.org; Mon, 03 Aug 2009 11:56:34 +0200 Message-ID: <4A76B450.8010206@mapper.nl> Date: Mon, 03 Aug 2009 11:56:32 +0200 From: Mark Stapper User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 0.96.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig29D8B7C15E497498ED9BBC48" Subject: zfs built-in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 10:08:56 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig29D8B7C15E497498ED9BBC48 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello, Would it be possible to built zfs support into the kernel? If so, what would I have to add to my kernel config file? Greetz, Mark --------------enig29D8B7C15E497498ED9BBC48 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkp2tFIACgkQN9xNqOOVnWBBIwCfbf1wQ95K7haXZmFmgKwZRi4b xDEAn2ru9zsDcOYwGJtZ0JkMMgD9E8Jf =7BDP -----END PGP SIGNATURE----- --------------enig29D8B7C15E497498ED9BBC48-- From owner-freebsd-fs@FreeBSD.ORG Mon Aug 3 11:06:56 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FCC7106567B for ; Mon, 3 Aug 2009 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6BFED8FC2D for ; Mon, 3 Aug 2009 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n73B6udt088584 for ; Mon, 3 Aug 2009 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n73B6tXT088580 for freebsd-fs@FreeBSD.org; Mon, 3 Aug 2009 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Aug 2009 11:06:55 GMT Message-Id: <200908031106.n73B6tXT088580@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 11:06:57 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136942 fs [zfs] zvol resize not reflected until reboot o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/136218 fs [zfs] Exported ZFS pools can't be imported into (Open) o kern/135594 fs [zfs] Single dataset unresponsive with Samba o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135480 fs [zfs] panic: lock &arg.lock already initialized o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o bin/135314 fs [zfs] assertion failed for zdb(8) usage o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot f kern/134496 fs [zfs] [panic] ZFS pool export occasionally causes a ke o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133980 fs [panic] [ffs] panic: ffs_valloc: dup alloc o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [smbfs] [panic] panic: ffs_truncate: read-only filesys o kern/133373 fs [zfs] umass attachment causes ZFS checksum errors, dat o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/133134 fs [zfs] Missing ZFS zpool labels o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132597 fs [tmpfs] [panic] tmpfs-related panic while interrupting o kern/132551 fs [zfs] ZFS locks up on extattr_list_link syscall o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes f kern/132068 fs [zfs] page fault when using ZFS over NFS on 7.1-RELEAS o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/131086 fs [ext2fs] [patch] mkfs.ext2 creates rotten partition o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129148 fs [zfs] [panic] panic on concurrent writing & rollback o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/128633 fs [zfs] [lor] lock order reversal in zfs o kern/128514 fs [zfs] [mpt] problems with ZFS and LSILogic SAS/SATA Ad f kern/128173 fs [ext2fs] ls gives "Input/output error" on mounted ext3 o kern/127659 fs [tmpfs] tmpfs memory leak o kern/127492 fs [zfs] System hang on ZFS input-output o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127213 fs [tmpfs] sendfile on tmpfs data corruption o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/125644 fs [zfs] [panic] zfs unfixable fs errors caused panic whe f kern/125536 fs [ext2fs] ext 2 mounts cleanly but fails on commands li o kern/125149 fs [nfs] [panic] changing into .zfs dir from nfs client c f kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122888 fs [zfs] zfs hang w/ prefetch on, zil off while running t o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o kern/122173 fs [zfs] [panic] Kernel Panic if attempting to replace a o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o kern/122047 fs [ext2fs] [patch] incorrect handling of UF_IMMUTABLE / o kern/122038 fs [tmpfs] [panic] tmpfs: panic: tmpfs_alloc_vp: type 0xc o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o kern/121770 fs [zfs] ZFS on i386, large file or heavy I/O leads to ke o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o bin/120288 fs zfs(8): "zfs share -a" does not send SIGHUP to mountd f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o misc/118855 fs [zfs] ZFS-related commands are nonfunctional in fixit o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118320 fs [zfs] [patch] NFS SETATTR sometimes fails to set file o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o kern/113180 fs [zfs] Setting ZFS nfsshare property does not cause inh o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/105093 fs [ext2fs] [patch] ext2fs on read-only media cannot be m o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/89991 fs [ufs] softupdates with mount -ur causes fs UNREFS o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/77826 fs [ext2fs] ext2fs usb filesystem will not mount RW o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 149 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 3 14:26:36 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07375106566B for ; Mon, 3 Aug 2009 14:26:36 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout0.freenet.de (mout0.freenet.de [IPv6:2001:748:100:40::2:2]) by mx1.freebsd.org (Postfix) with ESMTP id 95A1B8FC1A for ; Mon, 3 Aug 2009 14:26:35 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from [195.4.92.15] (helo=5.mx.freenet.de) by mout0.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #92) id 1MXyUk-0007po-Cu; Mon, 03 Aug 2009 16:26:34 +0200 Received: from t97e7.t.pppool.de ([89.55.151.231]:13709 helo=ernst.jennejohn.org) by 5.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #93) id 1MXyUj-0004cc-Qm; Mon, 03 Aug 2009 16:26:34 +0200 Date: Mon, 3 Aug 2009 16:26:32 +0200 From: Gary Jennejohn To: Mark Stapper Message-ID: <20090803162632.3da3eca4@ernst.jennejohn.org> In-Reply-To: <4A76B450.8010206@mapper.nl> References: <4A76B450.8010206@mapper.nl> X-Mailer: Claws Mail 3.7.2 (GTK+ 2.16.2; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-purgate-ID: 149285::1249309594-00002DAB-D4EDEF92/0-0/0-0 Cc: freebsd-fs@freebsd.org Subject: Re: zfs built-in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 14:26:36 -0000 On Mon, 03 Aug 2009 11:56:32 +0200 Mark Stapper wrote: > Hello, > > Would it be possible to built zfs support into the kernel? > If so, what would I have to add to my kernel config file? > Greetz, > Mark > Nope, looks like it's only available as a module. --- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Mon Aug 3 20:32:32 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 425BD106566C; Mon, 3 Aug 2009 20:32:32 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by mx1.freebsd.org (Postfix) with ESMTP id 9B72A8FC19; Mon, 3 Aug 2009 20:32:31 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by jpru.ffm.jpru.de (8.13.8/8.13.8) with ESMTP id n73KWU5f056924; Mon, 3 Aug 2009 22:32:30 +0200 (CEST) (envelope-from lists@jpru.de) Received: (from unger@localhost) by jpru.ffm.jpru.de (8.13.8/8.13.8/Submit) id n73KWRbM056903; Mon, 3 Aug 2009 22:32:27 +0200 (CEST) (envelope-from lists@jpru.de) X-Authentication-Warning: jpru.ffm.jpru.de: unger set sender to lists@jpru.de using -f Date: Mon, 3 Aug 2009 22:32:26 +0200 From: Juergen Unger To: Pawel Jakub Dawidek Message-ID: <20090803203226.GE5813@jpru.ffm.jpru.de> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090802093016.GB3071@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 20:32:32 -0000 Hi, On Sun, Aug 02, 2009 at 11:30:16AM +0200, Pawel Jakub Dawidek wrote: [...] > > > Could you guys, who can reproduce it, try this patch: > > > > > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch > > > > I tried the patch, restarted the whole thing yesterday morning > > and after less then 24 hours and approximately 3215 zfs-receive > > jobs it do not crashes anymore, but the last started zfs-receive > > jobs is hanging, cannot be killed, even not with -9. Even other > > zfs commands are hanging and cannot be killed, while zpool commands > > seems to be not affected. > > > > root 86397 0.0 0.0 3920 1308 ?? D 3:18AM 0:00.29 zfs receive -Fv zzzz/203 > > root 5001 0.0 0.0 3920 1208 0 D+ 10:45AM 0:00.00 zfs list -t snapshot > > root 5477 0.0 0.0 3920 1240 3 D+ 11:08AM 0:00.00 zfs list > > > > also the sync command I tried to execute hangs forever: > > > > root 5457 0.0 0.0 1528 492 2- D+ 11:05AM 0:00.04 sync > > > > Other parts of the system which do not have something todo with zfs > > are still working well. I will leave the machine running in this > > state, is there something I can do to retrieve other usefull information > > for you? > > If you can break into debugger and send me 'show alltrace' for starters. hmm, maybe you did not get my last mail. I put the log of this on -Juergen- -- ENOSIG From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 07:33:59 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0229106566C; Tue, 4 Aug 2009 07:33:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id D21998FC0A; Tue, 4 Aug 2009 07:33:58 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 1DDCA45C89; Tue, 4 Aug 2009 09:33:55 +0200 (CEST) Received: from localhost (pjd-w.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 81147456B1; Tue, 4 Aug 2009 09:33:49 +0200 (CEST) Date: Tue, 4 Aug 2009 09:34:16 +0200 From: Pawel Jakub Dawidek To: Juergen Unger Message-ID: <20090804073416.GA4479@garage.freebsd.pl> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cNdxnHkX5QqsyA0e" Content-Disposition: inline In-Reply-To: <20090803203226.GE5813@jpru.ffm.jpru.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 07:34:00 -0000 --cNdxnHkX5QqsyA0e Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 03, 2009 at 10:32:26PM +0200, Juergen Unger wrote: > Hi, >=20 > On Sun, Aug 02, 2009 at 11:30:16AM +0200, Pawel Jakub Dawidek wrote: > [...] > > > > Could you guys, who can reproduce it, try this patch: > > > >=20 > > > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch > > >=20 > > > I tried the patch, restarted the whole thing yesterday morning > > > and after less then 24 hours and approximately 3215 zfs-receive > > > jobs it do not crashes anymore, but the last started zfs-receive > > > jobs is hanging, cannot be killed, even not with -9. Even other > > > zfs commands are hanging and cannot be killed, while zpool commands > > > seems to be not affected. > > >=20 > > > root 86397 0.0 0.0 3920 1308 ?? D 3:18AM 0:00.29 zfs rec= eive -Fv zzzz/203 > > > root 5001 0.0 0.0 3920 1208 0 D+ 10:45AM 0:00.00 zfs lis= t -t snapshot > > > root 5477 0.0 0.0 3920 1240 3 D+ 11:08AM 0:00.00 zfs list > > >=20 > > > also the sync command I tried to execute hangs forever: > > >=20 > > > root 5457 0.0 0.0 1528 492 2- D+ 11:05AM 0:00.04 sync > > >=20 > > > Other parts of the system which do not have something todo with zfs > > > are still working well. I will leave the machine running in this > > > state, is there something I can do to retrieve other usefull informat= ion > > > for you? > >=20 > > If you can break into debugger and send me 'show alltrace' for starters. >=20 > hmm, maybe you did not get my last mail. > I put the log of this on I did get it, sorry for the delay, I'm quite busy with other stuff. I need to setup machine for HEAD testing, as my current test box is running perforce version. I'd also need 'show lock 0x87aac290' from this machine if its not too late. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --cNdxnHkX5QqsyA0e Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKd+R4ForvXbEpPzQRAqdmAKD6JebTNaU9lzb+nW2i5D0brcThdACgqYNV EcGnGtznD/mhkxBcKXaB6YY= =+fTJ -----END PGP SIGNATURE----- --cNdxnHkX5QqsyA0e-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 07:53:30 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D086F10656DB; Tue, 4 Aug 2009 07:53:30 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by mx1.freebsd.org (Postfix) with ESMTP id 4BAF88FC14; Tue, 4 Aug 2009 07:53:30 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by jpru.ffm.jpru.de (8.13.8/8.13.8) with ESMTP id n747rTLK073546; Tue, 4 Aug 2009 09:53:29 +0200 (CEST) (envelope-from lists@jpru.de) Received: (from unger@localhost) by jpru.ffm.jpru.de (8.13.8/8.13.8/Submit) id n747rTh2073545; Tue, 4 Aug 2009 09:53:29 +0200 (CEST) (envelope-from lists@jpru.de) X-Authentication-Warning: jpru.ffm.jpru.de: unger set sender to lists@jpru.de using -f Date: Tue, 4 Aug 2009 09:53:29 +0200 From: Juergen Unger To: Pawel Jakub Dawidek Message-ID: <20090804075329.GI5813@jpru.ffm.jpru.de> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090804073416.GA4479@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 07:53:31 -0000 Hi Pawel, On Tue, Aug 04, 2009 at 09:34:16AM +0200, Pawel Jakub Dawidek wrote: [...] > > > If you can break into debugger and send me 'show alltrace' for starters. > > > > hmm, maybe you did not get my last mail. > > I put the log of this on > > I did get it, sorry for the delay, I'm quite busy with other stuff. I > need to setup machine for HEAD testing, as my current test box is > running perforce version. > > I'd also need 'show lock 0x87aac290' from this machine if its not too > late. testbox# sysctl debug.kdb.enter=1 KDB: enter: sysctl debug.kdb.enter [thread pid 11635 tid 100472 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> show lock 0x87aac290 class: sx name: dp->dp_config_rwlock state: XLOCK: 0x879e8480 (tid 100130, pid 172, "txg_thread_enter") waiters: shared db> bye, -Juergen- -- ENOSIG From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 09:49:31 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9A8D106564A; Tue, 4 Aug 2009 09:49:31 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 2C1348FC0A; Tue, 4 Aug 2009 09:49:30 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id F148945C99; Tue, 4 Aug 2009 11:49:28 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1768C45684; Tue, 4 Aug 2009 11:49:24 +0200 (CEST) Date: Tue, 4 Aug 2009 11:49:50 +0200 From: Pawel Jakub Dawidek To: Juergen Unger Message-ID: <20090804094950.GD4479@garage.freebsd.pl> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> <20090804075329.GI5813@jpru.ffm.jpru.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pQhZXvAqiZgbeUkD" Content-Disposition: inline In-Reply-To: <20090804075329.GI5813@jpru.ffm.jpru.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 09:49:32 -0000 --pQhZXvAqiZgbeUkD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 04, 2009 at 09:53:29AM +0200, Juergen Unger wrote: > Hi Pawel, >=20 > On Tue, Aug 04, 2009 at 09:34:16AM +0200, Pawel Jakub Dawidek wrote: > [...] > > > > If you can break into debugger and send me 'show alltrace' for star= ters. > > >=20 > > > hmm, maybe you did not get my last mail. > > > I put the log of this on > >=20 > > I did get it, sorry for the delay, I'm quite busy with other stuff. I > > need to setup machine for HEAD testing, as my current test box is > > running perforce version. > >=20 > > I'd also need 'show lock 0x87aac290' from this machine if its not too > > late. >=20 > testbox# sysctl debug.kdb.enter=3D1 > KDB: enter: sysctl debug.kdb.enter > [thread pid 11635 tid 100472 ] > Stopped at kdb_enter+0x3a: movl $0,kdb_why > db> show lock 0x87aac290 > class: sx > name: dp->dp_config_rwlock > state: XLOCK: 0x879e8480 (tid 100130, pid 172, "txg_thread_enter") > waiters: shared > db>=20 Could you also try something like the following from DDB: x/bx 0x879ad8a0,52 --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --pQhZXvAqiZgbeUkD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeAQ+ForvXbEpPzQRAnMUAKDRebiWBcZA+0P8BNCRMFvId9u9KgCfTiGf e3HbC0/IgAYEcVCO3tckxlM= =tpEc -----END PGP SIGNATURE----- --pQhZXvAqiZgbeUkD-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 09:56:50 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FE431065672; Tue, 4 Aug 2009 09:56:50 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by mx1.freebsd.org (Postfix) with ESMTP id 8CD298FC0C; Tue, 4 Aug 2009 09:56:49 +0000 (UTC) (envelope-from lists@jpru.de) Received: from jpru.ffm.jpru.de (jpru.ffm.jpru.de [195.49.136.33]) by jpru.ffm.jpru.de (8.13.8/8.13.8) with ESMTP id n749umTC093624; Tue, 4 Aug 2009 11:56:48 +0200 (CEST) (envelope-from lists@jpru.de) Received: (from unger@localhost) by jpru.ffm.jpru.de (8.13.8/8.13.8/Submit) id n749um7e093623; Tue, 4 Aug 2009 11:56:48 +0200 (CEST) (envelope-from lists@jpru.de) X-Authentication-Warning: jpru.ffm.jpru.de: unger set sender to lists@jpru.de using -f Date: Tue, 4 Aug 2009 11:56:48 +0200 From: Juergen Unger To: Pawel Jakub Dawidek Message-ID: <20090804095648.GL5813@jpru.ffm.jpru.de> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> <20090804075329.GI5813@jpru.ffm.jpru.de> <20090804094950.GD4479@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090804094950.GD4479@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 09:56:50 -0000 On Tue, Aug 04, 2009 at 11:49:50AM +0200, Pawel Jakub Dawidek wrote: > > testbox# sysctl debug.kdb.enter=1 > > KDB: enter: sysctl debug.kdb.enter > > [thread pid 11635 tid 100472 ] > > Stopped at kdb_enter+0x3a: movl $0,kdb_why > > db> show lock 0x87aac290 > > class: sx > > name: dp->dp_config_rwlock > > state: XLOCK: 0x879e8480 (tid 100130, pid 172, "txg_thread_enter") > > waiters: shared > > db> > > Could you also try something like the following from DDB: > > x/bx 0x879ad8a0,52 db> x/bx 0x879ad8a0,52 0x879ad8a0: 82 1a 4b 87 0 0 71 2 0 0 0 0 0 0 0 0 0x879ad8b0: 1 0 0 0 91 1a 4b 87 4 0 0 0 40 92 cb 87 0x879ad8c0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x879ad8d0: 0 0 0 0 9 1f 4b 87 0 0 71 2 0 0 0 0 0x879ad8e0: 0 0 0 0 40 92 cb 87 e8 0 0 0 c8 0 0 0 0x879ad8f0: a8 bb db> -Juergen- -- ENOSIG From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 19:50:54 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BA661065DF0; Tue, 4 Aug 2009 19:50:54 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id C2CFA8FC1C; Tue, 4 Aug 2009 19:50:52 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 0309945C89; Tue, 4 Aug 2009 21:50:50 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0648445C8C; Tue, 4 Aug 2009 21:50:45 +0200 (CEST) Date: Tue, 4 Aug 2009 21:51:12 +0200 From: Pawel Jakub Dawidek To: Juergen Unger Message-ID: <20090804195112.GB2181@garage.freebsd.pl> References: <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> <20090804075329.GI5813@jpru.ffm.jpru.de> <20090804094950.GD4479@garage.freebsd.pl> <20090804095648.GL5813@jpru.ffm.jpru.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Pd0ReVV5GZGQvF3a" Content-Disposition: inline In-Reply-To: <20090804095648.GL5813@jpru.ffm.jpru.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 19:50:55 -0000 --Pd0ReVV5GZGQvF3a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 04, 2009 at 11:56:48AM +0200, Juergen Unger wrote: > On Tue, Aug 04, 2009 at 11:49:50AM +0200, Pawel Jakub Dawidek wrote: > > > testbox# sysctl debug.kdb.enter=3D1 > > > KDB: enter: sysctl debug.kdb.enter > > > [thread pid 11635 tid 100472 ] > > > Stopped at kdb_enter+0x3a: movl $0,kdb_why > > > db> show lock 0x87aac290 > > > class: sx > > > name: dp->dp_config_rwlock > > > state: XLOCK: 0x879e8480 (tid 100130, pid 172, "txg_thread_enter") > > > waiters: shared > > > db>=20 > >=20 > > Could you also try something like the following from DDB: > >=20 > > x/bx 0x879ad8a0,52 >=20 > db> x/bx 0x879ad8a0,52 > 0x879ad8a0: 82 1a 4b 87 0 0 71 2 0 0 0 0 0 0 0= 0 > 0x879ad8b0: 1 0 0 0 91 1a 4b 87 4 0 0 0 40 92 c= b 87 > 0x879ad8c0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0= 0 > 0x879ad8d0: 0 0 0 0 9 1f 4b 87 0 0 71 2 0 0 0= 0 > 0x879ad8e0: 0 0 0 0 40 92 cb 87 e8 0 0 0 c8 0 0= 0 > 0x879ad8f0: a8 bb > db> This is dump of ZFS-specific rrwlock that some threads are waiting for. We can see here that thread owning the lock is 0x87cb9240, which is pid 86397 (zfs recv process). I don't think we will be able to gather more info from here. I'm builing HEAD at the moment and hopefully will be able to reproduce it. Thanks for all the info. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Pd0ReVV5GZGQvF3a Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeJEwForvXbEpPzQRAvC0AJ99Mac7u8i/9Z+IkHYlX5G4s7szUgCgplRj FK7rLRYAfuP2QGxN8WI9RXM= =sk/E -----END PGP SIGNATURE----- --Pd0ReVV5GZGQvF3a-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 20:11:44 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C79B10656B2; Tue, 4 Aug 2009 20:11:44 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 0EFF38FC14; Tue, 4 Aug 2009 20:11:44 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:50297 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MYQLY-00061S-5S; Tue, 04 Aug 2009 22:10:58 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 8B8149FDC; Tue, 4 Aug 2009 22:10:54 +0200 (CEST) Message-Id: From: Thomas Backman To: Pawel Jakub Dawidek In-Reply-To: <20090804195112.GB2181@garage.freebsd.pl> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 4 Aug 2009 22:10:51 +0200 References: <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> <20090804075329.GI5813@jpru.ffm.jpru.de> <20090804094950.GD4479@garage.freebsd.pl> <20090804095648.GL5813@jpru.ffm.jpru.de> <20090804195112.GB2181@garage.freebsd.pl> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MYQLY-00061S-5S. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MYQLY-00061S-5S 43b42a2627c47128438645f442d9f371 Cc: freebsd-fs@FreeBSD.org, FreeBSD current Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 20:11:44 -0000 On Aug 4, 2009, at 21:51, Pawel Jakub Dawidek wrote: > I'm builing HEAD at the moment and hopefully will be > able to reproduce it. Thanks for all the info. Hey Pawel, Sorry to bother (again!), but... If you're building HEAD, could you please look in to the send -R / zfs recv segfault? http://lists.freebsd.org/pipermail/freebsd-current/2009-July/ 010156.html for the patch by its creator, and I hosted it at http://exscape.org/temp/libzfs_sendrecv.new.patch since spacing (I guess) made the patch not work for me by copying/ pasting. It's a simple patch, fixing a big bug (not being able to replicate whole pools properly!), and it works great. For each day that passes, it feels as if this, (and your equally important zfs_vnops work, which is also pretty vital), won't make it into 8.0... Which is why I keep either reminding or bugging people who both know this stuff and can actually make it happen. ;) Regards (and apologies), Thomas From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 20:25:12 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DB97106564A; Tue, 4 Aug 2009 20:25:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 4477B8FC14; Tue, 4 Aug 2009 20:25:10 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 83CC545C99; Tue, 4 Aug 2009 22:25:09 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id EF94145B36; Tue, 4 Aug 2009 22:25:02 +0200 (CEST) Date: Tue, 4 Aug 2009 22:25:28 +0200 From: Pawel Jakub Dawidek To: Thomas Backman Message-ID: <20090804202528.GE2181@garage.freebsd.pl> References: <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> <20090802093016.GB3071@garage.freebsd.pl> <20090803203226.GE5813@jpru.ffm.jpru.de> <20090804073416.GA4479@garage.freebsd.pl> <20090804075329.GI5813@jpru.ffm.jpru.de> <20090804094950.GD4479@garage.freebsd.pl> <20090804095648.GL5813@jpru.ffm.jpru.de> <20090804195112.GB2181@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WK3l2KTTmXPVedZ6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, FreeBSD current Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 20:25:12 -0000 --WK3l2KTTmXPVedZ6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 04, 2009 at 10:10:51PM +0200, Thomas Backman wrote: >=20 > On Aug 4, 2009, at 21:51, Pawel Jakub Dawidek wrote: > >I'm builing HEAD at the moment and hopefully will be > >able to reproduce it. Thanks for all the info. > Hey Pawel, > Sorry to bother (again!), but... If you're building HEAD, could you =20 > please look in to the send -R / zfs recv segfault? > http://lists.freebsd.org/pipermail/freebsd-current/2009-July/=20 > 010156.html for the patch by its creator, and I hosted it at=20 > http://exscape.org/temp/libzfs_sendrecv.new.patch since spacing (I guess)= =20 > made the patch not work for me by copying/ pasting. >=20 > It's a simple patch, fixing a big bug (not being able to replicate =20 > whole pools properly!), and it works great. For each day that passes, =20 > it feels as if this, (and your equally important zfs_vnops work, which = =20 > is also pretty vital), won't make it into 8.0... Which is why I keep =20 > either reminding or bugging people who both know this stuff and can =20 > actually make it happen. ;) This patch is one of the reasons I'm building HEAD, stay tuned:) --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --WK3l2KTTmXPVedZ6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeJk4ForvXbEpPzQRAouXAJ0exWXMo3aAJ/YJykkSbpGObYuS3gCfSSGs siS7RRTz+wPCt4e2j14aStA= =J7St -----END PGP SIGNATURE----- --WK3l2KTTmXPVedZ6-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 21:39:26 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C191210656DD for ; Tue, 4 Aug 2009 21:39:26 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 868948FC19 for ; Tue, 4 Aug 2009 21:39:26 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: (qmail 72011 invoked from network); 4 Aug 2009 21:39:25 -0000 Received: from unknown (HELO ?192.168.0.137?) (spawk@128.238.9.199) by acm.poly.edu with AES256-SHA encrypted SMTP; 4 Aug 2009 21:39:25 -0000 Message-ID: <4A78AA71.9050107@acm.poly.edu> Date: Tue, 04 Aug 2009 17:38:57 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 21:39:28 -0000 Ahoy. I have a seven-disk RAID-Z pool in a 8-BETA2/amd64 machine. One of the disks (ad13) failed to write something today, and the system proceeded to panic. I couldn't get a dump or any otherwise useful information, but the panic made reference to "vdev_is_dead". Upon reboot, it panics again, probably when "zfs mount" is called by its rc.d script: Fatal trap 9: general protection fault while in kernel mode instruction pointer = 0x20:0xffffffff807cbdbb stack pointer = 0x28:0xffffff8077bf54c0 frame pointer = 0x28:0xffffff8077bf54d0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 82 (zfs) panic: from debugger Uptime: 13s Physical memory: 4081 MB Dumping 1245 MB: 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko #0 doadump () at pcpu.h:223 223 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:223 #1 0xffffffff8058ff11 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 #2 0xffffffff805902eb in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:575 #3 0xffffffff801d9997 in db_panic (addr=Variable "addr" is not available. ) at /usr/src/sys/ddb/db_command.c:478 #4 0xffffffff801d9da1 in db_command (last_cmdp=0xffffffff80bd5120, cmd_table=Variable "cmd_table" is not available. ) at /usr/src/sys/ddb/db_command.c:445 #5 0xffffffff801d9ff0 in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #6 0xffffffff801dbf79 in db_trap (type=Variable "type" is not available. ) at /usr/src/sys/ddb/db_main.c:229 #7 0xffffffff805bbd94 in kdb_trap (type=9, code=0, tf=Variable "tf" is not available. ) at /usr/src/sys/kern/subr_kdb.c:534 #8 0xffffffff8086dc5d in trap_fatal (frame=0xffffff8077bf5410, eva=0) at /usr/src/sys/amd64/amd64/trap.c:847 #9 0xffffffff8086e74d in trap (frame=0xffffff8077bf5410) at /usr/src/sys/amd64/amd64/trap.c:639 #10 0xffffffff80857403 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:224 #11 0xffffffff807cbdbb in slab_alloc_item (zone=Variable "zone" is not available. ) at /usr/src/sys/vm/uma_core.c:2300 #12 0xffffffff807ce80e in zone_alloc_item (zone=0xffffff00dffae000, udata=0x0, flags=259) at /usr/src/sys/vm/uma_core.c:2475 #13 0xffffffff807cee03 in keg_alloc_slab (keg=0xffffff00dffad460, zone=0xffffff00dffac380, wait=259) at /usr/src/sys/vm/uma_core.c:826 #14 0xffffffff807cf177 in keg_fetch_slab (keg=0xffffff00dffad460, zone=0xffffff00dffac380, flags=259) at /usr/src/sys/vm/uma_core.c:2152 #15 0xffffffff807cf21e in zone_fetch_slab (zone=0xffffff00dffac380, keg=0xffffff00dffad460, flags=259) at /usr/src/sys/vm/uma_core.c:2212 #16 0xffffffff807d05eb in uma_zalloc_arg (zone=0xffffff00dffac380, udata=0x0, flags=259) at /usr/src/sys/vm/uma_core.c:2381 #17 0xffffffff8057e727 in malloc (size=Variable "size" is not available. ) at uma.h:305 #18 0xffffffff81060365 in metaslab_init (mg=0xffffff0004472980, smo=0xffffff8077bf5730, start=530428461056, size=2147483648, txg=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:294 #19 0xffffffff81071b3e in vdev_metaslab_init (vd=0xffffff0001ecf800, txg=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:796 #20 0xffffffff81071da5 in vdev_load (vd=0xffffff0001ecf800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1531 #21 0xffffffff81071c75 in vdev_load (vd=0xffffff0001ed1800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1526 #22 0xffffffff8106539c in spa_load (spa=0xffffff0001ff0000, config=Variable "config" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1361 #23 0xffffffff81064ee1 in spa_load (spa=0xffffff0001ff0000, config=Variable "config" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1189 #24 0xffffffff810658fd in spa_open_common (pool=Variable "pool" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1474 #25 0xffffffff81065a52 in spa_get_stats (name=0xffffff0001ff5000 "home", config=0xffffff8077bf59e0, altroot=0xffffff0001ff5400 "", buflen=1024) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1671 #26 0xffffffff81093e7c in zfs_ioc_pool_stats (zc=0xffffff0001ff5000) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:914 #27 0xffffffff810941c4 in zfsdev_ioctl (dev=Variable "dev" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:3022 #28 0xffffffff80511c76 in devfs_ioctl_f (fp=0xffffff0001f4bc80, com=3425196549, data=0xffffff0001ff5000, cred=Variable "cred" is not available. ) at /usr/src/sys/fs/devfs/devfs_vnops.c:659 #29 0xffffffff805cb166 in kern_ioctl (td=0xffffff0001f0c390, fd=3, com=3425196549, data=0xffffff0001ff5000 "home") at file.h:262 #30 0xffffffff805cb38e in ioctl (td=0xffffff0001f0c390, uap=0xffffff8077bf5bf0) at /usr/src/sys/kern/sys_generic.c:678 #31 0xffffffff8086e28f in syscall (frame=0xffffff8077bf5c80) at /usr/src/sys/amd64/amd64/trap.c:984 #32 0xffffffff808576e1 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:373 #33 0x0000000800fe1d0c in ?? () Booting the system without the disk causes any "zfs" or "zpool" commands to hang the system after a while. Breaking to DDB doesn't work using a keyboard and VGA (I don't have any other kind of gear here). In case it is relevant, the pool started life as version 6 and was upgraded using 7.2-STABLE shortly after the version 13 MFC. The output of "zdb" with all disks connected: home version=13 name='home' state=0 txg=16061492 pool_guid=14089219607492705674 hostid=413956888 hostname='unset' vdev_tree type='root' id=0 guid=14089219607492705674 children[0] type='raidz' id=0 guid=17899218839424019335 nparity=1 metaslab_array=14 metaslab_shift=31 ashift=9 asize=2800585539584 is_log=0 children[0] type='disk' id=0 guid=15839907043443901501 path='/dev/ad4' devid='ad:3QK08728' whole_disk=0 DTL=389 children[1] type='disk' id=1 guid=13623369126078337737 path='/dev/ad16' devid='ad:9QH04HJN' whole_disk=0 DTL=391 children[2] type='disk' id=2 guid=15619490422714555908 path='/dev/ad14' devid='ad:5NF1DDXR' whole_disk=0 DTL=390 children[3] type='disk' id=3 guid=6995275135550350664 path='/dev/ad15' devid='ad:9QG93JHX' whole_disk=0 DTL=386 children[4] type='disk' id=4 guid=10651992494569677081 path='/dev/ad13' devid='ad:9QH04GTY' whole_disk=0 DTL=388 children[5] type='disk' id=5 guid=10503557489947490214 path='/dev/ad18' devid='ad:5NF1DDVB' whole_disk=0 DTL=387 children[6] type='disk' id=6 guid=17574056058658811312 path='/dev/ad12' devid='ad:9QG90QA2' whole_disk=0 DTL=392 Can anyone help? I would be content to at least have access to the filesystem in degraded mode. -Boris From owner-freebsd-fs@FreeBSD.ORG Tue Aug 4 22:01:50 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE99D10656D2 for ; Tue, 4 Aug 2009 22:01:50 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5AA478FC20 for ; Tue, 4 Aug 2009 22:01:50 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: (qmail 72546 invoked from network); 4 Aug 2009 22:01:49 -0000 Received: from unknown (HELO ?192.168.0.137?) (spawk@128.238.9.199) by acm.poly.edu with AES256-SHA encrypted SMTP; 4 Aug 2009 22:01:49 -0000 Message-ID: <4A78AFB2.10103@acm.poly.edu> Date: Tue, 04 Aug 2009 18:01:22 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4A78AA71.9050107@acm.poly.edu> In-Reply-To: <4A78AA71.9050107@acm.poly.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2009 22:01:51 -0000 Boris Kochergin wrote: > Ahoy. I have a seven-disk RAID-Z pool in a 8-BETA2/amd64 machine. One > of the disks (ad13) failed to write something today, and the system > proceeded to panic. I couldn't get a dump or any otherwise useful > information, but the panic made reference to "vdev_is_dead". Upon > reboot, it panics again, probably when "zfs mount" is called by its > rc.d script: > > Fatal trap 9: general protection fault while in kernel mode > instruction pointer = 0x20:0xffffffff807cbdbb > stack pointer = 0x28:0xffffff8077bf54c0 > frame pointer = 0x28:0xffffff8077bf54d0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 82 (zfs) > panic: from debugger > Uptime: 13s > Physical memory: 4081 MB > Dumping 1245 MB: 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 > 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 > 814 798 782 766 750 734 718 702 686 670 654 638 622 606 590 574 558 > 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 > 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /boot/kernel/zfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols > from /boot/kernel/opensolaris.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > #0 doadump () at pcpu.h:223 > 223 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) where > #0 doadump () at pcpu.h:223 > #1 0xffffffff8058ff11 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:419 > #2 0xffffffff805902eb in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:575 > #3 0xffffffff801d9997 in db_panic (addr=Variable "addr" is not > available. > ) at /usr/src/sys/ddb/db_command.c:478 > #4 0xffffffff801d9da1 in db_command (last_cmdp=0xffffffff80bd5120, > cmd_table=Variable "cmd_table" is not available. > ) at /usr/src/sys/ddb/db_command.c:445 > #5 0xffffffff801d9ff0 in db_command_loop () at > /usr/src/sys/ddb/db_command.c:498 > #6 0xffffffff801dbf79 in db_trap (type=Variable "type" is not available. > ) at /usr/src/sys/ddb/db_main.c:229 > #7 0xffffffff805bbd94 in kdb_trap (type=9, code=0, tf=Variable "tf" > is not available. > ) at /usr/src/sys/kern/subr_kdb.c:534 > #8 0xffffffff8086dc5d in trap_fatal (frame=0xffffff8077bf5410, eva=0) > at /usr/src/sys/amd64/amd64/trap.c:847 > #9 0xffffffff8086e74d in trap (frame=0xffffff8077bf5410) at > /usr/src/sys/amd64/amd64/trap.c:639 > #10 0xffffffff80857403 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:224 > #11 0xffffffff807cbdbb in slab_alloc_item (zone=Variable "zone" is not > available. > ) at /usr/src/sys/vm/uma_core.c:2300 > #12 0xffffffff807ce80e in zone_alloc_item (zone=0xffffff00dffae000, > udata=0x0, flags=259) at /usr/src/sys/vm/uma_core.c:2475 > #13 0xffffffff807cee03 in keg_alloc_slab (keg=0xffffff00dffad460, > zone=0xffffff00dffac380, wait=259) at /usr/src/sys/vm/uma_core.c:826 > #14 0xffffffff807cf177 in keg_fetch_slab (keg=0xffffff00dffad460, > zone=0xffffff00dffac380, flags=259) at /usr/src/sys/vm/uma_core.c:2152 > #15 0xffffffff807cf21e in zone_fetch_slab (zone=0xffffff00dffac380, > keg=0xffffff00dffad460, flags=259) at /usr/src/sys/vm/uma_core.c:2212 > #16 0xffffffff807d05eb in uma_zalloc_arg (zone=0xffffff00dffac380, > udata=0x0, flags=259) at /usr/src/sys/vm/uma_core.c:2381 > #17 0xffffffff8057e727 in malloc (size=Variable "size" is not available. > ) at uma.h:305 > #18 0xffffffff81060365 in metaslab_init (mg=0xffffff0004472980, > smo=0xffffff8077bf5730, start=530428461056, size=2147483648, txg=0) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:294 > > #19 0xffffffff81071b3e in vdev_metaslab_init (vd=0xffffff0001ecf800, > txg=0) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:796 > > #20 0xffffffff81071da5 in vdev_load (vd=0xffffff0001ecf800) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1531 > > #21 0xffffffff81071c75 in vdev_load (vd=0xffffff0001ed1800) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1526 > > #22 0xffffffff8106539c in spa_load (spa=0xffffff0001ff0000, > config=Variable "config" is not available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1361 > > #23 0xffffffff81064ee1 in spa_load (spa=0xffffff0001ff0000, > config=Variable "config" is not available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1189 > > #24 0xffffffff810658fd in spa_open_common (pool=Variable "pool" is not > available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1474 > > #25 0xffffffff81065a52 in spa_get_stats (name=0xffffff0001ff5000 > "home", config=0xffffff8077bf59e0, altroot=0xffffff0001ff5400 "", > buflen=1024) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1671 > > #26 0xffffffff81093e7c in zfs_ioc_pool_stats (zc=0xffffff0001ff5000) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:914 > > #27 0xffffffff810941c4 in zfsdev_ioctl (dev=Variable "dev" is not > available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:3022 > > #28 0xffffffff80511c76 in devfs_ioctl_f (fp=0xffffff0001f4bc80, > com=3425196549, data=0xffffff0001ff5000, cred=Variable "cred" is not > available. > ) at /usr/src/sys/fs/devfs/devfs_vnops.c:659 > #29 0xffffffff805cb166 in kern_ioctl (td=0xffffff0001f0c390, fd=3, > com=3425196549, data=0xffffff0001ff5000 "home") at file.h:262 > #30 0xffffffff805cb38e in ioctl (td=0xffffff0001f0c390, > uap=0xffffff8077bf5bf0) at /usr/src/sys/kern/sys_generic.c:678 > #31 0xffffffff8086e28f in syscall (frame=0xffffff8077bf5c80) at > /usr/src/sys/amd64/amd64/trap.c:984 > #32 0xffffffff808576e1 in Xfast_syscall () at > /usr/src/sys/amd64/amd64/exception.S:373 > #33 0x0000000800fe1d0c in ?? () > > Booting the system without the disk causes any "zfs" or "zpool" > commands to hang the system after a while. Breaking to DDB doesn't > work using a keyboard and VGA (I don't have any other kind of gear > here). In case it is relevant, the pool started life as version 6 and > was upgraded using 7.2-STABLE shortly after the version 13 MFC. The > output of "zdb" with all disks connected: > > home > version=13 > name='home' > state=0 > txg=16061492 > pool_guid=14089219607492705674 > hostid=413956888 > hostname='unset' > vdev_tree > type='root' > id=0 > guid=14089219607492705674 > children[0] > type='raidz' > id=0 > guid=17899218839424019335 > nparity=1 > metaslab_array=14 > metaslab_shift=31 > ashift=9 > asize=2800585539584 > is_log=0 > children[0] > type='disk' > id=0 > guid=15839907043443901501 > path='/dev/ad4' > devid='ad:3QK08728' > whole_disk=0 > DTL=389 > children[1] > type='disk' > id=1 > guid=13623369126078337737 > path='/dev/ad16' > devid='ad:9QH04HJN' > whole_disk=0 > DTL=391 > children[2] > type='disk' > id=2 > guid=15619490422714555908 > path='/dev/ad14' > devid='ad:5NF1DDXR' > whole_disk=0 > DTL=390 > children[3] > type='disk' > id=3 > guid=6995275135550350664 > path='/dev/ad15' > devid='ad:9QG93JHX' > whole_disk=0 > DTL=386 > children[4] > type='disk' > id=4 > guid=10651992494569677081 > path='/dev/ad13' > devid='ad:9QH04GTY' > whole_disk=0 > DTL=388 > children[5] > type='disk' > id=5 > guid=10503557489947490214 > path='/dev/ad18' > devid='ad:5NF1DDVB' > whole_disk=0 > DTL=387 > children[6] > type='disk' > id=6 > guid=17574056058658811312 > path='/dev/ad12' > devid='ad:9QG90QA2' > whole_disk=0 > DTL=392 > > Can anyone help? I would be content to at least have access to the > filesystem in degraded mode. > > -Boris > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" In a subsequent attempt at "zfs mount -a", the following panic happened: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xffffffff813dadb5 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff805951a5 stack pointer = 0x28:0xffffff8077eb3360 frame pointer = 0x28:0xffffff8077eb3370 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 832 (zfs) panic: from debugger Uptime: 2m32s Physical memory: 4082 MB Dumping 1282 MB: 1267 1251 1235 1219 1203 1187 1171 1155 1139 1123 1107 1091 1075 1059 1043 1027 1011 995 979 963 947 931 915 899 883 867 851 835 819 803 787 771 755 739 723 707 691 675 659 643 627 611 595 579 563 547 531 515 499 483 467 451 435 419 403 387 371 355 339 323 307 291 275 259 243 227 211 195 179 163 147 131 115 99 83 67 51 35 19 3 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko #0 doadump () at pcpu.h:223 223 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:223 #1 0xffffffff8058d881 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 #2 0xffffffff8058dc5b in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:575 #3 0xffffffff801d9767 in db_panic (addr=Variable "addr" is not available. ) at /usr/src/sys/ddb/db_command.c:478 #4 0xffffffff801d9b71 in db_command (last_cmdp=0xffffffff80bd2120, cmd_table=Variable "cmd_table" is not available. ) at /usr/src/sys/ddb/db_command.c:445 #5 0xffffffff801d9dc0 in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #6 0xffffffff801dbd49 in db_trap (type=Variable "type" is not available. ) at /usr/src/sys/ddb/db_main.c:229 #7 0xffffffff805b9704 in kdb_trap (type=12, code=0, tf=Variable "tf" is not available. ) at /usr/src/sys/kern/subr_kdb.c:534 #8 0xffffffff8086b5cd in trap_fatal (frame=0xffffff8077eb32b0, eva=18446744071582887349) at /usr/src/sys/amd64/amd64/trap.c:847 #9 0xffffffff8086b994 in trap_pfault (frame=0xffffff8077eb32b0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:768 #10 0xffffffff8086c16b in trap (frame=0xffffff8077eb32b0) at /usr/src/sys/amd64/amd64/trap.c:494 #11 0xffffffff80854d73 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:224 #12 0xffffffff805951a5 in _sx_xlock (sx=0xffffffff813dad9d, opts=0, file=0xffffffff810f57f0 "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c", line=967) at atomic.h:147 #13 0xffffffff810392e5 in add_reference (ab=0xffffff002c03b340, hash_lock=Variable "hash_lock" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:967 #14 0xffffffff8103d377 in arc_buf_add_ref (buf=0xffffff0003ee87e0, tag=0xffffff002c046c40) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1208 #15 0xffffffff8103fe0d in dbuf_hold_impl (dn=0xffffff0003eec300, level=Variable "level" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1633 #16 0xffffffff81040ddb in dbuf_hold (dn=Variable "dn" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1689 #17 0xffffffff8104d5bc in dnode_hold_impl (os=0xffffff0003a01400, object=754, flag=1, tag=Variable "tag" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:584 #18 0xffffffff81042c5a in dmu_bonus_hold (os=Variable "os" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:147 #19 0xffffffff81071bb7 in vdev_metaslab_init (vd=0xffffff00036dc800, txg=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:787 #20 0xffffffff81071da5 in vdev_load (vd=0xffffff00036dc800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1531 #21 0xffffffff81071c75 in vdev_load (vd=0xffffff00036db800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1526 #22 0xffffffff8106539c in spa_load (spa=0xffffff00034f7000, config=Variable "config" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1361 #23 0xffffffff81064ee1 in spa_load (spa=0xffffff00034f7000, config=Variable "config" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1189 #24 0xffffffff810658fd in spa_open_common (pool=Variable "pool" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1474 #25 0xffffffff810512af in dsl_dir_open_spa (spa=0x0, name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c:314 #26 0xffffffff8105627b in dsl_dataset_hold (name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:571 #27 0xffffffff8104867f in dmu_objset_open (name=0xffffff0003013000 "home", type=DMU_OST_ANY, mode=9, osp=0xffffff8077eb39e0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:349 #28 0xffffffff810936e2 in zfs_ioc_objset_stats (zc=0xffffff0003013000) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:1231 #29 0xffffffff810941c4 in zfsdev_ioctl (dev=Variable "dev" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:3022 #30 0xffffffff80511a46 in devfs_ioctl_f (fp=0xffffff00034eb230, com=3425196561, data=0xffffff0003013000, cred=Variable "cred" is not available. ) at /usr/src/sys/fs/devfs/devfs_vnops.c:659 #31 0xffffffff805c8ad6 in kern_ioctl (td=0xffffff00037ac720, fd=3, com=3425196561, data=0xffffff0003013000 "home") at file.h:262 #32 0xffffffff805c8cfe in ioctl (td=0xffffff00037ac720, uap=0xffffff8077eb3bf0) at /usr/src/sys/kern/sys_generic.c:678 #33 0xffffffff8086bbff in syscall (frame=0xffffff8077eb3c80) at /usr/src/sys/amd64/amd64/trap.c:984 #34 0xffffffff80855051 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:373 #35 0x0000000800fe1d0c in ?? () This isn't related to the problems described in the "zfs: Fatal trap 12: page fault while in kernel mode" thread, is it? I've poked around on it and the panics look different. -Boris From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 00:59:59 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B53A5106566C for ; Wed, 5 Aug 2009 00:59:59 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 76A648FC0A for ; Wed, 5 Aug 2009 00:59:59 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 708D11731C; Wed, 5 Aug 2009 10:45:10 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-56-25.lns10.syd7.internode.on.net [121.44.56.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 69AB81725C; Wed, 5 Aug 2009 10:45:06 +1000 (EST) Message-ID: <4A78D597.8030907@modulus.org> Date: Wed, 05 Aug 2009 10:43:03 +1000 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: Boris Kochergin , freebsd-fs@freebsd.org References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> In-Reply-To: <4A78AFB2.10103@acm.poly.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 01:00:00 -0000 Have you tried setting the "failmode = continue" property on the zpool? The default failmode is "panic". - Andrew From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 01:38:58 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A81FB106566B for ; Wed, 5 Aug 2009 01:38:58 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 468638FC22 for ; Wed, 5 Aug 2009 01:38:57 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: (qmail 76993 invoked from network); 5 Aug 2009 01:38:56 -0000 Received: from unknown (HELO ?192.168.0.2?) (spawk@69.123.45.64) by acm.poly.edu with AES256-SHA encrypted SMTP; 5 Aug 2009 01:38:56 -0000 Message-ID: <4A78E294.2000206@acm.poly.edu> Date: Tue, 04 Aug 2009 21:38:28 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: Andrew Snow References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <4A78D597.8030907@modulus.org> In-Reply-To: <4A78D597.8030907@modulus.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 01:38:59 -0000 Andrew Snow wrote: > > Have you tried setting the "failmode = continue" property on the > zpool? The default failmode is "panic". > > > - Andrew > > The default failmode appears to be "wait," as that is what it is on all my other machines with which I have not fiddled. I wouldn't be able to get far enough to set it, either way. Thanks, though. -Boris From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 06:50:07 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E92F1065670; Wed, 5 Aug 2009 06:50:07 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 88D7E8FC1D; Wed, 5 Aug 2009 06:50:05 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B2B7845C89; Wed, 5 Aug 2009 08:50:03 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B784145684; Wed, 5 Aug 2009 08:49:57 +0200 (CEST) Date: Wed, 5 Aug 2009 08:50:22 +0200 From: Pawel Jakub Dawidek To: Thomas Backman Message-ID: <20090805065022.GI2181@garage.freebsd.pl> References: <4A719CA4.4060400@freebsd.org> <19347561-3CE6-40B3-930A-EB9925D3AFD1@exscape.org> <4A71AD29.10705@freebsd.org> <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GBuTPvBEOL0MYPgd" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 06:50:07 -0000 --GBuTPvBEOL0MYPgd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 31, 2009 at 11:05:01AM +0200, Thomas Backman wrote: > I'm able to reliably reproduce this panic, by having zfs recv destroy =20 > a filesystem on the receiving end. >=20 > 1) Use DDEBUG=3D1, I guess > 2) Create a FS on the source pool you don't care about: zfs create -o =20 > mountpoint=3D/testfs source/testfs > 3) Clone a pool to another: zfs snapshot -r source@snap && zfs send -R = =20 > source@snap | zfs recv -Fvd target > 4) zfs destroy -r source/testfs > 4) zfs snapshot -r source@snap2 && zfs send -R -I snap source@snap2 | =20 > zfs recv -Fvd target > 5) ^ Panic while receiving the FS the destroyed one is mounted under. =20 > In my case, this was tank/root three times out of three; I then tried =20 > creating testfs under /tmp (tank/tmp/testfs), *mounting* it under /usr/= =20 > testfs, and it panics on receiving tank/usr: [...] I repeated precisevly those steps and it doesn't panic for me. Could you confirm that you use this patch? http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch If so, could you give me exact steps and all of them how to reproduce it? Starting with pool creation. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --GBuTPvBEOL0MYPgd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeSuuForvXbEpPzQRAuS/AJwOZlaxRRNfY0kHKqIZYqPHLGsCpACeOr0h M3eYympUqyLOKJFgtKHsNXw= =eUAy -----END PGP SIGNATURE----- --GBuTPvBEOL0MYPgd-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 07:09:34 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6421106564A; Wed, 5 Aug 2009 07:09:34 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 539AA8FC13; Wed, 5 Aug 2009 07:09:34 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:45611 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MYacV-00069z-4O; Wed, 05 Aug 2009 09:09:09 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 60EE318301B; Wed, 5 Aug 2009 09:09:03 +0200 (CEST) Message-Id: <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> From: Thomas Backman To: Pawel Jakub Dawidek In-Reply-To: <20090805065022.GI2181@garage.freebsd.pl> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 5 Aug 2009 09:09:01 +0200 References: <4A719CA4.4060400@freebsd.org> <19347561-3CE6-40B3-930A-EB9925D3AFD1@exscape.org> <4A71AD29.10705@freebsd.org> <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> <20090805065022.GI2181@garage.freebsd.pl> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MYacV-00069z-4O. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MYacV-00069z-4O 72405053903911bc3b59f22783f4bf82 Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 07:09:35 -0000 On Aug 5, 2009, at 08:50, Pawel Jakub Dawidek wrote: > On Fri, Jul 31, 2009 at 11:05:01AM +0200, Thomas Backman wrote: >> I'm able to reliably reproduce this panic, by having zfs recv destroy >> a filesystem on the receiving end. >> >> 1) Use DDEBUG=1, I guess >> 2) Create a FS on the source pool you don't care about: zfs create -o >> mountpoint=/testfs source/testfs >> 3) Clone a pool to another: zfs snapshot -r source@snap && zfs send >> -R >> source@snap | zfs recv -Fvd target >> 4) zfs destroy -r source/testfs >> 4) zfs snapshot -r source@snap2 && zfs send -R -I snap source@snap2 | >> zfs recv -Fvd target >> 5) ^ Panic while receiving the FS the destroyed one is mounted under. >> In my case, this was tank/root three times out of three; I then tried >> creating testfs under /tmp (tank/tmp/testfs), *mounting* it under / >> usr/ >> testfs, and it panics on receiving tank/usr: > [...] > > I repeated precisevly those steps and it doesn't panic for me. > Could you confirm that you use this patch? > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch > > If so, could you give me exact steps and all of them how to > reproduce it? > Starting with pool creation. Yup, I'm using that patch (I diffed the diffs, heh). I'll try to write a script to recreate the panic; I hope it's as easy as in real-world conditions though. Regards, Thomas From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 07:21:26 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76BB0106566B; Wed, 5 Aug 2009 07:21:26 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id E7F618FC25; Wed, 5 Aug 2009 07:21:25 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:47802 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MYanz-00018A-4y; Wed, 05 Aug 2009 09:21:01 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 163E1183053; Wed, 5 Aug 2009 09:21:00 +0200 (CEST) Message-Id: <3ECC4BA0-F1EF-4039-9F39-68532851B572@exscape.org> From: Thomas Backman To: Pawel Jakub Dawidek In-Reply-To: <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 5 Aug 2009 09:20:58 +0200 References: <4A719CA4.4060400@freebsd.org> <19347561-3CE6-40B3-930A-EB9925D3AFD1@exscape.org> <4A71AD29.10705@freebsd.org> <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> <20090805065022.GI2181@garage.freebsd.pl> <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MYanz-00018A-4y. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MYanz-00018A-4y c85bfb93361f1f8a5672bd2a92ceb328 Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 07:21:27 -0000 On Aug 5, 2009, at 09:09, Thomas Backman wrote: > > On Aug 5, 2009, at 08:50, Pawel Jakub Dawidek wrote: > >> On Fri, Jul 31, 2009 at 11:05:01AM +0200, Thomas Backman wrote: >>> I'm able to reliably reproduce this panic, by having zfs recv >>> destroy >>> a filesystem on the receiving end. >>> >>> 1) Use DDEBUG=1, I guess >>> 2) Create a FS on the source pool you don't care about: zfs create >>> -o >>> mountpoint=/testfs source/testfs >>> 3) Clone a pool to another: zfs snapshot -r source@snap && zfs >>> send -R >>> source@snap | zfs recv -Fvd target >>> 4) zfs destroy -r source/testfs >>> 4) zfs snapshot -r source@snap2 && zfs send -R -I snap >>> source@snap2 | >>> zfs recv -Fvd target >>> 5) ^ Panic while receiving the FS the destroyed one is mounted >>> under. >>> In my case, this was tank/root three times out of three; I then >>> tried >>> creating testfs under /tmp (tank/tmp/testfs), *mounting* it under / >>> usr/ >>> testfs, and it panics on receiving tank/usr: >> [...] >> >> I repeated precisevly those steps and it doesn't panic for me. >> Could you confirm that you use this patch? >> >> http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch >> >> If so, could you give me exact steps and all of them how to >> reproduce it? >> Starting with pool creation. > Yup, I'm using that patch (I diffed the diffs, heh). I'll try to > write a script to recreate the panic; I hope it's as easy as in real- > world conditions though. Oh! I noticed that I actually finised my test case for this panic; I thought I stopped midway, but that was something else. Here are all the details: http://lists.freebsd.org/pipermail/freebsd-fs/2009-July/006585.html (If you have the libzfs_sendrecv patch, your own vnops patch and DDEBUG=1, there's no need to patch anything at all.) Regards, Thomas From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 09:38:07 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25339106566B; Wed, 5 Aug 2009 09:38:07 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 5B82F8FC14; Wed, 5 Aug 2009 09:38:05 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E939F45B36; Wed, 5 Aug 2009 11:38:03 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4115B45684; Wed, 5 Aug 2009 11:37:59 +0200 (CEST) Date: Wed, 5 Aug 2009 11:38:26 +0200 From: Pawel Jakub Dawidek To: Thomas Backman Message-ID: <20090805093825.GC1784@garage.freebsd.pl> References: <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> <20090805065022.GI2181@garage.freebsd.pl> <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> <3ECC4BA0-F1EF-4039-9F39-68532851B572@exscape.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZJcv+A0YCCLh2VIg" Content-Disposition: inline In-Reply-To: <3ECC4BA0-F1EF-4039-9F39-68532851B572@exscape.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 09:38:07 -0000 --ZJcv+A0YCCLh2VIg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 05, 2009 at 09:20:58AM +0200, Thomas Backman wrote: > Oh! I noticed that I actually finised my test case for this panic; I =20 > thought I stopped midway, but that was something else. > Here are all the details:=20 > http://lists.freebsd.org/pipermail/freebsd-fs/2009-July/006585.html > (If you have the libzfs_sendrecv patch, your own vnops patch and =20 > DDEBUG=3D1, there's no need to patch anything at all.) I belive it is safe to do the following: http://people.freebsd.org/~pjd/patches/zfs_vfsops.c.2.patch Could you give it a try? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ZJcv+A0YCCLh2VIg Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeVMRForvXbEpPzQRAtHdAKCA7i+WC9ooMYi1IJZAkIafBHU3XwCgx/1e qZLPiK9IzTjjQr1dt3JSh6U= =0KFn -----END PGP SIGNATURE----- --ZJcv+A0YCCLh2VIg-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 10:37:08 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB320106566C; Wed, 5 Aug 2009 10:37:08 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 7AC708FC15; Wed, 5 Aug 2009 10:37:08 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:34033 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MYdre-0004eg-6R; Wed, 05 Aug 2009 12:37:00 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 15BAA18301F; Wed, 5 Aug 2009 12:36:58 +0200 (CEST) Message-Id: <1DA8C406-D35E-4AF3-990F-58753F305444@exscape.org> From: Thomas Backman To: Pawel Jakub Dawidek In-Reply-To: <20090805093825.GC1784@garage.freebsd.pl> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 5 Aug 2009 12:36:56 +0200 References: <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> <20090805065022.GI2181@garage.freebsd.pl> <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> <3ECC4BA0-F1EF-4039-9F39-68532851B572@exscape.org> <20090805093825.GC1784@garage.freebsd.pl> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MYdre-0004eg-6R. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MYdre-0004eg-6R fea2bdc749184c7309441808bb159ac3 Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 10:37:09 -0000 On Aug 5, 2009, at 11:38, Pawel Jakub Dawidek wrote: > On Wed, Aug 05, 2009 at 09:20:58AM +0200, Thomas Backman wrote: >> Oh! I noticed that I actually finised my test case for this panic; I >> thought I stopped midway, but that was something else. >> Here are all the details: >> http://lists.freebsd.org/pipermail/freebsd-fs/2009-July/006585.html >> (If you have the libzfs_sendrecv patch, your own vnops patch and >> DDEBUG=1, there's no need to patch anything at all.) > > I belive it is safe to do the following: > > http://people.freebsd.org/~pjd/patches/zfs_vfsops.c.2.patch > > Could you give it a try? Not right now (today), I'm afraid. I'll look in to it as soon as I can (likely tomorrow), though. Regards, Thomas From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 11:56:03 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A89B6106564A for ; Wed, 5 Aug 2009 11:56:03 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id F2EDC8FC0C for ; Wed, 5 Aug 2009 11:56:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 12EA445C8A; Wed, 5 Aug 2009 13:56:00 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 22ADD456B1; Wed, 5 Aug 2009 13:55:55 +0200 (CEST) Date: Wed, 5 Aug 2009 13:56:21 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090805115621.GG1784@garage.freebsd.pl> References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NzX0AQGjRQPusK/O" Content-Disposition: inline In-Reply-To: <4A78AFB2.10103@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 11:56:03 -0000 --NzX0AQGjRQPusK/O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 04, 2009 at 06:01:22PM -0400, Boris Kochergin wrote: > In a subsequent attempt at "zfs mount -a", the following panic happened: >=20 > Fatal trap 12: page fault while in kernel mode [...] Could you try to mount file systems one by one? For example you have: tank tank/foo tank/foo/bar tank/baz And you do: # mount -t zfs tank /tank # mount -t zfs tank/foo /tank/foo # mount -t zfs tank/foo/bar /tank/foo/bar # mount -t zfs tank/baz /tank/baz --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --NzX0AQGjRQPusK/O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeXNlForvXbEpPzQRAhmhAKCC9ljKuiFq9NJNHgjLhsXUYC2irACePIUv /kJAiBq7RJngIxhyrVXiPCg= =Zjp1 -----END PGP SIGNATURE----- --NzX0AQGjRQPusK/O-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 12:07:16 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D29A2106567A; Wed, 5 Aug 2009 12:07:16 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 8395A8FC08; Wed, 5 Aug 2009 12:07:16 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:40408 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MYfGw-00025c-4p; Wed, 05 Aug 2009 14:07:12 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 1678A182FE1; Wed, 5 Aug 2009 14:07:11 +0200 (CEST) Message-Id: <4F8F7C4A-A760-427A-A97B-92C548DA7BEE@exscape.org> From: Thomas Backman To: Pawel Jakub Dawidek In-Reply-To: <20090805093825.GC1784@garage.freebsd.pl> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 5 Aug 2009 14:07:08 +0200 References: <7544AED1-1216-4A24-B287-F54117641F76@exscape.org> <4A71B239.8060007@freebsd.org> <3AA3C1CB-CEF7-46CC-A9C7-1648093D679E@exsca!pe.org> <4A71BED8.7050300@freebsd.org> <20090805065022.GI2181@garage.freebsd.pl> <7C3499A8-A389-4F28-A800-B6C31B9E09C4@exscape.org> <3ECC4BA0-F1EF-4039-9F39-68532851B572@exscape.org> <20090805093825.GC1784@garage.freebsd.pl> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MYfGw-00025c-4p. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MYfGw-00025c-4p c7fd3707a462c73abece6f994d13433d Cc: freebsd-fs@freebsd.org, FreeBSD current , Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 12:07:17 -0000 On Aug 5, 2009, at 11:38, Pawel Jakub Dawidek wrote: > On Wed, Aug 05, 2009 at 09:20:58AM +0200, Thomas Backman wrote: >> Oh! I noticed that I actually finised my test case for this panic; I >> thought I stopped midway, but that was something else. >> Here are all the details: >> http://lists.freebsd.org/pipermail/freebsd-fs/2009-July/006585.html >> (If you have the libzfs_sendrecv patch, your own vnops patch and >> DDEBUG=1, there's no need to patch anything at all.) > > I belive it is safe to do the following: > > http://people.freebsd.org/~pjd/patches/zfs_vfsops.c.2.patch > > Could you give it a try? OK, I was wrong, I could give it a try now. :) It seems to work! -DDEBUG=1 and no panic. Just to be sure I tried to revert the patch, and sure enough, solaris assert panic at that line. The backup script (aka. real world test) also did not panic, which it did before. Regards, Thomas From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 13:33:37 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF66B10656C9 for ; Wed, 5 Aug 2009 13:33:37 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6BCED8FC1C for ; Wed, 5 Aug 2009 13:33:36 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: (qmail 94616 invoked from network); 5 Aug 2009 13:33:35 -0000 Received: from unknown (HELO ?192.168.0.137?) (spawk@128.238.9.199) by acm.poly.edu with AES256-SHA encrypted SMTP; 5 Aug 2009 13:33:35 -0000 Message-ID: <4A798A12.4070408@acm.poly.edu> Date: Wed, 05 Aug 2009 09:33:06 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> In-Reply-To: <20090805115621.GG1784@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 13:33:38 -0000 Pawel Jakub Dawidek wrote: > On Tue, Aug 04, 2009 at 06:01:22PM -0400, Boris Kochergin wrote: > >> In a subsequent attempt at "zfs mount -a", the following panic happened: >> >> Fatal trap 12: page fault while in kernel mode >> > [...] > > Could you try to mount file systems one by one? For example you have: > > tank > tank/foo > tank/foo/bar > tank/baz > > And you do: > > # mount -t zfs tank /tank > # mount -t zfs tank/foo /tank/foo > # mount -t zfs tank/foo/bar /tank/foo/bar > # mount -t zfs tank/baz /tank/baz > > There is only one filesystem (home), but "mount -t zfs home /usr/home" did work while the problem disk (ad13) was disconnected from the system. I started moving its data off to a new geom_raid3 array, and there was a panic shortly after: Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xffffffffffffffe9 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8103a9e7 stack pointer = 0x28:0xffffff8077f26430 frame pointer = 0x28:0xffffff8077f26500 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 972 (cp) panic: from debugger Uptime: 4m28s Physical memory: 4082 MB Dumping 2532 MB: 2517 2501 2485 2469 2453 2437 2421 2405 2389 2373 2357 2341 2325 2309 2293 2277 2261 2245 2229 2213 2197 2181 2165 2149 2133 2117 2101 2085 2069 2053 2037 2021 2005 1989 1973 1957 1941 1925 1909 1893 1877 1861 1845 1829 1813 1797 1781 1765 1749 1733 1717 1701 1685 1669 1653 1637 1621 1605 1589 1573 1557 1541 1525 1509 1493 1477 1461 1445 1429 1413 1397 1381 1365 1349 1333 1317 1301 1285 1269 1253 1237 1221 1205 1189 1173 1157 1141 1125 1109 1093 1077 1061 1045 1029 1013 997 981 965 949 933 917 901 885 869 853 837 821 805 789 773 757 741 725 709 693 677 661 645 629 613 597 581 565 549 533 517 501 485 469 453 437 421 405 389 373 357 341 325 309 293 277 261 245 229 213 197 181 165 149 133 117 101 85 69 53 37 21 5 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /usr/src/sys/modules/geom/geom_raid3/geom_raid3.ko...done. Loaded symbols for /usr/src/sys/modules/geom/geom_raid3/geom_raid3.ko #0 doadump () at pcpu.h:223 223 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:223 #1 0xffffffff8058d881 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 #2 0xffffffff8058dc5b in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:575 #3 0xffffffff801d9767 in db_panic (addr=Variable "addr" is not available. ) at /usr/src/sys/ddb/db_command.c:478 #4 0xffffffff801d9b71 in db_command (last_cmdp=0xffffffff80bd2120, cmd_table=Variable "cmd_table" is not available. ) at /usr/src/sys/ddb/db_command.c:445 #5 0xffffffff801d9dc0 in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #6 0xffffffff801dbd49 in db_trap (type=Variable "type" is not available. ) at /usr/src/sys/ddb/db_main.c:229 #7 0xffffffff805b9704 in kdb_trap (type=12, code=0, tf=Variable "tf" is not available. ) at /usr/src/sys/kern/subr_kdb.c:534 #8 0xffffffff8086b5cd in trap_fatal (frame=0xffffff8077f26380, eva=18446744073709551593) at /usr/src/sys/amd64/amd64/trap.c:847 #9 0xffffffff8086b994 in trap_pfault (frame=0xffffff8077f26380, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:768 #10 0xffffffff8086c16b in trap (frame=0xffffff8077f26380) at /usr/src/sys/amd64/amd64/trap.c:494 #11 0xffffffff80854d73 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:224 #12 0xffffffff8103a9e7 in arc_evict (state=Variable "state" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1489 #13 0xffffffff8103b049 in arc_get_data_buf (buf=0xffffff00873d23f0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2170 #14 0xffffffff8103b46e in arc_buf_alloc (spa=0xffffff0003536000, size=16384, tag=Variable "tag" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1156 #15 0xffffffff8103c6a0 in arc_read_nolock (pio=0xffffff00039a92d0, spa=0xffffff0003536000, bp=0xffffff800947a380, done=0xffffffff8103f360 , private=0xffffff008740dc40, priority=0, zio_flags=1, arc_flags=0xffffff8077f266ec, zb=0xffffff8077f266c0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2607 #16 0xffffffff8103cd6c in arc_read (pio=0xffffff00039a92d0, spa=0xffffff0003536000, bp=0xffffff800947a380, pbuf=0xffffff002d89f5a0, done=0xffffffff8103f360 , private=0xffffff008740dc40, priority=0, zio_flags=1, arc_flags=0xffffff8077f266ec, zb=0xffffff8077f266c0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2508 #17 0xffffffff8103f7e9 in dbuf_read (db=0xffffff008740dc40, zio=0xffffff00039a92d0, flags=14) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:521 #18 0xffffffff8103fd56 in dbuf_findbp (dn=Variable "dn" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1381 #19 0xffffffff8103fe62 in dbuf_hold_impl (dn=0xffffff002d526300, level=Variable "level" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1617 #20 0xffffffff81040ddb in dbuf_hold (dn=Variable "dn" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1689 #21 0xffffffff81042f4d in dmu_buf_hold_array_by_dnode (dn=0xffffff002d526300, offset=Variable "offset" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:223 #22 0xffffffff810433e2 in dmu_buf_hold_array (os=Variable "os" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:284 #23 0xffffffff8104357f in dmu_read_uio (os=Variable "os" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:649 #24 0xffffffff810a21b1 in zfs_freebsd_read (ap=Variable "ap" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:591 #25 0xffffffff806244c0 in vn_read (fp=0xffffff0003435460, uio=0xffffff8077f26b00, active_cred=0xffffff002d376900, flags=Variable "flags" is not available. ) at vnode_if.h:384 #26 0xffffffff805c93a1 in dofileread (td=0xffffff0003780ab0, fd=3, fp=0xffffff0003435460, auio=0xffffff8077f26b00, offset=Variable "offset" is not available. ) at file.h:227 #27 0xffffffff805c9720 in kern_readv (td=0xffffff0003780ab0, fd=3, auio=0xffffff8077f26b00) at /usr/src/sys/kern/sys_generic.c:237 #28 0xffffffff805c9815 in read (td=Variable "td" is not available. ) at /usr/src/sys/kern/sys_generic.c:153 #29 0xffffffff8086bbff in syscall (frame=0xffffff8077f26c80) at /usr/src/sys/amd64/amd64/trap.c:984 #30 0xffffffff80855051 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:373 #31 0x0000000800737d6c in ?? () Previous frame inner to this frame (corrupt stack?) I reconnected the bad disk and tried "mount -t zfs home /usr/home" but the command does not return (it's been running for a few minutes at the time of this writing). However, the machine does not panic or lock up. Thank you for your help. -Boris From owner-freebsd-fs@FreeBSD.ORG Wed Aug 5 14:23:17 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86FC0106564A for ; Wed, 5 Aug 2009 14:23:17 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 228838FC23 for ; Wed, 5 Aug 2009 14:23:16 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4DBA2456B1; Wed, 5 Aug 2009 16:23:15 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 6E0DA45684; Wed, 5 Aug 2009 16:23:10 +0200 (CEST) Date: Wed, 5 Aug 2009 16:23:36 +0200 From: Pawel Jakub Dawidek To: "James R. Van Artsdalen" Message-ID: <20090805142336.GJ1784@garage.freebsd.pl> References: <4A3B1020.2010305@jrv.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="C7Ke/meiCZutM6I/" Content-Disposition: inline In-Reply-To: <4A3B1020.2010305@jrv.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs , Randy Bush Subject: Re: adding drive to raidz1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2009 14:23:17 -0000 --C7Ke/meiCZutM6I/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 18, 2009 at 11:12:16PM -0500, James R. Van Artsdalen wrote: > As a feature suggestion why not reject an "zpool add" of a non-redundant > vdev to a pool of redundant vdev's unless -f is given? A command of > that sort is almost always a mistake so requiring -f would seem no > hardship for anyone... This is rejected: # zpool create foobar raidz md0 md1 md2 # zpool add foobar md3 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses raidz and new vdev is disk --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --C7Ke/meiCZutM6I/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKeZXoForvXbEpPzQRAg+5AJ0VxrO9v1ZCfEVEd0rPBgQMt/12wwCfcvm6 3G+Uu3lNBfCgtgoKHH9vWFU= =0r/S -----END PGP SIGNATURE----- --C7Ke/meiCZutM6I/-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 6 14:21:12 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 144E71065677 for ; Thu, 6 Aug 2009 14:21:12 +0000 (UTC) (envelope-from trevor.hearn@Vanderbilt.Edu) Received: from mailgate.vanderbilt.edu (mailgate.vanderbilt.edu [129.59.4.20]) by mx1.freebsd.org (Postfix) with ESMTP id CC5DD8FC1D for ; Thu, 6 Aug 2009 14:21:11 +0000 (UTC) Received: from its-hcwnem22.ds.vanderbilt.edu ([10.1.137.30]) by mailgate06.csm.vanderbilt.edu (8.14.1/8.14.1) with ESMTP id n76Dp5Sb029275 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Thu, 6 Aug 2009 08:51:05 -0500 Received: from its-hcwnem03.ds.Vanderbilt.edu ([10.1.137.103]) by ITS-SCWNEM24.ds.Vanderbilt.edu ([::1]) with mapi; Thu, 6 Aug 2009 08:51:05 -0500 From: "Hearn, Trevor" To: "freebsd-fs@freebsd.org" Date: Thu, 6 Aug 2009 08:51:04 -0500 Thread-Topic: UFS Filesystem issues, and the loss of my hair... Thread-Index: AQHKFpzLuuBuJvtxq0OeiuOtMxVDrQ== Message-ID: <8E9591D8BCB72D4C8DE0884D9A2932DC35BD34C3@ITS-HCWNEM03.ds.Vanderbilt.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=1.12.8161:2.4.5, 1.2.40, 4.0.166 definitions=2009-08-06_06:2009-07-24, 2009-08-06, 2009-08-06 signatures=0 X-PPS: No, score=0 Subject: UFS Filesystem issues, and the loss of my hair... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Aug 2009 14:21:12 -0000 First off, let me state that I love FreeBSD. I've used it for years, and ha= ve not had any major problems with it... Until now. As you can tell, I work for a major university. I setup a large storage arr= ay to hold data for a project they have here. No great shakes, just some st= andard files and such. The fun started when I started loading users onto th= e system, and they started using it... Isn't that always the case? Now, I g= et ufs_dirbad errors, and the system hard locks. This isn't the worst thing= that could happen, but when you're talking about file partitions the size = that I am using, the fsck takes FOREVER. Somewhere on the order of 1.5 hour= s. During that time, I am bringing the individual shares/partitions online,= but the users suffer. I've asked about this before, in a different forum, = but got no usable information that I could see. So, here goes... The system is as such. A dell 2950 1U server, with a Qlogic Fibre Channel c= ard. It is connected to two Promise Array chassis, 610 series, each with 16= drives. Each chassis is running RAID 6, which gives me about 12.73tb of st= orage per chassis. From there, the logical drives are sliced up into smalle= r partitions. At most, I have a 3.6tb partition. The smallest is a 100gig p= artition. Filesystem Size Used Avail Capacity Mounted on /dev/mfid0s1a 197G 10G 170G 6% / devfs 1.0K 1.0K 0B 100% /dev /dev/da0p1 1.8T 1.5T 130G 92% /slice1 /dev/da0p5 2.7T 1.8T 661G 74% /slice2 /dev/da0p9 250G 21G 209G 9% /slice3 /dev/da1p3 103G 12G 83G 12% /slice4 /dev/da1p4 205G 54G 135G 29% /slice5 /dev/da1p5 103G 7.3G 87G 8% /slice6 /dev/da1p6 103G 22G 72G 23% /slice7 etc... I had to use GPT to setup the partitions, and they are using UFS2 for the f= ilesystem. Now... If that's not fun enough... I have TWO of these creatures= , which RSYNC every 4 hours. The secondary system is across campus, and sit= s idle 99% of the time. Every 4 hours, in a stepped schedule, the primary a= rray syncs to the secondary array. If the primary goes down, I FSCK, and an= y files that are fried, I bring back across from the secondary and replace = them. This has worked OK for a while, but now I am getting Kernel Panics on= a regular basis. I've been told to migrate to a different filesystem, but = my options are ZFS and using GJOURNAL with UFS, from what I can tell. I nee= d something repeatable, simple, and I need something robust. I have NO idea= why I keep getting errors like this, but I imagine it's a cascading effect= of other hangs that have caused more corruption. I'd buy a fella, or gal, a cup of coffee and a pop-tart if they could help = a brother out. I have checked out this link: http://phaq.phunsites.net/2007/07/01/ufs_dirbad-panic-with-mangled-entries-= in-ufs/ and decided that I need to give this a shot after hours, but being the kind= a guy I am, I need to make sure I am covering all of my bases.=20 Anyone got any ideas? Thanks! -T From owner-freebsd-fs@FreeBSD.ORG Thu Aug 6 16:05:58 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0804E106564A for ; Thu, 6 Aug 2009 16:05:58 +0000 (UTC) (envelope-from jamie.ostrowski@gmail.com) Received: from mail-fx0-f224.google.com (mail-fx0-f224.google.com [209.85.220.224]) by mx1.freebsd.org (Postfix) with ESMTP id 81B6D8FC15 for ; Thu, 6 Aug 2009 16:05:57 +0000 (UTC) Received: by fxm24 with SMTP id 24so986339fxm.36 for ; Thu, 06 Aug 2009 09:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=Tq1qtaZ0c9L/0/+uPgtl+vAdGDnIINt50d3qjw0Cp2w=; b=s38hRI9HK1xX2PLBmOpaNc92fGE/mVvn37oqO90kMvNowi/4sRzLDQguGqk0J54sVd LAyOMpxTGid/yOkw0IGdsSkEKqIHgrLjp42L31yuqwdukRsR7dnsPYW1dtYxf/tP51kr G1/+rkqRNSRufNKDWz9XGevQJhQc+NMPO//H8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=PoDSc3UcrBlLPrUw2PNMnUtWF47vhxPyyOohihaKJ8/GPUvKnHMRRgSlwqnrkhh31s BjEYIlrgkrMv7Ce3tiDjBegUM5kb3kD55Sd4MFgh1oxQ9w2noMEnpH3Zn0xmYIRuGily 6/fK6UHT/Z5lQ0tgXS2TMCVIuZ3IpLi7C0oWI= MIME-Version: 1.0 Received: by 10.204.120.16 with SMTP id b16mr2201053bkr.64.1249573197903; Thu, 06 Aug 2009 08:39:57 -0700 (PDT) Date: Thu, 6 Aug 2009 10:39:57 -0500 Message-ID: <29ae62fc0908060839u430fb073hf5b9f7837f9bc8b6@mail.gmail.com> From: Jamie Ostrowski To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Extracting block pointer list -- ffsinfo? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Aug 2009 16:05:58 -0000 I'm a student studying filesystems, and I'd like to find a way to list the block pointers in an inode. Are there any tools in FreeBSD that can do that? For example, I've tried the following command, but I'm not seeing a list of the block pointers: ffsinfo -i 2 -l 256 /dev/da0s1f ===== START UFS2 INODE DUMP ===== # 0@28202200: Inode 0x00000002 mode u_int16_t 040755 nlink int16_t 0x0012 uid u_int32_t 0x00000000 gid u_int32_t 0x00000000 blksize u_int32_t 0x00000000 size u_int64_t 0x0000000000000200 blocks u_int64_t 0x0000000000000004 atime ufs_time_t 1249545661 mtime ufs_time_t 1243012475 ctime ufs_time_t 1243012475 birthtime ufs_time_t 1230822454 mtimensec int32_t 0x00000000 atimensec int32_t 0x00000000 ctimensec int32_t 0x00000000 birthnsec int32_t 0x00000000 gen int32_t 0x50291104 kernflags u_int32_t 0x00000000 flags u_int32_t 0x00000000 extsize int32_t 0x00000000 db ufs2_daddr_t[0] 0x bc8 ===== END UFS2 INODE DUMP ===== From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 05:44:14 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F156106566B; Fri, 7 Aug 2009 05:44:14 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 4C8488FC17; Fri, 7 Aug 2009 05:44:12 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2837D45CA0; Fri, 7 Aug 2009 07:44:11 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 08CD6456B1; Fri, 7 Aug 2009 07:44:05 +0200 (CEST) Date: Fri, 7 Aug 2009 07:44:31 +0200 From: Pawel Jakub Dawidek To: Juergen Unger Message-ID: <20090807054431.GA2500@garage.freebsd.pl> References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <20090802092714.GA5813@jpru.ffm.jpru.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YZ5djTAD1cGYuMQK" Content-Disposition: inline In-Reply-To: <20090802092714.GA5813@jpru.ffm.jpru.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, "O. Hartmann" , freebsd-current@FreeBSD.org, spambox@haruhiism.net, Andriy Gapon Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 05:44:14 -0000 --YZ5djTAD1cGYuMQK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 02, 2009 at 11:27:14AM +0200, Juergen Unger wrote: > I tried the patch, restarted the whole thing yesterday morning > and after less then 24 hours and approximately 3215 zfs-receive > jobs it do not crashes anymore, but the last started zfs-receive > jobs is hanging, cannot be killed, even not with -9. Even other > zfs commands are hanging and cannot be killed, while zpool commands > seems to be not affected. Unfortunatel I wasn't able to reproduce it. The good news is that something was just committed to OpenSolaris which might fix it (see bug 6868108 on http://bugs.opensolaris.org). The bad news is that the fix is too complex to backport to our ZFS version... --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --YZ5djTAD1cGYuMQK Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKe78/ForvXbEpPzQRApJJAKCqp/QJAntfveRu1eDA0yILQRaghgCdHdCy 9qLwTrmuxfVAHN+y8X0WGOE= =NkZ6 -----END PGP SIGNATURE----- --YZ5djTAD1cGYuMQK-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 07:37:19 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30590106566C for ; Fri, 7 Aug 2009 07:37:19 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 6A8E68FC19 for ; Fri, 7 Aug 2009 07:37:17 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 823BA45CD9; Fri, 7 Aug 2009 09:37:16 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 819C5456B1; Fri, 7 Aug 2009 09:37:11 +0200 (CEST) Date: Fri, 7 Aug 2009 09:37:38 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090807073738.GA1607@garage.freebsd.pl> References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mYCpIKhGyMATD0i+" Content-Disposition: inline In-Reply-To: <4A798A12.4070408@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 07:37:19 -0000 --mYCpIKhGyMATD0i+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 05, 2009 at 09:33:06AM -0400, Boris Kochergin wrote: > Fatal trap 12: page fault while in kernel mode > fault virtual address =3D 0xffffffffffffffe9 > fault code =3D supervisor read data, page not present > instruction pointer =3D 0x20:0xffffffff8103a9e7 > stack pointer =3D 0x28:0xffffff8077f26430 > frame pointer =3D 0x28:0xffffff8077f26500 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 972 (cp) [...] > /usr/src/sys/amd64/amd64/trap.c:494 > #11 0xffffffff80854d73 in calltrap () at=20 > /usr/src/sys/amd64/amd64/exception.S:224 > #12 0xffffffff8103a9e7 in arc_evict (state=3DVariable "state" is not=20 > available. > ) at=20 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /arc.c:1489 Could you tell me what do you have at this line in your source? I don't think you use HEAD... What exact FreeBSD version are you using? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --mYCpIKhGyMATD0i+ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKe9nCForvXbEpPzQRAgnhAJ9fn8NUoDeQRbc7ycsehKViUchwbwCg4TyH /y2Mwzm1ZeE5ZanPVL6HE08= =k+cc -----END PGP SIGNATURE----- --mYCpIKhGyMATD0i+-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 07:43:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49CD71065678 for ; Fri, 7 Aug 2009 07:43:41 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 8E8158FC2A for ; Fri, 7 Aug 2009 07:43:40 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E002845C9F; Fri, 7 Aug 2009 09:43:38 +0200 (CEST) Received: from localhost (pjd-w.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id CD88145C8A; Fri, 7 Aug 2009 09:43:33 +0200 (CEST) Date: Fri, 7 Aug 2009 09:44:00 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090807074400.GB1607@garage.freebsd.pl> References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7ZAtKRhVyVSsbBD2" Content-Disposition: inline In-Reply-To: <20090807073738.GA1607@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 07:43:41 -0000 --7ZAtKRhVyVSsbBD2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 07, 2009 at 09:37:38AM +0200, Pawel Jakub Dawidek wrote: > On Wed, Aug 05, 2009 at 09:33:06AM -0400, Boris Kochergin wrote: > > Fatal trap 12: page fault while in kernel mode > > fault virtual address =3D 0xffffffffffffffe9 > > fault code =3D supervisor read data, page not present > > instruction pointer =3D 0x20:0xffffffff8103a9e7 > > stack pointer =3D 0x28:0xffffff8077f26430 > > frame pointer =3D 0x28:0xffffff8077f26500 > > code segment =3D base 0x0, limit 0xfffff, type 0x1b > > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > > current process =3D 972 (cp) > [...] > > /usr/src/sys/amd64/amd64/trap.c:494 > > #11 0xffffffff80854d73 in calltrap () at=20 > > /usr/src/sys/amd64/amd64/exception.S:224 > > #12 0xffffffff8103a9e7 in arc_evict (state=3DVariable "state" is not=20 > > available. > > ) at=20 > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z= fs/arc.c:1489 >=20 > Could you tell me what do you have at this line in your source? I don't > think you use HEAD... What exact FreeBSD version are you using? You already gave version number in your first mail, sorry about that. 8.0-BETA2 should be very close to HEAD (or it actually was HEAD), so I guess this is the code we are looking at: 1488: /* "lookahead" for better eviction candidate */ 1489: if (recycle && ab->b_size !=3D bytes && 1490: ab_prev && ab_prev->b_size =3D=3D bytes) 1491: continue; If 'ab' is corrupted it should panic earlier, so it seems ab_prev is corrupted, can you see what it points to in gdb? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --7ZAtKRhVyVSsbBD2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKe9tAForvXbEpPzQRAoYQAJ93SwrZRKfO+oIxSIHSGoXPC4MSvQCeKp0b 4R79PwGqH+NIW3oGUYhD1mM= =Fs3k -----END PGP SIGNATURE----- --7ZAtKRhVyVSsbBD2-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 12:44:43 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6569C1065680 for ; Fri, 7 Aug 2009 12:44:43 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1EC858FC24 for ; Fri, 7 Aug 2009 12:44:43 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9874946B03; Fri, 7 Aug 2009 08:44:42 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id DA9BE8A0AB; Fri, 7 Aug 2009 08:44:41 -0400 (EDT) From: John Baldwin To: freebsd-fs@freebsd.org Date: Fri, 7 Aug 2009 08:29:54 -0400 User-Agent: KMail/1.9.7 References: <8E9591D8BCB72D4C8DE0884D9A2932DC35BD34C3@ITS-HCWNEM03.ds.Vanderbilt.edu> In-Reply-To: <8E9591D8BCB72D4C8DE0884D9A2932DC35BD34C3@ITS-HCWNEM03.ds.Vanderbilt.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200908070829.54571.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 07 Aug 2009 08:44:41 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Re: UFS Filesystem issues, and the loss of my hair... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 12:44:44 -0000 On Thursday 06 August 2009 9:51:04 am Hearn, Trevor wrote: > First off, let me state that I love FreeBSD. I've used it for years, and have not had any major problems with it... Until now. > > As you can tell, I work for a major university. I setup a large storage array to hold data for a project they have here. No great shakes, just some standard files and such. The fun started when I started loading users onto the system, and they started using it... Isn't that always the case? Now, I get ufs_dirbad errors, and the system hard locks. This isn't the worst thing that could happen, but when you're talking about file partitions the size that I am using, the fsck takes FOREVER. Somewhere on the order of 1.5 hours. During that time, I am bringing the individual shares/partitions online, but the users suffer. I've asked about this before, in a different forum, but got no usable information that I could see. So, here goes... > > The system is as such. A dell 2950 1U server, with a Qlogic Fibre Channel card. It is connected to two Promise Array chassis, 610 series, each with 16 drives. Each chassis is running RAID 6, which gives me about 12.73tb of storage per chassis. From there, the logical drives are sliced up into smaller partitions. At most, I have a 3.6tb partition. The smallest is a 100gig partition. > > Filesystem Size Used Avail Capacity Mounted on > /dev/mfid0s1a 197G 10G 170G 6% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/da0p1 1.8T 1.5T 130G 92% /slice1 > /dev/da0p5 2.7T 1.8T 661G 74% /slice2 > /dev/da0p9 250G 21G 209G 9% /slice3 > /dev/da1p3 103G 12G 83G 12% /slice4 > /dev/da1p4 205G 54G 135G 29% /slice5 > /dev/da1p5 103G 7.3G 87G 8% /slice6 > /dev/da1p6 103G 22G 72G 23% /slice7 > etc... > > I had to use GPT to setup the partitions, and they are using UFS2 for the filesystem. Now... If that's not fun enough... I have TWO of these creatures, which RSYNC every 4 hours. The secondary system is across campus, and sits idle 99% of the time. Every 4 hours, in a stepped schedule, the primary array syncs to the secondary array. If the primary goes down, I FSCK, and any files that are fried, I bring back across from the secondary and replace them. This has worked OK for a while, but now I am getting Kernel Panics on a regular basis. I've been told to migrate to a different filesystem, but my options are ZFS and using GJOURNAL with UFS, from what I can tell. I need something repeatable, simple, and I need something robust. I have NO idea why I keep getting errors like this, but I imagine it's a cascading effect of other hangs that have caused more corruption. > > I'd buy a fella, or gal, a cup of coffee and a pop-tart if they could help a brother out. I have checked out this link: > http://phaq.phunsites.net/2007/07/01/ufs_dirbad-panic-with-mangled-entries-in-ufs/ > and decided that I need to give this a shot after hours, but being the kinda guy I am, I need to make sure I am covering all of my bases. Are you seeing ufs_dirbad panics? Specifically, can you capture the messages on the console when the machine panics? -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 13:46:14 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC7B9106566C for ; Fri, 7 Aug 2009 13:46:14 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 889048FC20 for ; Fri, 7 Aug 2009 13:46:14 +0000 (UTC) Received: (qmail 48517 invoked from network); 7 Aug 2009 13:46:13 -0000 Received: from unknown (HELO ?10.0.0.135?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 7 Aug 2009 13:46:13 -0000 Message-ID: <4A7C3002.8000003@acm.poly.edu> Date: Fri, 07 Aug 2009 09:45:38 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> In-Reply-To: <20090807074400.GB1607@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 13:46:15 -0000 Pawel Jakub Dawidek wrote: > On Fri, Aug 07, 2009 at 09:37:38AM +0200, Pawel Jakub Dawidek wrote: > >> On Wed, Aug 05, 2009 at 09:33:06AM -0400, Boris Kochergin wrote: >> >>> Fatal trap 12: page fault while in kernel mode >>> fault virtual address = 0xffffffffffffffe9 >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff8103a9e7 >>> stack pointer = 0x28:0xffffff8077f26430 >>> frame pointer = 0x28:0xffffff8077f26500 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = interrupt enabled, resume, IOPL = 0 >>> current process = 972 (cp) >>> >> [...] >> >>> /usr/src/sys/amd64/amd64/trap.c:494 >>> #11 0xffffffff80854d73 in calltrap () at >>> /usr/src/sys/amd64/amd64/exception.S:224 >>> #12 0xffffffff8103a9e7 in arc_evict (state=Variable "state" is not >>> available. >>> ) at >>> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1489 >>> >> Could you tell me what do you have at this line in your source? I don't >> think you use HEAD... What exact FreeBSD version are you using? >> > > You already gave version number in your first mail, sorry about that. > 8.0-BETA2 should be very close to HEAD (or it actually was HEAD), so I > guess this is the code we are looking at: > > 1488: /* "lookahead" for better eviction candidate */ > 1489: if (recycle && ab->b_size != bytes && > 1490: ab_prev && ab_prev->b_size == bytes) > 1491: continue; > > If 'ab' is corrupted it should panic earlier, so it seems ab_prev is > corrupted, can you see what it points to in gdb? > > Yeah, that's what the code looks like. For convenience, I've put the source tree the system was built using up at: http://acm.poly.edu/~spawk/src/ Maybe my kgdb chops aren't up to par, but I can't seem to see what ab_prev points to: (kgdb) up #12 0xffffffff8103a9e7 in arc_evict (state=Variable "state" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1489 1489 if (recycle && ab->b_size != bytes && Current language: auto; currently c (kgdb) list 1484 LBOLT - ab->b_arc_access < arc_min_prefetch_lifespan)) { 1485 skipped++; 1486 continue; 1487 } 1488 /* "lookahead" for better eviction candidate */ 1489 if (recycle && ab->b_size != bytes && 1490 ab_prev && ab_prev->b_size == bytes) 1491 continue; 1492 hash_lock = HDR_LOCK(ab); 1493 have_lock = MUTEX_HELD(hash_lock); (kgdb) print ab $13 = (arc_buf_hdr_t *) 0xffffff0003ebc410 (kgdb) print ab->b_size $14 = 1 (kgdb) print bytes $15 = 16384 (kgdb) print ab_prev No symbol "ab_prev" in current context. -Boris From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 19:13:16 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC1EC10656A6 for ; Fri, 7 Aug 2009 19:13:16 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 37A5E8FC16 for ; Fri, 7 Aug 2009 19:13:15 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 32E6445DD8; Fri, 7 Aug 2009 21:13:14 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 213FE45CD9; Fri, 7 Aug 2009 21:13:09 +0200 (CEST) Date: Fri, 7 Aug 2009 21:13:34 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090807191334.GA1814@garage.freebsd.pl> References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> <4A7C3002.8000003@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <4A7C3002.8000003@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 19:13:17 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 07, 2009 at 09:45:38AM -0400, Boris Kochergin wrote: > Maybe my kgdb chops aren't up to par, but I can't seem to see what=20 > ab_prev points to: >=20 > (kgdb) up > #12 0xffffffff8103a9e7 in arc_evict (state=3DVariable "state" is not=20 > available. > ) at=20 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /arc.c:1489 > 1489 if (recycle && ab->b_size !=3D bytes && > Current language: auto; currently c > (kgdb) list > 1484 LBOLT - ab->b_arc_access <=20 > arc_min_prefetch_lifespan)) { > 1485 skipped++; > 1486 continue; > 1487 } > 1488 /* "lookahead" for better eviction candidate */ > 1489 if (recycle && ab->b_size !=3D bytes && > 1490 ab_prev && ab_prev->b_size =3D=3D bytes) > 1491 continue; > 1492 hash_lock =3D HDR_LOCK(ab); > 1493 have_lock =3D MUTEX_HELD(hash_lock); > (kgdb) print ab > $13 =3D (arc_buf_hdr_t *) 0xffffff0003ebc410 > (kgdb) print ab->b_size > $14 =3D 1 > (kgdb) print bytes > $15 =3D 16384 > (kgdb) print ab_prev > No symbol "ab_prev" in current context. Yeah, that's strange indeed. Could you try: print ab->b_arc_node.list_prev print ab->b_arc_node.list_next --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKfHzeForvXbEpPzQRAifkAJsE/r1XUHmK6YlpetGXJyKwagTG7ACg5QbI 4l19ImcEfJgtsDs45NPpjCM= =DI5N -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 19:35:10 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F034C1065672 for ; Fri, 7 Aug 2009 19:35:10 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id B79688FC1A for ; Fri, 7 Aug 2009 19:35:10 +0000 (UTC) Received: (qmail 55256 invoked from network); 7 Aug 2009 19:35:09 -0000 Received: from unknown (HELO ?10.0.0.135?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 7 Aug 2009 19:35:09 -0000 Message-ID: <4A7C81CA.2040303@acm.poly.edu> Date: Fri, 07 Aug 2009 15:34:34 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> <4A7C3002.8000003@acm.poly.edu> <20090807191334.GA1814@garage.freebsd.pl> In-Reply-To: <20090807191334.GA1814@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 19:35:11 -0000 Pawel Jakub Dawidek wrote: > On Fri, Aug 07, 2009 at 09:45:38AM -0400, Boris Kochergin wrote: > >> Maybe my kgdb chops aren't up to par, but I can't seem to see what >> ab_prev points to: >> >> (kgdb) up >> #12 0xffffffff8103a9e7 in arc_evict (state=Variable "state" is not >> available. >> ) at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1489 >> 1489 if (recycle && ab->b_size != bytes && >> Current language: auto; currently c >> (kgdb) list >> 1484 LBOLT - ab->b_arc_access < >> arc_min_prefetch_lifespan)) { >> 1485 skipped++; >> 1486 continue; >> 1487 } >> 1488 /* "lookahead" for better eviction candidate */ >> 1489 if (recycle && ab->b_size != bytes && >> 1490 ab_prev && ab_prev->b_size == bytes) >> 1491 continue; >> 1492 hash_lock = HDR_LOCK(ab); >> 1493 have_lock = MUTEX_HELD(hash_lock); >> (kgdb) print ab >> $13 = (arc_buf_hdr_t *) 0xffffff0003ebc410 >> (kgdb) print ab->b_size >> $14 = 1 >> (kgdb) print bytes >> $15 = 16384 >> (kgdb) print ab_prev >> No symbol "ab_prev" in current context. >> > > Yeah, that's strange indeed. Could you try: > > print ab->b_arc_node.list_prev > print ab->b_arc_node.list_next > > (kgdb) print ab->b_arc_node.list_prev $1 = (struct list_node *) 0x1 (kgdb) print ab->b_arc_node.list_next $2 = (struct list_node *) 0xffffffff811064f0 -Boris From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 19:38:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB2AD106564A for ; Fri, 7 Aug 2009 19:38:27 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 2AFFC8FC23 for ; Fri, 7 Aug 2009 19:38:27 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3C14945685; Fri, 7 Aug 2009 21:38:24 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 2FCD745683; Fri, 7 Aug 2009 21:38:17 +0200 (CEST) Date: Fri, 7 Aug 2009 21:38:42 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090807193842.GA2487@garage.freebsd.pl> References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> <4A7C3002.8000003@acm.poly.edu> <20090807191334.GA1814@garage.freebsd.pl> <4A7C81CA.2040303@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="d6Gm4EdcadzBjdND" Content-Disposition: inline In-Reply-To: <4A7C81CA.2040303@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 19:38:28 -0000 --d6Gm4EdcadzBjdND Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 07, 2009 at 03:34:34PM -0400, Boris Kochergin wrote: > Pawel Jakub Dawidek wrote: > >Yeah, that's strange indeed. Could you try: > > > > print ab->b_arc_node.list_prev > > print ab->b_arc_node.list_next > > > > =20 > (kgdb) print ab->b_arc_node.list_prev > $1 =3D (struct list_node *) 0x1 Yeah, list_prev is corrupted. If it panics on you everytime, I could send you a patch which will try to catch where the corruption occurs. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --d6Gm4EdcadzBjdND Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKfILCForvXbEpPzQRAkINAKDOJsmgGMUA7NBujYlPbgck/pnD1gCgyEJ5 1vsGpkDUu9HK38BLJ53bdUI= =U9+p -----END PGP SIGNATURE----- --d6Gm4EdcadzBjdND-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 20:00:43 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6EE3F1065677 for ; Fri, 7 Aug 2009 20:00:43 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 092178FC2B for ; Fri, 7 Aug 2009 20:00:42 +0000 (UTC) Received: (qmail 56024 invoked from network); 7 Aug 2009 20:00:40 -0000 Received: from unknown (HELO ?10.0.0.135?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 7 Aug 2009 20:00:40 -0000 Message-ID: <4A7C87C5.1070608@acm.poly.edu> Date: Fri, 07 Aug 2009 16:00:05 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4A78AA71.9050107@acm.poly.edu> <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> <4A7C3002.8000003@acm.poly.edu> <20090807191334.GA1814@garage.freebsd.pl> <4A7C81CA.2040303@acm.poly.edu> <20090807193842.GA2487@garage.freebsd.pl> In-Reply-To: <20090807193842.GA2487@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 20:00:43 -0000 Pawel Jakub Dawidek wrote: > On Fri, Aug 07, 2009 at 03:34:34PM -0400, Boris Kochergin wrote: > >> Pawel Jakub Dawidek wrote: >> >>> Yeah, that's strange indeed. Could you try: >>> >>> print ab->b_arc_node.list_prev >>> print ab->b_arc_node.list_next >>> >>> >>> >> (kgdb) print ab->b_arc_node.list_prev >> $1 = (struct list_node *) 0x1 >> > > Yeah, list_prev is corrupted. If it panics on you everytime, I could > send you a patch which will try to catch where the corruption occurs. > > I eventually get the arc_evict panic every time I successfully manage to mount the filesystem, but it usually panics (with the other backtrace) as soon as I try to mount it, or mount just hangs. I'll gladly try the patch, though--the data on the array is important to me. Thanks. -Boris From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 20:27:39 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1591E106566B for ; Fri, 7 Aug 2009 20:27:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 523448FC1D for ; Fri, 7 Aug 2009 20:27:37 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 0338C45E82; Fri, 7 Aug 2009 22:27:35 +0200 (CEST) Received: from localhost (chello087206049004.chello.pl [87.206.49.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D07E545E6F; Fri, 7 Aug 2009 22:27:30 +0200 (CEST) Date: Fri, 7 Aug 2009 22:27:56 +0200 From: Pawel Jakub Dawidek To: Boris Kochergin Message-ID: <20090807202756.GB2487@garage.freebsd.pl> References: <4A78AFB2.10103@acm.poly.edu> <20090805115621.GG1784@garage.freebsd.pl> <4A798A12.4070408@acm.poly.edu> <20090807073738.GA1607@garage.freebsd.pl> <20090807074400.GB1607@garage.freebsd.pl> <4A7C3002.8000003@acm.poly.edu> <20090807191334.GA1814@garage.freebsd.pl> <4A7C81CA.2040303@acm.poly.edu> <20090807193842.GA2487@garage.freebsd.pl> <4A7C87C5.1070608@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TakKZr9L6Hm6aLOc" Content-Disposition: inline In-Reply-To: <4A7C87C5.1070608@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RAID-Z panic on vdev failure + subsequent panics and hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 20:27:39 -0000 --TakKZr9L6Hm6aLOc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 07, 2009 at 04:00:05PM -0400, Boris Kochergin wrote: > Pawel Jakub Dawidek wrote: > >On Fri, Aug 07, 2009 at 03:34:34PM -0400, Boris Kochergin wrote: > > =20 > >>Pawel Jakub Dawidek wrote: > >> =20 > >>>Yeah, that's strange indeed. Could you try: > >>> > >>> print ab->b_arc_node.list_prev > >>> print ab->b_arc_node.list_next > >>> > >>>=20 > >>> =20 > >>(kgdb) print ab->b_arc_node.list_prev > >>$1 =3D (struct list_node *) 0x1 > >> =20 > > > >Yeah, list_prev is corrupted. If it panics on you everytime, I could > >send you a patch which will try to catch where the corruption occurs. > > > > =20 > I eventually get the arc_evict panic every time I successfully manage to= =20 > mount the filesystem, but it usually panics (with the other backtrace)=20 > as soon as I try to mount it, or mount just hangs. I'll gladly try the=20 > patch, though--the data on the array is important to me. Thanks. To get the data from there you could also try to 'zfs send' it without mounting the dataset at all (just in case). --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --TakKZr9L6Hm6aLOc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKfI5MForvXbEpPzQRAjCvAKDMyoSmq4mjNswlFcbFAg02aIJ0MQCdE8nN d1i61OoGjZlpcojbZTFl3Xs= =e+gh -----END PGP SIGNATURE----- --TakKZr9L6Hm6aLOc-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 7 20:46:07 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9AD0106564A for ; Fri, 7 Aug 2009 20:46:07 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.227]) by mx1.freebsd.org (Postfix) with ESMTP id 869D08FC1F for ; Fri, 7 Aug 2009 20:46:07 +0000 (UTC) Received: by rv-out-0506.google.com with SMTP id f9so492743rvb.43 for ; Fri, 07 Aug 2009 13:46:07 -0700 (PDT) Received: by 10.140.225.19 with SMTP id x19mr24282rvg.94.1249676624686; Fri, 07 Aug 2009 13:23:44 -0700 (PDT) Received: from mattintosh.spry.com (isaid.donotdelete.com [64.79.222.10]) by mx.google.com with ESMTPS id g31sm8714910rvb.46.2009.08.07.13.23.43 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 07 Aug 2009 13:23:43 -0700 (PDT) Message-Id: <04A4A2CB-828B-46BF-A2B6-50B64F06E96E@spry.com> From: Matt Simerson To: freebsd-fs@freebsd.org In-Reply-To: <8E9591D8BCB72D4C8DE0884D9A2932DC35BD34C3@ITS-HCWNEM03.ds.Vanderbilt.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Fri, 7 Aug 2009 13:23:41 -0700 References: <8E9591D8BCB72D4C8DE0884D9A2932DC35BD34C3@ITS-HCWNEM03.ds.Vanderbilt.edu> X-Mailer: Apple Mail (2.936) Cc: "Hearn, Trevor" Subject: Re: UFS Filesystem issues, and the loss of my hair... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2009 20:46:07 -0000 On Aug 6, 2009, at 6:51 AM, Hearn, Trevor wrote: > First off, let me state that I love FreeBSD. I've used it for years, > and have not had any major problems with it... Until now. > > As you can tell, I work for a major university. I setup a large > storage array to hold data for a project they have here. No great > shakes, just some standard files and such. > > I'd buy a fella, or gal, a cup of coffee and a pop-tart if they > could help a brother out. I have checked out this link: > http://phaq.phunsites.net/2007/07/01/ufs_dirbad-panic-with-mangled-entries-in-ufs/ > and decided that I need to give this a shot after hours, but being > the kinda guy I am, I need to make sure I am covering all of my bases. > > Anyone got any ideas? > > Thanks! Have you given any consideration to ZFS? With ZFS there's no reason to have all those slices. Just stripe the two RAID 6 arrays together and have a single 26TB zpool. No GPT or UFS to mess with. Just point ZFS at the raw disks and off you go. I'm doing that with Areca 1231ML controllers in boxes with 24 disks each. The two 12 channel RAID cards each present a RAID volume to the OS and zpool stripes them together. One of the more useful features of ZFS is file system compression. You may find that with file system compression, you can get by with 13TB of storage. Then you have one RAID 6 array as the data store and the 2nd array for backups on each machine. With ZFS, you can send snapshots of the data partition to the backup every hour, or even every minute without any appreciable impact. back01# zfs get compression back01/var NAME PROPERTY VALUE SOURCE back01/var compression gzip local back01# zfs get compressratio back01/var NAME PROPERTY VALUE SOURCE back01/var compressratio 2.16x - I'm using gzip compression and I fit over twice as much data on the filesystem as I'd otherwise be getting. You can get more aggressive with gzip-9 if you need. You could use your backup server as a proof-of-concept. Install FreeBSD 8-BETA2 amd64 on it. Unmount the existing GPT partitions, wipe the MBR clean using dd, and create a zpool on just one of the RAID 6 volumes. Set ZFS compression=gzip on your filesystem and use rsync to copy all the files from your 'primary' server. I suspect you'll find that you have ample storage. Then you can create another zpool on that same box using the other RAID 6 volume for backups. You can experiment there with zfs send/receive, or rsnapshot, or whatever you use. Then get a subset of your users to start testing on it and see how it fares. I suspect you'll be quite pleased. If it works out wonderfully, you can rebuild the other GPT/UFS system on ZFS as well. Set it up with both RAID 6 volumes in one ZFS pool and start pushing your backups from the primary server to it. Once successfully backed up, you can add the 2nd RAID 6 volume on the primary server into the storage pool to double it's disk space. Matt From owner-freebsd-fs@FreeBSD.ORG Sat Aug 8 02:37:35 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D89CD106566B for ; Sat, 8 Aug 2009 02:37:35 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from fallbackmx07.syd.optusnet.com.au (fallbackmx07.syd.optusnet.com.au [211.29.132.9]) by mx1.freebsd.org (Postfix) with ESMTP id 644EC8FC1A for ; Sat, 8 Aug 2009 02:37:35 +0000 (UTC) Received: from mail15.syd.optusnet.com.au (mail15.syd.optusnet.com.au [211.29.132.196]) by fallbackmx07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n780WRMK001754 for ; Sat, 8 Aug 2009 10:32:27 +1000 Received: from server.vk2pj.dyndns.org (c122-106-201-175.belrs3.nsw.optusnet.com.au [122.106.201.175]) by mail15.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n780WJi6025381 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Aug 2009 10:32:25 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n780WJN0056594; Sat, 8 Aug 2009 10:32:19 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n780WI0D056593; Sat, 8 Aug 2009 10:32:18 +1000 (EST) (envelope-from peter) Date: Sat, 8 Aug 2009 10:32:18 +1000 From: Peter Jeremy To: Jamie Ostrowski Message-ID: <20090808003218.GA56430@server.vk2pj.dyndns.org> References: <29ae62fc0908060839u430fb073hf5b9f7837f9bc8b6@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pf9I7BMVVzbSWLtt" Content-Disposition: inline In-Reply-To: <29ae62fc0908060839u430fb073hf5b9f7837f9bc8b6@mail.gmail.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-fs@freebsd.org Subject: Re: Extracting block pointer list -- ffsinfo? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2009 02:37:36 -0000 --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Aug-06 10:39:57 -0500, Jamie Ostrowski = wrote: > I'm a student studying filesystems, and I'd like to find a way to list >the block pointers in an inode. Are there any tools in FreeBSD that can do >that? ffsinfo(8) or fsdb(8) >db ufs2_daddr_t[0] 0x bc8 It might not be obvious without looking in the source code (this particular output comes from /usr/src/sbin/growfs/debug.c) but that actually _is_ the list of blocks. It is more obvious if you run ffsinfo on a larger file. --=20 Peter Jeremy --pf9I7BMVVzbSWLtt Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkp8x5IACgkQ/opHv/APuIcE7wCfUN8+IL0cSoKmx6tWPW/d/7wY EOoAoLLORZ3G+zRwy0QeDRmoyrKkVUgF =WCAg -----END PGP SIGNATURE----- --pf9I7BMVVzbSWLtt--