From owner-freebsd-fs@FreeBSD.ORG Thu Jul 30 12:55:44 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B725A10656C4; Thu, 30 Jul 2009 12:55:44 +0000 (UTC) (envelope-from mdounin@mdounin.ru) Received: from mdounin.cust.ramtel.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mx1.freebsd.org (Postfix) with ESMTP id 7006E8FC1D; Thu, 30 Jul 2009 12:55:44 +0000 (UTC) (envelope-from mdounin@mdounin.ru) Received: from mdounin.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mdounin.cust.ramtel.ru (Postfix) with ESMTP id F40FD1700F; Thu, 30 Jul 2009 16:55:42 +0400 (MSD) Date: Thu, 30 Jul 2009 16:55:42 +0400 From: Maxim Dounin To: Pawel Jakub Dawidek Message-ID: <20090730125542.GI8794@mdounin.ru> References: <20090730013857.GB8794@mdounin.ru> <20090730055001.GB2130@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090730055001.GB2130@garage.freebsd.pl> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: another zfs panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jul 2009 12:55:45 -0000 Hello! On Thu, Jul 30, 2009 at 07:50:01AM +0200, Pawel Jakub Dawidek wrote: > On Thu, Jul 30, 2009 at 05:38:57AM +0400, Maxim Dounin wrote: > > Hello! > > > > Here is zfs panic I'm able to reproduce by running an scp from > > remote machine to zfs volume and 3 parallel untars of ports tree > > in cycle. Not sure that everything is required, but the above > > workload triggers panic in several hours. > > > > This is on fresh current with GENERIC kernel: > > > > panic: sx_xlock() of destroyed sx @ > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_rlock.c:535 > > cpuid = 6 > > KDB: enter: panic > > [thread pid 36 tid 100071 ] > > Stopped at kdb_enter+0x3d: movq $0,0x68a040(%rip) > > db> bt > > Tracing pid 36 tid 100071 td 0xffffff00040f3720 > > kdb_enter() at kdb_enter+0x3d > > panic() at panic+0x17b > > _sx_xlock() at _sx_xlock+0xfc > > zfs_range_unlock() at zfs_range_unlock+0x38 > > zfs_get_data() at zfs_get_data+0xc1 > > zil_commit() at zil_commit+0x532 > > zfs_sync() at zfs_sync+0xa6 > > sync_fsync() at sync_fsync+0x13a > > sync_vnode() at sync_vnode+0x157 > > sched_sync() at sched_sync+0x1d1 > > fork_exit() at fork_exit+0x12a > > fork_trampoline() at fork_trampoline+0xe > > --- trap 0, rip = 0, rsp = 0xffffff80e7ee3d30, rbp = 0 --- > > > > Machine is otherwise idle. The only zfs-related tuning applied is > > compression=gzip-9. > > > > Please let me know if you want me to test some patches. > > The kernel syncer tries to sync vnode which has its znode already > destroyed. There is one place (that we know of) where vrecycle() is > missing. Could you try this patch: > > http://people.freebsd.org/~pjd/patches/zfs_vnops.c.2.patch Still here with patch applied: panic: sx_xlock() of destroyed sx @ /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_rlock.c:535 cpuid = 3 KDB: enter: panic [thread pid 37 tid 100072 ] Stopped at kdb_enter+0x3d: movq $0,0x68a040(%rip) db> bt Tracing pid 37 tid 100072 td 0xffffff00040f3390 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b _sx_xlock() at _sx_xlock+0xfc zfs_range_unlock() at zfs_range_unlock+0x38 zfs_get_data() at zfs_get_data+0xc1 zil_commit() at zil_commit+0x532 zfs_sync() at zfs_sync+0xa6 sync_fsync() at sync_fsync+0x13a sync_vnode() at sync_vnode+0x157 sched_sync() at sched_sync+0x1d1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80e7ee8d30, rbp = 0 --- Maxim Dounin