From owner-freebsd-fs@FreeBSD.ORG Thu May 10 03:52:58 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 10B0F16A402 for ; Thu, 10 May 2007 03:52:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay02.kiev.sovam.com (relay02.kiev.sovam.com [62.64.120.197]) by mx1.freebsd.org (Postfix) with ESMTP id 9850213C448 for ; Thu, 10 May 2007 03:52:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay02.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1Hlzi3-0001AK-AE for freebsd-fs@freebsd.org; Thu, 10 May 2007 06:52:56 +0300 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id l4A3qmmW001692 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 10 May 2007 06:52:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1) with ESMTP id l4A3qm5j098783; Thu, 10 May 2007 06:52:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1/Submit) id l4A3qlmo098782; Thu, 10 May 2007 06:52:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 10 May 2007 06:52:47 +0300 From: Kostik Belousov To: Cristian KLEIN Message-ID: <20070510035247.GR83173@deviant.kiev.zoral.com.ua> References: <59558.86.125.188.48.1177802342.squirrel@intranet.utcluj.ro> <4637A640.6050700@freebsd.org> <46390F78.5080206@net.utcluj.ro> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xSu31lw3TgkWXnjh" Content-Disposition: inline In-Reply-To: <46390F78.5080206@net.utcluj.ro> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.7, clamav-milter version 0.88.7 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,SPF_NEUTRAL autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on fw.zoral.com.ua X-Scanner-Signature: 84ad75c442e4d00ba4f39fd4f5f00a84 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 1031 [May 09 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: freebsd-fs@freebsd.org Subject: Re: panic: softdep_setup_inomapdep: found inode already exists in 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 May 2007 03:52:58 -0000 --xSu31lw3TgkWXnjh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, May 03, 2007 at 01:23:52AM +0300, Cristian KLEIN wrote: > On Mar, Mai 1, 2007 11:42 pm, Eric Anderson wrote: > > On 04/28/07 18:19, Cristian KLEIN wrote: > > > >> Hi everybody, > >> > >> > >> I am running a FreeBSD 6.2-p3, on which I am experiencing exactly the > >> same simtoms as one item of the TODO list of 6.0: > >> http://www.freebsd.org/releases/6.0R/todo.html > >> > >> > >> panic: softdep_setup_inomapdep: found inode Needs testing Tor Egge > >> Found by stress tests at > >> http://www.holm.cc/stress/log/cons138.html > >> > >> > >> Does anybody know whether this bug should have been solved in 6.2? > >> Should > >> I file a PR? > >> > > > > > > Sorry if I missed it, but were you able to provide a backtrace? If you > > can, you should compile your kernel with debugging, so at least you cou= ld > > make a little more out of the crash. See the handbook if you need help= on > > that. >=20 > Hi, >=20 > I haven't mentioned any technical details yet, as I wasn't sure whether > this issue is known or not. >=20 > First, let me tell you what I did. I wanted to better protect data on > /jail/mail/home by doing daily snapshots, saved like > /jail/mail/home/.snap/2007-04-03-03-22-02. I mounted one of these > snapshots (not the latest) in /mnt/home and rsync'd it to a server. I > might have rsync'd while taking a new snapshot. >=20 > Server started crashing randomly, either while rsync-ing, or in the > morning, during heavy load. Now I removed the snapshots and the system is > stable. >=20 > Other random information that might be useful: I am using gmirror and > jail. Disabling SMP has not effect, WITNESS didn't say anything. The > filesystem has userquotas. The filesystem stores maildir and has about > 1.6Minodes. >=20 > Config is GENERIC + SMP + QUOTA - unused hardware devices. >=20 > root# uname -a > FreeBSD bavaria.xxxxxx 6.2-RELEASE-p3 FreeBSD 6.2-RELEASE-p3 #5: Fri Apr > 27 20:01:20 EEST 2007 > cristi@bavaria.xxxxxx:/usr/obj/usr/src/sys/BAVARIA-SMP i386 >=20 > root# kgdb kernel.debug /var/crash/vmcore.3 > [GDB will not be able to debug user-mode threads: > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd". >=20 > Unread portion of the kernel message buffer: > panic: softdep_setup_inomapdep: found inode already exists > cpuid =3D 0 > KDB: enter: panic > Uptime: 5h42m12s > Dumping 1023 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 1023MB (261837 pages) 1007 991 975 959 943 927 911 895 879 863 > 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 > 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 > 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 >=20 > #0 doadump () at pcpu.h:165 > 165 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc0545e18 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :409 > #2 0xc0546149 in panic (fmt=3D0xc075660c "softdep_setup_inomapdep: found > inode already exists") > at /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc068c64a in softdep_setup_inomapdep (bp=3D0xd8c2ba68, ip=3D0x0, new= inum=3D0) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:1527 > #4 0xc067d4dd in ffs_nodealloccg (ip=3D0xc68e818c, cg=3D0, ipref=3DUnhan= dled > dwarf expression opcode 0x93 > ) at /usr/src/sys/ufs/ffs/ffs_alloc.c:1762 > #5 0xc067bb83 in ffs_hashalloc (ip=3D0xc68e818c, cg=3D0, pref=3DUnhandle= d dwarf > expression opcode 0x93 > ) at /usr/src/sys/ufs/ffs/ffs_alloc.c:1248 > #6 0xc067b232 in ffs_valloc (pvp=3D0xc6883440, mode=3D33024, cred=3D0xc5= 0d3a80, > vpp=3D0xe733d4e8) > at /usr/src/sys/ufs/ffs/ffs_alloc.c:932 > #7 0xc06a6bd7 in ufs_makeinode (mode=3D33024, dvp=3D0xc6883440, > vpp=3D0xe733d8f0, cnp=3D0xe733d904) > at /usr/src/sys/ufs/ufs/ufs_vnops.c:2220 > #8 0xc06a348f in ufs_create (ap=3D0x0) at > /usr/src/sys/ufs/ufs/ufs_vnops.c:189 > #9 0xc071e879 in VOP_CREATE_APV (vop=3D0x0, a=3D0xe733d7fc) at vnode_if.= c:204 > #10 0xc0684018 in ffs_snapshot (mp=3D0xc4e28000, > snapfile=3D0xc699c200 "/jail/mail/home/.snap/2007-04-28-03-34-31") at > vnode_if.h:111 > #11 0xc06964d7 in ffs_mount (mp=3D0xc4e28000, td=3D0xc50e1300) at > /usr/src/sys/ufs/ffs/ffs_vfsops.c:312 > #12 0xc059eb6e in vfs_domount (td=3D0xc50e1300, fstype=3D0xc4efac90 "ufs", > fspath=3D0xc4efa670 "/jail/mail/home", > fsflags=3D16842752, fsdata=3D0xc4efaca0) at > /usr/src/sys/kern/vfs_mount.c:928 > #13 0xc059e1da in vfs_donmount (td=3D0x0, fsflags=3D16842752, fsoptions= =3D0x0) > at /usr/src/sys/kern/vfs_mount.c:676 > #14 0xc05a0f84 in kernel_mount (ma=3D0xc4efac50, flags=3D0) at pcpu.h:162 > #15 0xc06966fb in ffs_cmount (ma=3D0xc4efac50, data=3D0x0, flags=3D0, > td=3D0xc50e1300) > at /usr/src/sys/ufs/ffs/ffs_vfsops.c:392 > #16 0xc059e3f0 in mount (td=3D0xc50e1300, uap=3D0xe733dd04) at > /usr/src/sys/kern/vfs_mount.c:742 > #17 0xc070b5bb in syscall (frame=3D > {tf_fs =3D 59, tf_es =3D 59, tf_ds =3D 59, tf_edi =3D -1077944004, = tf_esi =3D > -1077941276, tf_ebp =3D -1077943864, tf_isp =3D -416031388, tf_ebx =3D > -1077943824, tf_edx =3D -1, tf_ecx =3D -1077940507, tf_eax =3D 21, > tf_trapno =3D 12, tf_err =3D 2, tf_eip =3D 671885143, tf_cs =3D 51, > tf_eflags =3D 582, tf_esp =3D -1077944036, tf_ss =3D 59}) > at /usr/src/sys/i386/i386/trap.c:983 > #18 0xc06f60df in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:200 > #19 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) >=20 > Now that I have KDB on serial console I might be able to crash (or even > better, test patches) at night on that system. >=20 > Please don't hesitate to request further information. I will try to > elaborate a recipe for crashing the system. Do you have ddb backtrace for this instance of panic ? Or, for any other such panic, please, show both ddb backtrace and kgdb output of "bt full". --xSu31lw3TgkWXnjh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGQpcPC3+MBN1Mb4gRAsp4AKCGkq3gdHLpfknTFcZ8YRBsNe79JgCgnREm qNjtwJvxADqpCTGF/5emHNE= =gWzN -----END PGP SIGNATURE----- --xSu31lw3TgkWXnjh--