From owner-freebsd-stable Wed Feb 13 13:29:30 2002 Delivered-To: freebsd-stable@freebsd.org Received: from smtp-1.enteract.com (smtp-1.enteract.com [207.229.143.33]) by hub.freebsd.org (Postfix) with ESMTP id B961737B400 for ; Wed, 13 Feb 2002 13:29:20 -0800 (PST) Received: from bjorn.goddamnbastard.org (bjorn.goddamnbastard.org [216.80.6.225]) by smtp-1.enteract.com (Postfix) with SMTP id 3F4056555 for ; Wed, 13 Feb 2002 15:29:16 -0600 (CST) Received: (qmail 27171 invoked by uid 1000); 13 Feb 2002 21:29:15 -0000 Date: Wed, 13 Feb 2002 15:29:15 -0600 From: ryan beasley To: freebsd-stable@FreeBSD.org Subject: panic: softupdates related? Message-ID: <20020213212915.GB26598@bjorn.goddamnbastard.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="O5XBE6gyVG5Rl6Rj" Content-Disposition: inline User-Agent: Mutt/1.3.25i Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --O5XBE6gyVG5Rl6Rj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi! After pulling down a snapshot from earlier this week (2002.02.10 - 1030 GMT), I've noticed my notebook panicing upon issuing "shutdown -r". I have a crash dump, and I can upload it somewhere if any developers want to take a look. Below is what I believe to be some useful info extracted via gdb. (Just ask if any add'l info is required.) -----BEGIN DEBUG STUFF----- $FreeBSD src/sys/ufs/ffs/README,v 1.4 1999/12/03 00 34 26 billf Exp $ $FreeBSD src/sys/ufs/ffs/README.softupdates,v 1.7.2.1 2000/06/26 14 09 01 = sheldonh Exp $ $FreeBSD src/sys/ufs/ffs/ffs_alloc.c,v 1.64.2.2 2001/09/21 19 15 21 dillon= Exp $ $FreeBSD src/sys/ufs/ffs/ffs_balloc.c,v 1.26 2000/02/24 20 43 20 dillon Ex= p $ $FreeBSD src/sys/ufs/ffs/ffs_extern.h,v 1.30 2000/01/09 22 40 02 mckusick = Exp $ $FreeBSD src/sys/ufs/ffs/ffs_inode.c,v 1.56.2.5 2002/02/05 18 35 03 dillon= Exp $ $FreeBSD src/sys/ufs/ffs/ffs_softdep.c,v 1.57.2.11 2002/02/05 18 46 53 dil= lon Exp $ $FreeBSD src/sys/ufs/ffs/ffs_softdep_stub.c,v 1.7.2.1 2000/12/28 11 01 45 = ps Exp $ $FreeBSD src/sys/ufs/ffs/ffs_subr.c,v 1.25 1999/12/29 04 55 04 peter Exp $ $FreeBSD src/sys/ufs/ffs/ffs_tables.c,v 1.7 1999/08/28 00 52 22 peter Exp $ $FreeBSD src/sys/ufs/ffs/ffs_vfsops.c,v 1.117.2.8 2002/02/05 18 35 03 dill= on Exp $ $FreeBSD src/sys/ufs/ffs/ffs_vnops.c,v 1.64 2000/01/10 12 04 25 phk Exp $ $FreeBSD src/sys/ufs/ffs/fs.h,v 1.14.2.3 2001/09/21 19 15 22 dillon Exp $ $FreeBSD src/sys/ufs/ffs/softdep.h,v 1.7.2.1 2000/06/22 19 27 42 peter Exp= $ $FreeBSD src/sys/sys/queue.h, 1.32.2.6 2001/12/18 10:09:02 ru Exp $ (kgdb) exec-file kernel.0 (kgdb) symbol-file /usr/obj/usr/src/sys/M1/kernel.debug Reading symbols from /usr/obj/usr/src/sys/M1/kernel.debug...core-done. (kgdb) core-file vmcore.0 IdlePTD at phsyical address 0x004b2000 initial pcb at physical address 0x002d50c0 panicstr: worklist_remove: not on list panic messages: --- --- #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:485 485 if (dumping++) { (kgdb) where #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:485 #1 0xc014c14b in boot (howto=3D256) at /usr/src/sys/kern/kern_shutdown.c:3= 14 #2 0xc014c589 in panic (fmt=3D0xc027d4ff "worklist_remove: not on list") at /usr/src/sys/kern/kern_shutdown.c:593 #3 0xc01e681f in worklist_remove (item=3D0xc1a6e0a0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:470 #4 0xc01ea222 in handle_written_inodeblock (inodedep=3D0xc1bb3500,=20 bp=3D0xc6d3f2a4) at /usr/src/sys/ufs/ffs/ffs_softdep.c:3549 #5 0xc01e9be6 in softdep_disk_write_complete (bp=3D0xc6d3f2a4) at /usr/src/sys/ufs/ffs/ffs_softdep.c:3242 #6 0xc0173a2d in biodone (bp=3D0xc6d3f2a4) at /usr/src/sys/kern/vfs_bio.c:= 2706 #7 0xc0231927 in ad_interrupt (request=3D0xc1bb0e80) at /usr/src/sys/dev/ata/ata-disk.c:703 #8 0xc022c20e in ata_intr (data=3D0xc1a5ae00) at /usr/src/sys/dev/ata/ata-all.c:1231 #9 0xc024c312 in vec14 () #10 0xc014bcec in reboot (p=3D0xcc26ce00, uap=3D0xcc273f80) at /usr/src/sys/kern/kern_shutdown.c:149 #11 0xc02571a9 in syscall2 (frame=3D{tf_fs =3D 47, tf_es =3D 47, tf_ds =3D = 47,=20 tf_edi =3D -1077936612, tf_esi =3D -1077936624, tf_ebp =3D -107793683= 6,=20 tf_isp =3D -869842988, tf_ebx =3D -1077936732, tf_edx =3D -1, tf_ecx = =3D 4,=20 tf_eax =3D 55, tf_trapno =3D 7, tf_err =3D 2, tf_eip =3D 134543392, t= f_cs =3D 31,=20 tf_eflags =3D 643, tf_esp =3D -1077937056, tf_ss =3D 47}) at /usr/src/sys/i386/i386/trap.c:1167 #12 0xc024ae75 in Xint0x80_syscall () #13 0x80486e6 in ?? () #14 0x8048471 in ?? () #15 0x8048135 in ?? () (kgdb) up 4 #4 0xc01ea222 in handle_written_inodeblock (inodedep=3D0xc1bb3500,=20 bp=3D0xc6d3f2a4) at /usr/src/sys/ufs/ffs/ffs_softdep.c:3549 3549 WORKLIST_REMOVE(wk); > [ postprocessed ffs_softdep.c ~3549 ] > while ((wk =3D (( &inodedep->id_bufwait )->lh_first) ) !=3D 0= ) { > worklist_remove( wk ) ; > switch (wk->wk_type) { >=20 > case 9 : (kgdb) print wk $1 =3D (struct worklist *) 0xc1a6e0a0 [DEBUG/DIAGNOSTIC defined further up, so we use real functions, not macros.] (kdgb) down 1 #3 0xc01e681f in worklist_remove (item=3D0xc1a6e0a0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:470 470 panic("worklist_remove: not on list"); > [ postprocessed ffs_softdep.c ~470 ] > static void > worklist_remove(item) > struct worklist *item; >=20 > if (lk.lkt_held =3D=3D -1) > panic("worklist_remove: lock not held"); > if ((item->wk_state & 0x8000 ) =3D=3D 0) { > free_lock( &lk ) ; > panic("worklist_remove: not on list"); > } > item->wk_state &=3D ~0x8000 ; > do { if ((( ( item ) )-> wk_list .le_next) !=3D 0 ) = (( ( ite > m ) )-> wk_list .le_next) -> wk_list .le_prev =3D ( item )-> wk= _list .le_pre > v; *( item )-> wk_list .le_prev =3D (( ( item ) )-> wk_list = .le_next) ;=20 > } while (0) ; > } (kgdb) print item $2 =3D (struct worklist *) 0x0 [ So, um, how did this suddenly become a NULL pointer? Did I mistrace something up there? ] (kgdb) print &item Address requested for identifier "item" which is in a register. (kgdb) info registers eax 0x0 0 ecx 0x0 0 edx 0x0 0 ebx 0xc1a6e0a0 -1046028128 esp 0xcc273d9c 0xcc273d9c ebp 0xcc273df0 0xcc273df0 esi 0x0 0 edi 0x0 0 eip 0xc01e681f 0xc01e681f eflags 0x0 0 cs 0x0 0 ss 0x0 0 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x2f 47 [ I admittedly don't know much (if at all) about debugging x86 registers. ] [ Look at ebx. Look familiar? (kgdb) print wk =20 $1 =3D (struct worklist *) 0xc1a6e0a0 ] (kgdb) p ({struct worklist *}(0xc1a6e0a0))->wk_state $3 =3D 61 [ Would need to be >=3D 32768 ... ] (kgdb) p ({struct worklist *}(0xc1a6e0a0))->wk_state & 0x8000 $4 =3D 0 [ Unless I'm missing something, ONWORKLIST/0x8000 is only cleared if we run through WORKLIST_REMOVE, worklist_remove, softdep_disk_io_initiation (with directory write dependency?). I really need to read the softupdates papers someday... reading through and understanding the vfs code might help as well, no?] This leads us to the panic. =20 -----END DEBUG STUFF----- --=20 ryan beasley professional fat bastard http://www.goddamnbastard.org GPG ID 0x36321D13 --O5XBE6gyVG5Rl6Rj Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE8atqqCbo22TYyHRMRArYqAJ482JdHFC1DO27Lh0A2fnmlNYg6DgCghOYb wyubComy8EFvWyKyU/Ey6V0= =/OsP -----END PGP SIGNATURE----- --O5XBE6gyVG5Rl6Rj-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message