Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 20 May 2003 20:11:32 -0300
From:      "Daniel C. Sobral" <dcs@tcoip.com.br>
To:        Robert Watson <rwatson@FreeBSD.ORG>
Cc:        CURRENT <freebsd-current@FreeBSD.ORG>
Subject:   Re: /dev/null and KSE panic 100% reproducible
Message-ID:  <3ECAB624.5020703@tcoip.com.br>
In-Reply-To: <Pine.NEB.3.96L.1030520100942.68795C-100000@fledge.watson.org>
References:  <Pine.NEB.3.96L.1030520100942.68795C-100000@fledge.watson.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Robert Watson wrote:
> Just for my benefit, could you check and see if you still get the
> reproduceable KSE panic without the MAC stuff compiled into the kernel?=
=20
> If not, it could have gone away because the bug is in the MAC code,
> because the bug was encouraged by the MAC code, or because it was a
> ordering/timing thing and it was a fluke that it occured consistently.
> Regardless, if you can reproduce it without MAC, it will also mean it's=

> likely not my fault :-).

Congratulations, the child is yours! :-)

No panics without mac. Sorry. :-) At least I got a backtrace:

GNU gdb 5.2.1 (FreeBSD)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you =
are
welcome to change it and/or distribute copies of it under certain=20
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for detail=
s.
This GDB was configured as "i386-undermydesk-freebsd"...
panic: from debugger
panic messages:
---
panic: No strategy on dev null responsible for buffer 0xc77a6050

Stack backtrace:
panic: from debugger
Uptime: 56s
Dumping 255 MB
ata0: resetting devices ..
done
  16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
---
Reading symbols from /boot/kernel/snd_cmi.ko...done.
Loaded symbols for /boot/kernel/snd_cmi.ko
Reading symbols from /boot/kernel/snd_pcm.ko...done.
Loaded symbols for /boot/kernel/snd_pcm.ko
Reading symbols from=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/mac_biba/mac_biba.ko=
=2Edebug...done.
Loaded symbols for=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/mac_biba/mac_biba.ko=
=2Edebug
Reading symbols from=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/mac_mls/mac_mls.ko.d=
ebug...done.
Loaded symbols for=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/mac_mls/mac_mls.ko.d=
ebug
Reading symbols from=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/acpi/acpi.ko.debug..=
=2Edone.
Loaded symbols for=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/acpi/acpi.ko.debug
Reading symbols from /boot/kernel/green_saver.ko...done.
Loaded symbols for /boot/kernel/green_saver.ko
Reading symbols from=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/linux/linux.ko.debug=
=2E..done.
Loaded symbols for=20
/usr/obj/usr/src/sys/DCS/modules/usr/src/sys/modules/linux/linux.ko.debug=

#0  doadump () at /usr/src/sys/kern/kern_shutdown.c:238
238             dumping++;
(kgdb) bt full
#0  doadump () at /usr/src/sys/kern/kern_shutdown.c:238
No locals.
#1  0xc01e7353 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c=
:370
No locals.
#2  0xc01e769b in panic () at /usr/src/sys/kern/kern_shutdown.c:543
         td =3D (struct thread *) 0xc2a72000
         bootopt =3D 260
         newpanic =3D 0
         buf =3D "from debugger\0 dev null responsible for buffer=20
0xc77a6050\n", '\0' <repeats 197 times>
#3  0xc0128812 in db_panic () at /usr/src/sys/ddb/db_command.c:448
No locals.
#4  0xc0128792 in db_command (last_cmdp=3D0xc033e120, cmd_table=3D0x0,=20
aux_cmd_tablep=3D0xc0339460,
     aux_cmd_tablep_end=3D0xc0339464) at /usr/src/sys/ddb/db_command.c:34=
6
         cmd =3D (struct command *) 0xc0302d60
         t =3D 0
         modif =3D=20
"\0j6=C0h}<=C0\220=C5i=D2\r\0\0\0=E0h;=C0\r\0\0\0\001\0\0\0=B0=C5i=D2f=E7=
,=C0@O:=C0\aK\0 `i;=C0=20
\035:=C0 j6=C0x\0\0\0 j6=C0h}<=C0=D4=C5i=D2=B1=A4\022=C0ku1=C0 =A3\022=C0=
\0\0\0\0\020\0\0\0h}<=C0 j6=C0\036\235\022=C0 j6=C0=E0a6=C0x\0\0\0\003\0\=
0"
         addr =3D -1070757260
         count =3D -1
         have_addr =3D 0
         result =3D 0
#5  0xc01288a6 in db_command_loop () at /usr/src/sys/ddb/db_command.c:470=

No locals.
#6  0xc012b63a in db_trap (type=3D3, code=3D0) at /usr/src/sys/ddb/db_tra=
p.c:72
         bkpt =3D 0
#7  0xc02d87d5 in kdb_trap (type=3D3, code=3D0, regs=3D0xd269c6cc) at=20
/usr/src/sys/i386/i386/db_interface.c:170
         ef =3D 70
         ddb_mode =3D 1
#8  0xc02ea17c in trap (frame=3D
       {tf_fs =3D 24, tf_es =3D 16, tf_ds =3D 16, tf_edi =3D 256, tf_esi =
=3D=20
-1029234688, tf_ebp =3D -764819688, tf_isp =3D -764819720, tf_ebx =3D 0, =

tf_edx =3D 0, tf_ecx =3D 1920, tf_eax =3D 18, tf_trapno =3D 3, tf_err =3D=
 0,=20
tf_eip =3D -1070757260, tf_cs =3D 8, tf_eflags =3D 642, tf_esp =3D -10703=
82643,=20
tf_ss =3D -1070452718}) at /usr/src/sys/i386/i386/trap.c:593
         td =3D (struct thread *) 0xc2a72000
         p =3D (struct proc *) 0xc2adc780
         sticks =3D 926376246
---Type <return> to continue, or q <return> to quit---
         i =3D 0
         ucode =3D 0
         type =3D 3
         code =3D 0
         eva =3D 0
#9  0xc02da128 in calltrap () at {standard input}:96
No locals.
#10 0xc01e763b in panic (fmt=3D0x0) at /usr/src/sys/kern/kern_shutdown.c:=
527
         td =3D (struct thread *) 0xc2a72000
         bootopt =3D 256
         newpanic =3D 1
         buf =3D "from debugger\0 dev null responsible for buffer=20
0xc77a6050\n", '\0' <repeats 197 times>
#11 0xc019fca0 in spec_xstrategy (vp=3D0xc2b94a44, bp=3D0xc77a6050) at=20
/usr/src/sys/fs/specfs/spec_vnops.c:506
         mp =3D (struct mount *) 0x0
         error =3D 0
         dsw =3D (struct cdevsw *) 0x0
         td =3D (struct thread *) 0xc2a72000
#12 0xc019feeb in spec_specstrategy (ap=3D0x0) at=20
/usr/src/sys/fs/specfs/spec_vnops.c:550
No locals.
#13 0xc019ee88 in spec_vnoperate (ap=3D0x0) at=20
/usr/src/sys/fs/specfs/spec_vnops.c:123
No locals.
#14 0xc029dc88 in ufs_vnoperatespec (ap=3D0x0) at=20
/usr/src/sys/ufs/ufs/ufs_vnops.c:2805
No locals.
#15 0xc022babe in breadn (vp=3D0xc2b94a44, blkno=3D0, size=3D0, rablkno=3D=
0x0,=20
rabsize=3D0x0, cnt=3D0, cred=3D0x0, bpp=3D0x0)
     at vnode_if.h:1089
         bp =3D (struct buf *) 0xc77a6050
         rabp =3D (struct buf *) 0xc2b94a44
         i =3D 256
         rv =3D 0
         readwait =3D 0
#16 0xc022b98c in bread (vp=3D0x0, blkno=3D0, size=3D0, cred=3D0x0, bpp=3D=
0x0) at=20
/usr/src/sys/kern/vfs_bio.c:683
No locals.
#17 0xc028f735 in ffs_extread (vp=3D0xc2b94a44, uio=3D0xd269c8e0, ioflag=3D=
1028)
     at /usr/src/sys/ufs/ffs/ffs_vnops.c:1007
         ip =3D (struct inode *) 0xc2bdabd0
---Type <return> to continue, or q <return> to quit---
         dp =3D (struct ufs2_dinode *) 0xc2bdcd00
         fs =3D (struct fs *) 0xc27f1800
         bp =3D (struct buf *) 0xc77a6050
         lbn =3D 0
         nextlbn =3D 1
         bytesinfile =3D -3284878155370116540
         size =3D 2048
         xfersize =3D 256
         blkoffset =3D 0
         error =3D 0
         orig_resid =3D 256
#18 0xc028ff69 in ffs_rdextattr (p=3D0x0, vp=3D0xc2b94a44, td=3D0x0, extr=
a=3D0)
     at /usr/src/sys/ufs/ffs/ffs_vnops.c:1303
         ip =3D (struct inode *) 0x0
         dp =3D (struct ufs2_dinode *) 0x0
         luio =3D {uio_iov =3D 0xd269c8d8, uio_iovcnt =3D 1, uio_offset =3D=
 0,=20
uio_resid =3D 256,
   uio_segflg =3D UIO_SYSSPACE, uio_rw =3D UIO_READ, uio_td =3D 0xc2a7200=
0}
         liovec =3D {iov_base =3D 0xc2733400, iov_len =3D 256}
         easize =3D 256
         error =3D 256
         eae =3D (
     u_char *) 0xc2733400=20
"=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=
=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=
=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=
=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=
=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=
=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=
=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=
=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=DE=DE=C0=AD=
=DE"...
#19 0xc028ffe6 in ffs_open_ea (vp=3D0x0, cred=3D0x0, td=3D0x0) at=20
/usr/src/sys/ufs/ffs/ffs_vnops.c:1326
         ip =3D (struct inode *) 0xc2bdabd0
         dp =3D (struct ufs2_dinode *) 0xc2bdcd00
         error =3D 0
#20 0xc02902f0 in ffs_getextattr (ap=3D0xd269c978) at=20
/usr/src/sys/ufs/ffs/ffs_vnops.c:1495
         ip =3D (struct inode *) 0xc2bdabd0
         fs =3D (struct fs *) 0x0
         eae =3D (u_char *) 0xd269c978 "\200S6=C0DJ=B9=C2\002"
         p =3D (u_char *) 0x0
         pe =3D (u_char *) 0xc2bdabd0 ""
         pn =3D (u_char *) 0x0
---Type <return> to continue, or q <return> to quit---
         easize =3D 3267210192
         ul =3D 3224917536
         error =3D -1027748608
         ealen =3D 0
         stand_alone =3D -1070078640
#21 0xc024a8db in VOP_GETEXTATTR (vp=3D0x0, attrnamespace=3D0, name=3D0x0=
,=20
uio=3D0x0, size=3D0x0, cred=3D0x0, td=3D0x0)
     at vnode_if.h:1543
         a =3D {a_desc =3D 0xc0365380, a_vp =3D 0xc2b94a44, a_attrnamespa=
ce =3D=20
2, a_name =3D 0xc0471225 "mac_mls",
   a_uio =3D 0xd269c9d4, a_size =3D 0x0, a_cred =3D 0x0, a_td =3D 0xc2a72=
000}
         rc =3D 0
#22 0xc024a64f in vn_extattr_get (vp=3D0xc2b94a44, ioflg=3D8,=20
attrnamespace=3D0, attrname=3D0x0, buflen=3D0xd269ca24,
     buf=3D0x0, td=3D0xc2a72000) at /usr/src/sys/kern/vfs_vnops.c:1077
         auio =3D {uio_iov =3D 0xd269c9bc, uio_iovcnt =3D 1, uio_offset =3D=
 0,=20
uio_resid =3D 112,
   uio_segflg =3D UIO_SYSSPACE, uio_rw =3D UIO_READ, uio_td =3D 0xc2a7200=
0}
         iov =3D {iov_base =3D 0xd269ca2c, iov_len =3D 112}
         error =3D -764818900
#23 0xc046f759 in mac_mls_associate_vnode_extattr (mp=3D0xc2611a00,=20
fslabel=3D0x0, vp=3D0x0, vlabel=3D0x0)
     at /usr/src/sys/security/mac_mls/mac_mls.c:894
         temp =3D {mm_flags =3D 0, mm_single =3D {mme_type =3D 0, mme_lev=
el =3D 0,
     mme_compartments =3D '\0' <repeats 31 times>}, mm_rangelow =3D=20
{mme_type =3D 0, mme_level =3D 0,
     mme_compartments =3D '\0' <repeats 31 times>}, mm_rangehigh =3D=20
{mme_type =3D 0, mme_level =3D 0,
     mme_compartments =3D '\0' <repeats 31 times>}}
         source =3D (struct mac_mls *) 0xc27fae00
         dest =3D (struct mac_mls *) 0xc2bd9580
         buflen =3D 112
         error =3D 0
#24 0xc01d1e52 in mac_associate_vnode_extattr (mp=3D0xc2611a00, vp=3D0xc2=
b94a44)
     at /usr/src/sys/kern/kern_mac.c:1437
         mpc =3D (struct mac_policy_conf *) 0xc0472a40
         error =3D 0
#25 0xc028d9a2 in ffs_vget (mp=3D0xc2611a00, ino=3D452761, flags=3D2,=20
vpp=3D0xd269cc0c)
     at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1370
         td =3D (struct thread *) 0xc2a72000
         fs =3D (struct fs *) 0xc27f1800
         ip =3D (struct inode *) 0xc2bdabd0
---Type <return> to continue, or q <return> to quit---
         ump =3D (struct ufsmount *) 0xc2733800
         bp =3D (struct buf *) 0xc77a33a0
         vp =3D (struct vnode *) 0xc2b94a44
         dev =3D (struct cdev *) 0x0
         error =3D -1027757104
#26 0xc0299d6e in ufs_mknod (ap=3D0xd269cba8) at=20
/usr/src/sys/ufs/ufs/ufs_vnops.c:248
         vap =3D (struct vattr *) 0xd269cc48
         vpp =3D (struct vnode **) 0xd269cc0c
         ip =3D (struct inode *) 0x6e899
         ino =3D 452761
         error =3D 0
#27 0xc029dc48 in ufs_vnoperate (ap=3D0x0) at=20
/usr/src/sys/ufs/ufs/ufs_vnops.c:2787
No locals.
#28 0xc0242d94 in kern_mknod (td=3D0xc2a72000, path=3D---Can't read=20
userspace from dump, or kernel process---

) at vnode_if.h:179
         vp =3D (struct vnode *) 0x0
         mp =3D (struct mount *) 0xc2611a00
         vattr =3D {va_type =3D VCHR, va_mode =3D 420, va_nlink =3D -1, v=
a_uid =3D=20
4294967295, va_gid =3D 4294967295,
   va_fsid =3D 4294967295, va_fileid =3D -1, va_size =3D 1844674407370955=
1615,=20
va_blocksize =3D -1, va_atime =3D {
     tv_sec =3D -1, tv_nsec =3D -1}, va_mtime =3D {tv_sec =3D -1, tv_nsec=
 =3D -1},=20
va_ctime =3D {tv_sec =3D -1,
     tv_nsec =3D -1}, va_birthtime =3D {tv_sec =3D -1, tv_nsec =3D -1}, v=
a_gen =3D=20
4294967295, va_flags =3D 4294967295,
   va_rdev =3D 514, va_bytes =3D 18446744073709551615, va_filerev =3D 0, =

va_vaflags =3D 0, va_spare =3D 0}
         error =3D -1028001792
         whiteout =3D 0
         nd =3D {ni_dirp =3D 0xbfbffc86---Can't read userspace from dump,=
 or=20
kernel process---


I also enabled trace on panic, since I was expecting the KSE bug to hide =

everything again (it didn't -- I hope it comes back tomorrow for=20
Julian's test :), and got the following, which preceded a backtrace that =

looks like the one above:

VOP_SPECSTRATEGY on non-VCHR: 0xc2b94b68: tag ufs, type VCHR, usecount=20
1, writecount 0, refcount 1, lock type ufs: EXCL(count 1) by thread=20
0xc2a72000
	Ino 452761, on dev ad0s2h (4,21)
Stack backtrace:
backtrace()+xxx
vop_nospecstrategy()+0x2d
vop_defaultop()+0x18
ufs_vnoperate()+0x18
bwrite()+0x337
ffs_extwrite()+0x319
ffs_close_ea()+0xf3
ffs_closeextattr()+0x50
mac_create_vnode_extattr()+0x23f
ufs_makeinode()+0x3da
ufs_mknod()+0x33a
mknod()+0x30
syscall(2f,2f,2f,21b6,5)+0x26e
Xint0x80_syscall()+0x1d


>=20
> I'll try installing the linux emulator base stuff on some test machines=

> today -- just installing it is enough to trigger it?

Looks like it.

--=20
Daniel C. Sobral                   (8-DCS)
Gerencia de Operacoes
Divisao de Comunicacao de Dados
Coordenacao de Seguranca
VIVO Centro Oeste Norte
Fones: 55-61-313-7654/Cel: 55-61-9618-0904
E-mail: Daniel.Capo@tco.net.br
         Daniel.Sobral@tcoip.com.br
         dcs@tcoip.com.br

Outros:
	dcs@newsguy.com
	dcs@freebsd.org
	capo@notorious.bsdconspiracy.net

Must I hold a candle to my shames?
		-- William Shakespeare, "The Merchant of Venice"



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3ECAB624.5020703>