Date: Sun, 02 Mar 2008 03:49:03 -0800 From: LI Xin <delphij@delphij.net> To: FreeBSD Stable <freebsd-stable@freebsd.org> Cc: Pawel Jakub Dawidek <pjd@FreeBSD.org>, Quake Lee <quakelee@geekcn.org> Subject: Reliably trigger-able ZFS panic Message-ID: <47CA942F.2080901@delphij.net>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigD32EDA9C0DB3DBBF8BAA1DD9
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Hi,
The following iozone test case on ZFS would reliably trigger panic:
/usr/local/bin/iozone -M -e -+u -T -t 128 -S 4096 -L 64 -R -r 4k -s 30g=20
-i 0 -i 1 -i 2 -i 8 -+p 70 -C
Unfortunately the kgdb can not reveal useful backtrace. I have tried=20
KDB_TRACE, but have not yet be able to further investigate it.
fs12# kgdb /boot/kernel/kernel.symbols vmcore.0
[GDB will not be able to debug user-mode threads:=20
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you =
are
welcome to change it and/or distribute copies of it under certain=20
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for detail=
s.
This GDB was configured as "amd64-marcel-freebsd".
Unread portion of the kernel message buffer:
Fatal trap 12: page fault while in kernel mode
cpuid =3D 5; apic id =3D 05
fault virtual address =3D 0x18
fault code =3D supervisor read data, page not present
instruction pointer =3D 0x8:0xffffffff80763d16
stack pointer =3D 0x10:0xffffffffd94798f0
frame pointer =3D 0x10:0xffffffffd9479920
code segment =3D base 0x0, limit 0xfffff, type 0x1b
=3D DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags =3D interrupt enabled, resume, IOPL =3D 0
current process =3D 340 (txg_thread_enter)
trap number =3D 12
panic: page fault
cpuid =3D 5
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x17a
trap_fatal() at trap_fatal+0x29f
trap_pfault() at trap_pfault+0x294
trap() at trap+0x2ea
calltrap() at calltrap+0x8
--- trap 0xc, rip =3D 0xffffffff80763d16, rsp =3D 0xffffffffd94798f0, rbp=
=3D=20
0xffffffffd9479920 ---
dmu_objset_sync_dnodes() at dmu_objset_sync_dnodes+0x26
dmu_objset_sync() at dmu_objset_sync+0x12d
dsl_pool_sync() at dsl_pool_sync+0x72
spa_sync() at spa_sync+0x390
txg_sync_thread() at txg_sync_thread+0x12f
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip =3D 0, rsp =3D 0xffffffffd9479d30, rbp =3D 0 ---
Uptime: 25m7s
Physical memory: 4081 MB
Dumping 1139 MB: 1124 1108 1092 1076 1060 1044 1028 1012 996 980 964 948 =
932 916 900 884 868 852 836 820 804 788 772 756 740 724 708 692 676 660=20
644 628 612 596 580 564 548 532 516 500 484 468 452 436 420 404 388 372=20
356 340 324 308 292 276 260 244 228 212 196 180 164 148 132 116 100 84=20
68 52 36 20 4
#0 doadump () at pcpu.h:194
194 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) add-symbol-file /boot/kernel/zfs.ko.symbols
add symbol table from file "/boot/kernel/zfs.ko.symbols" at
(y or n) y
Reading symbols from /boot/kernel/zfs.ko.symbols...done.
(kgdb) where
#0 doadump () at pcpu.h:194
#1 0xffffffff80277aa8 in boot (howto=3D260) at=20
/usr/src/sys/kern/kern_shutdown.c:409
#2 0xffffffff80277f07 in panic (fmt=3DVariable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3 0xffffffff80465a1f in trap_fatal (frame=3D0xc, eva=3DVariable "eva" i=
s=20
not available.
) at /usr/src/sys/amd64/amd64/trap.c:724
#4 0xffffffff80465e04 in trap_pfault (frame=3D0xffffffffd9479840, usermo=
de=3D0)
at /usr/src/sys/amd64/amd64/trap.c:641
#5 0xffffffff8046677a in trap (frame=3D0xffffffffd9479840) at=20
/usr/src/sys/amd64/amd64/trap.c:410
#6 0xffffffff8044babe in calltrap () at=20
/usr/src/sys/amd64/amd64/exception.S:169
#7 0xffffffff80763d16 in ?? ()
#8 0x0000000000000004 in adjust_ace_pair ()
#9 0x0000000000000004 in adjust_ace_pair ()
#10 0xffffffffd94799e0 in ?? ()
#11 0xffffffff80763e7d in ?? ()
#12 0xffffff0004275a80 in ?? ()
#13 0xffffff00045a1190 in ?? ()
#14 0xffffffff807639b0 in ?? ()
#15 0xffffffff80763f20 in ?? ()
#16 0xffffff00042dc800 in ?? ()
#17 0x0000000000000004 in adjust_ace_pair ()
#18 0xffffffffd9479990 in ?? ()
#19 0x000000000000b55d in z_deflateInit2_ (strm=3D0xffffff00042dc8e0,=20
level=3D70109184, method=3D68351768,
windowBits=3D68351600, memLevel=3D76231808, strategy=3D76231808,=20
version=3DCannot access memory at address 0xffffffff00040010
)
at=20
/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/zmod/deflat=
e.c:318
Previous frame inner to this frame (corrupt stack?)
--=20
Xin LI <delphij@delphij.net> http://www.delphij.net/
FreeBSD - The Power to Serve!
--------------enigD32EDA9C0DB3DBBF8BAA1DD9
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHypQvOfuToMruuMARCvWnAJ4/TF2S8Q0BlMXGVvFeCLH0NPj3bACeMGZN
7C4iVCxjFyIFwtIZaL/WOtg=
=JcN0
-----END PGP SIGNATURE-----
--------------enigD32EDA9C0DB3DBBF8BAA1DD9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47CA942F.2080901>