Date: Wed, 25 Jun 2003 18:20:33 +0200 From: Ian Freislich <ianf@za.uu.net> To: freebsd-current@freebsd.org Subject: ULE crash Message-ID: <7075.1056558033@mci.com>
next in thread | raw e-mail | index | archive | help
Hi
About 4.5 minutes after rebooting with a SCHED_ULE kernel (I give
ULE a go every few months), top started looking really wierd (the
CPU % just kept on accumulating for each process). Before dnetc
started, httpd showed 17% CPU, but the system was supposedly 100%
idle at the time according to top. Then dnetc started and things
got wierd.
last pid: 607; load averages: 1.83, 0.63, 0.25 up 0+00:04:23 16:00:48
35 processes: 3 running, 32 sleeping
CPU states: 0.0% user, 99.0% nice, 0.6% system, 0.4% interrupt, 0.0% idle
Mem: 20M Active, 14M Inact, 19M Wired, 20K Cache, 25M Buf, 130M Free
Swap: 512M Total, 512M Free
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
603 ianf 139 20 1072K 880K RUN 0 0:39 105.47% 105.47% dnetc
575 ianf 139 20 1072K 880K CPU1 1 1:15 102.34% 102.34% dnetc
505 root 76 0 7208K 5420K select 0 0:01 17.97% 17.97% httpd
375 root 4 0 1276K 948K accept 0 0:00 9.38% 9.38% nfsd
526 nobody 76 0 9280K 8564K select 1 0:04 5.47% 5.47% squid
607 ianf 76 0 2196K 1444K CPU0 0 0:00 2.34% 2.34% top
Then it froze. When I got home I found that it had at least dumped
vmcore.24. I'll keep it around for a while and perform any inspections
people want me to. This was with sources updated at 13h30 GMT today.
panic: page fault
panic messages:
---
Fatal trap 12: page fault while in kernel mode
cpuid = 1; lapic.id = 01000000
fault virtual address = 0x38
fault code = supervisor read, page not present
instruction pointer = 0x8:0xc01e094d
stack pointer = 0x10:0xce772be4
frame pointer = 0x10:0xce772bf4
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 603 (dnetc)
trap number = 12
panic: page fault
cpuid = 1; lapic.id = 01000000
Stack backtrace:
boot() called on cpu#1
syncing disks, buffers remaining... panic: absolutely cannot call smp_ipi_shootdown with interrupts already disabled
cpuid = 1; lapic.id = 01000000
boot() called on cpu#1
Uptime: 4m15s
Dumping 191 MB
ata0: resetting devices ..
done
16 32 48 64 80 96 112 128 144 160 176
---
(kgdb) bt
#0 doadump () at ../../../kern/kern_shutdown.c:240
#1 0xc01cbe7f in boot (howto=260) at ../../../kern/kern_shutdown.c:372
#2 0xc01cc2b8 in panic () at ../../../kern/kern_shutdown.c:550
#3 0xc02e8f89 in smp_tlb_shootdown (vector=0, addr1=0, addr2=0)
at ../../../i386/i386/mp_machdep.c:2356
#4 0xc02e92a9 in smp_invlpg_range (addr1=0, addr2=0)
at ../../../i386/i386/mp_machdep.c:2488
#5 0xc02eb548 in pmap_invalidate_range (pmap=0xc03996e0, sva=3365310464,
eva=3365314560) at ../../../i386/i386/pmap.c:721
#6 0xc02eb83d in pmap_qenter (sva=3365310464, m=0xce772884, count=0)
at ../../../i386/i386/pmap.c:948
#7 0xc0218a31 in vm_hold_load_pages (bp=0xc76039a0, from=0, to=3365318656)
at ../../../kern/vfs_bio.c:3574
#8 0xc0216f5a in allocbuf (bp=0xc76039a0, size=8192)
at ../../../kern/vfs_bio.c:2752
#9 0xc0216cee in geteblk (size=8192) at ../../../kern/vfs_bio.c:2634
#10 0xc0213980 in bwrite (bp=0xc75b65d8) at ../../../kern/vfs_bio.c:818
#11 0xc02142dc in bawrite (bp=0x0) at ../../../kern/vfs_bio.c:1153
#12 0xc021d89a in vop_stdfsync (ap=0xce772a14)
at ../../../kern/vfs_default.c:742
#13 0xc0193570 in spec_fsync (ap=0xce772a14)
at ../../../fs/specfs/spec_vnops.c:417
#14 0xc0192a38 in spec_vnoperate (ap=0x0)
at ../../../fs/specfs/spec_vnops.c:122
#15 0xc0294c62 in ffs_sync (mp=0xc3950a00, waitfor=2, cred=0xc0d06e80,
td=0xc03702a0) at vnode_if.h:624
#16 0xc022b15b in sync (td=0xc03702a0, uap=0x0)
at ../../../kern/vfs_syscalls.c:142
#17 0xc01cb9a1 in boot (howto=256) at ../../../kern/kern_shutdown.c:281
#18 0xc01cc2b8 in panic () at ../../../kern/kern_shutdown.c:550
#19 0xc02f0da2 in trap_fatal (frame=0xce772ba4, eva=0)
at ../../../i386/i386/trap.c:836
#20 0xc02f0333 in trap (frame=
{tf_fs = -1060044776, tf_es = -831062000, tf_ds = -1071775728, tf_edi = -1014422336, tf_esi = -1070107520, tf_ebp = -831050764, tf_isp = -831050800, tf_ebx = 0, tf_edx = 0, tf_ecx = -1059988168, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1071773363, tf_cs = 8, tf_eflags = 66194, tf_esp = -1070107520, tf_ss = 0}) at ../../../i386/i386/trap.c:256
#21 0xc02d8eb8 in calltrap () at {standard input}:97
#22 0xc01e188b in sched_choose () at ../../../kern/sched_ule.c:1161
#23 0xc01d25e6 in choosethread () at ../../../kern/kern_switch.c:140
#24 0xc01d422f in mi_switch () at ../../../kern/kern_synch.c:525
#25 0xc01c1db6 in _mtx_lock_sleep (m=0xc0374a40, opts=0, file=0x0, line=0)
at ../../../kern/kern_mutex.c:636
#26 0xc01ca585 in getrusage (td=0x0, uap=0xce772d10)
at ../../../kern/kern_resource.c:773
#27 0xc02f10fc in syscall (frame=
{tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 135360172, tf_esi = 135336096, tf_ebp = -1077938416, tf_isp = -831050380, tf_ebx = -1077938416, tf_edx = 0, tf_ecx = 0, tf_eax = 117, tf_trapno = 0, tf_err = 2, tf_eip = 134789976, tf_cs = 31, tf_eflags = 659, tf_esp = -1077938572, tf_ss = 47})
at ../../../i386/i386/trap.c:1023
#28 0xc02d8f0d in Xint0x80_syscall () at {standard input}:139
---Can't read userspace from dump, or kernel process---
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7075.1056558033>