Date: Fri, 17 Mar 2006 21:01:05 -0600 From: Vlad <marchenko@gmail.com> To: Vlad <marchenko@gmail.com>, "Kris Kennaway" <kris@obsecurity.org>, freebsd-stable@freebsd.org Subject: Re: Fatal trap 12: page fault while in kernel mode / current process=12 (swi1: net) Message-ID: <cd70c6810603171901t64cf8aectebd9c288164b32b0@mail.gmail.com> In-Reply-To: <20060317212755.GC38277@nickel.oss.ntelos.net> References: <cd70c6810603170859s3636d11bi615c88c6c1332329@mail.gmail.com> <20060317173651.GA27314@xor.obsecurity.org> <cd70c6810603170941m92dbd5cw9cb0cae75cbcd730@mail.gmail.com> <20060317212755.GC38277@nickel.oss.ntelos.net>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_252_5751521.1142650865633
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Ok, thanks for Joe's hint I was able to get stuff captured:
# kgdb kernel.debug /var/crash/vmcore.0
[GDB will not be able to debug user-mode threads:
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you ar=
e
welcome to change it and/or distribute copies of it under certain condition=
s.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd".
Unread portion of the kernel message buffer:
kernel trap 12 with interrupts disabled
Fatal trap 12: page fault while in kernel mode
fault virtual address =3D 0x48
fault code =3D supervisor read, page not present
instruction pointer =3D 0x8:0xffffffff8026d5f6
stack pointer =3D 0x10:0xffffffffa52ee870
frame pointer =3D 0x10:0xffffffffa52ee8b0
code segment =3D base 0x0, limit 0xfffff, type 0x1b
=3D DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags =3D resume, IOPL =3D 0
current process =3D 12 (swi1: net)
panic: from debugger
Uptime: 4h15m12s
Dumping 1023 MB (2 chunks)
chunk 0: 1MB (152 pages) ... ok
chunk 1: 1023MB (261888 pages) 1008 992 976 960 944 928 912 896 880
864 848 832 816 800 784 768 752 736 720 704 688 672 656 640 624 608
592 576 560 544 528 512 496 480 464 448 432 416 400 384 368 352 336
320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32
16
#0 doadump () at pcpu.h:172
172 __asm __volatile("movq %%gs:0,%0" : "=3Dr" (td));
(kgdb) backtrace
#0 doadump () at pcpu.h:172
#1 0xffffffff802456c3 in boot (howto=3D260) at ../../../kern/kern_shutdown=
.c:402
#2 0xffffffff80245cf7 in panic (fmt=3D0xffffff003dc19be0 "@\203(c)=3D")
at ../../../kern/kern_shutdown.c:558
#3 0xffffffff8017eaa2 in db_panic (addr=3D0, have_addr=3D0, count=3D0,
modif=3D0x0) at ../../../ddb/db_command.c:438
#4 0xffffffff8017efe5 in db_command_loop () at ../../../ddb/db_command.c:3=
50
#5 0xffffffff80180ef3 in db_trap (type=3D-1523653024, code=3D0) at
../../../ddb/db_main.c:221
#6 0xffffffff8026386b in kdb_trap (type=3D12, code=3D0,
tf=3D0xffffffffa52ee7c0) at ../../../kern/subr_kdb.c:473
#7 0xffffffff803909cd in trap_fatal (frame=3D0xffffffffa52ee7c0,
eva=3D18446742975234022368)
at ../../../amd64/amd64/trap.c:651
#8 0xffffffff80390f48 in trap (frame=3D
{tf_rdi =3D -1098532350784, tf_rsi =3D 40, tf_rdx =3D 180, tf_rcx =3D
4294967294, tf_r8 =3D -1098696124736, tf_r9 =3D 0, tf_rax =3D 40, tf_rbx =
=3D
-1098475529248, tf_rbp =3D -1523652432, tf_r10 =3D -1098532350782, tf_r11
=3D 0, tf_r12 =3D -1098532350784, tf_r13 =3D -1098475529248, tf_r14 =3D
-2143307873, tf_r15 =3D 0, tf_trapno =3D 12, tf_addr =3D 72, tf_flags =3D
-1098685787648, tf_err =3D 0, tf_rip =3D -2144938506, tf_cs =3D 8, tf_rflag=
s
=3D 65666, tf_rsp =3D -1523652480, tf_ss =3D 16}) at
../../../amd64/amd64/trap.c:238
#9 0xffffffff8037ee2b in calltrap () at ../../../amd64/amd64/exception.S:1=
68
#10 0xffffffff8026d5f6 in propagate_priority (td=3D0xffffff003a5e94c0)
at ../../../kern/subr_turnstile.c:233
#11 0xffffffff8026de2f in turnstile_wait (lock=3D0xffffffff805710c0,
owner=3D0x0) at ../../../kern/subr_turnstile.c:628
#12 0xffffffff8023b4a9 in _mtx_lock_sleep (m=3D0xffffffff805710c0,
tid=3D18446742975234022368, opts=3D180,
file=3D0xfffffffe <Address 0xfffffffe out of bounds>,
line=3D815503040) at ../../../kern/kern_mutex.c:565
#13 0xffffffff80293f03 in sf_buf_mext (addr=3D0xffffff003a5e94c0,
args=3D0xffffff003f059328)
at ../../../kern/uipc_syscalls.c:1710
#14 0xffffffff80287aa4 in mb_free_ext (m=3D0xffffff003d909600) at
../../../kern/uipc_mbuf.c:272
#15 0xffffffff8028e328 in sbdrop_locked (sb=3D0xffffff000c8ce3c0,
len=3D540) at mbuf.h:486
#16 0xffffffff8029099a in sbdrop (sb=3D0xffffff000c8ce3c0, len=3D1460) at
../../../kern/uipc_socket2.c:1208
#17 0xffffffff802f02de in tcp_input (m=3D0xffffff0029ebe300,
off0=3D668661232) at ../../../netinet/tcp_input.c:1212
#18 0xffffffff802e7e70 in ip_input (m=3D0xffffff0029ebe300) at
../../../netinet/ip_input.c:786
#19 0xffffffff802d3778 in netisr_processqueue (ni=3D0xffffffff8056c290)
at ../../../net/netisr.c:236
#20 0xffffffff802d3a4d in swi_net (dummy=3D0xffffff003a5e94c0) at
../../../net/netisr.c:349
#21 0xffffffff8022c262 in ithread_loop (arg=3D0xffffff00000246c0) at
../../../kern/kern_intr.c:673
#22 0xffffffff8022ad56 in fork_exit (callout=3D0xffffffff8022c100
<ithread_loop>, arg=3D0xffffff00000246c0,
frame=3D0xffffffffa52eec50) at ../../../kern/kern_fork.c:789
#23 0xffffffff8037f18e in fork_trampoline () at
../../../amd64/amd64/exception.S:394
#24 0x0000000000000000 in ?? ()
Previous frame identical to this frame (corrupt stack?)
(kgdb) list *0xffffffff8026d5f6
0xffffffff8026d5f6 is in propagate_priority
(../../../kern/subr_turnstile.c:235).
230 /*
231 * Pick up the lock that td is blocked on.
232 */
233 ts =3D td->td_blocked;
234 MPASS(ts !=3D NULL);
235 tc =3D TC_LOOKUP(ts->ts_lockobj);
236 mtx_lock_spin(&tc->tc_lock);
237
238 /* Resort td on the list if needed. */
239 if (!turnstile_adjust_thread(ts, td)) {
(kgdb)
also, attached additional debug info (config, ddb, kgdb)
On 3/17/06, Joe Talbott <josepht@cstone.net> wrote:
> On Fri, Mar 17, 2006 at 11:41:58AM -0600, Vlad wrote:
> > no, nothing like that. and it reboots several times a day.
> >
> > also, I have my swap twice less than physical mem, so I can't get a
> > dump (btw, there was a patch for to gzip core before it stores it into
> > swap, so it can be fit in swap of smaller size - anyone has it?)
> >
>
> You could try setting hw.physmem to an appropriate size.
>
> Joe
>
--
Vlad
------=_Part_252_5751521.1142650865633
Content-Type: text/plain; name=db_console.txt; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Attachment-Id: f_ekxcqmr8
Content-Disposition: attachment; filename="db_console.txt"
db> bt
Tracing pid 12 tid 100001 td 0xffffff003dc19be0
propagate_priority() at propagate_priority+0x66
turnstile_wait() at turnstile_wait+0x20f
_mtx_lock_sleep() at _mtx_lock_sleep+0x89
sf_buf_mext() at sf_buf_mext+0xa3
mb_free_ext() at mb_free_ext+0x64
sbdrop_locked() at sbdrop_locked+0xb8
sbdrop() at sbdrop+0x4a
tcp_input() at tcp_input+0x14ee
ip_input() at ip_input+0x100
netisr_processqueue() at netisr_processqueue+0x78
swi_net() at swi_net+0x14d
ithread_loop() at ithread_loop+0x162
fork_exit() at fork_exit+0x86
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffffffa52eed00, rbp = 0 ---
db> ps
pid proc uid ppid pgrp flag stat wmesg wchan cmd
4989 ffffff0038cae340 1002 381 381 0002100 [SLPQ user map 0xffffff003b8cc2f0][SLP] httpd
4987 ffffff0003667000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4986 ffffff002c275680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4985 ffffff002a769680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4984 ffffff0028648340 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4983 ffffff002c1f4340 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4982 ffffff000882d000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4981 ffffff000882d680 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4980 ffffff0038cae9c0 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4979 ffffff002c275000 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4977 ffffff0028dcc000 1002 381 381 0000100 [LOCK vm page queue mutex ffffff00309b9680] httpd
4976 ffffff002da40000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4975 ffffff002c275340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4974 ffffff0028beb000 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4973 ffffff0028beb9c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4972 ffffff002dfbb340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4970 ffffff002a769000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4968 ffffff0028dcc680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4966 ffffff002f7ac000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4965 ffffff002c1f49c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4964 ffffff002dd82680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4958 ffffff003ad83000 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4957 ffffff0003667340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4956 ffffff002da259c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4953 ffffff002c1b9340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4952 ffffff002dd82340 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4949 ffffff003a96c340 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4945 ffffff002c1b9680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4943 ffffff0003667680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4942 ffffff003ad83680 1002 381 381 0000100 [SLPQ accept 0xffffff00306b352e][SLP] httpd
4939 ffffff002d684000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4937 ffffff002dbeb340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4935 ffffff0028648680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4930 ffffff002a7699c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4929 ffffff00279c29c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4919 ffffff0035470000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4896 ffffff0008047340 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4894 ffffff002c62d680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4884 ffffff001b9ae680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4880 ffffff000bb49000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4859 ffffff0008047000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4858 ffffff0014d289c0 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4853 ffffff002dfbb680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4851 ffffff002dbeb000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4838 ffffff003998f680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4817 ffffff003a96c680 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4782 ffffff002c1b9000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
4473 ffffff0038cae000 1002 381 381 0000100 [SLPQ select 0xffffffff80569bf0][SLP] httpd
539 ffffff002d684680 0 1 539 0004002 [SLPQ ttyin 0xffffff003d89b010][SLP] getty
538 ffffff002d6849c0 0 1 538 0004002 [SLPQ ttyin 0xffffff0000852c10][SLP] getty
537 ffffff002da25000 0 1 537 0004002 [SLPQ ttyin 0xffffff003d8d7010][SLP] getty
536 ffffff002da25340 0 1 536 0004002 [SLPQ ttyin 0xffffff003d8d6c10][SLP] getty
535 ffffff002da25680 0 1 535 0004002 [SLPQ ttyin 0xffffff003d8d6410][SLP] getty
534 ffffff002e5f9340 0 1 534 0004002 [SLPQ ttyin 0xffffff003d8d6010][SLP] getty
533 ffffff002da40340 0 1 533 0004002 [SLPQ ttyin 0xffffff003d8d3410][SLP] getty
532 ffffff003d856680 0 1 532 0004002 [SLPQ ttyin 0xffffff003d8d3810][SLP] getty
531 ffffff002e5f9680 0 1 531 0004002 [SLPQ ttyin 0xffffff003d8d3c10][SLP] getty
489 ffffff002da40680 0 1 489 0000000 [SLPQ nanslp 0xffffffff80560de0][SLP] cron
480 ffffff002e5f99c0 25 1 480 0000100 [SLPQ pause 0xffffff002e5f9a28][SLP] sendmail
476 ffffff002da409c0 0 1 476 0000100 [SLPQ select 0xffffffff80569bf0][SLP] sendmail
465 ffffff002f96d9c0 0 1 465 0000100 [SLPQ select 0xffffffff80569bf0][SLP] sshd
413 ffffff002f96d000 0 1 413 0000101 [SLPQ select 0xffffffff80569bf0][SLP] proftpd
408 ffffff002f7ac340 0 405 381 0004000 [SLPQ piperd 0xffffff002f2e5600][SLP] cronolog
407 ffffff002f96d680 0 404 381 0004000 [SLPQ piperd 0xffffff003160a000][SLP] cronolog
406 ffffff002f7ac680 0 398 381 0004000 [SLPQ piperd 0xffffff0031609000][SLP] cronolog
405 ffffff003d856000 0 381 381 0004000 [SLPQ wait 0xffffff003d856000][SLP] sh
404 ffffff002f7ac9c0 0 381 381 0004000 [SLPQ wait 0xffffff002f7ac9c0][SLP] sh
403 ffffff003d875000 0 401 381 0004000 [SLPQ piperd 0xffffff0031609600][SLP] cronolog
402 ffffff002f3d5340 0 399 381 0004000 [SLPQ piperd 0xffffff003160a300][SLP] cronolog
401 ffffff003d85d9c0 0 381 381 0004000 [SLPQ wait 0xffffff003d85d9c0][SLP] sh
400 ffffff002f53c9c0 0 397 381 0004000 [SLPQ piperd 0xffffff003160a900][SLP] cronolog
399 ffffff002f53c340 0 381 381 0004000 [SLPQ wait 0xffffff002f53c340][SLP] sh
398 ffffff002f37d000 0 381 381 0004000 [SLPQ wait 0xffffff002f37d000][SLP] sh
397 ffffff002f53c680 0 381 381 0004000 [SLPQ wait 0xffffff002f53c680][SLP] sh
396 ffffff003d875340 0 393 381 0004000 [SLPQ piperd 0xffffff002f2e5000][SLP] cronolog
395 ffffff002f3d59c0 0 390 381 0004000 [SLPQ piperd 0xffffff003160a600][SLP] cronolog
394 ffffff002f96d340 0 389 381 0004000 [SLPQ piperd 0xffffff002f2e5300][SLP] cronolog
393 ffffff002f37d340 0 381 381 0004000 [SLPQ wait 0xffffff002f37d340][SLP] sh
392 ffffff003d877340 0 388 381 0004000 [SLPQ piperd 0xffffff0031609900][SLP] cronolog
391 ffffff002f37d680 0 387 381 0004000 [SLPQ piperd 0xffffff0031609c00][SLP] cronolog
390 ffffff003d877680 0 381 381 0004000 [SLPQ wait 0xffffff003d877680][SLP] sh
389 ffffff002f37d9c0 0 381 381 0004000 [SLPQ wait 0xffffff002f37d9c0][SLP] sh
388 ffffff003d8779c0 0 381 381 0004000 [SLPQ wait 0xffffff003d8779c0][SLP] sh
387 ffffff002f53c000 0 381 381 0004000 [SLPQ wait 0xffffff002f53c000][SLP] sh
381 ffffff002f3d5680 0 1 381 0000000 [SLPQ nanslp 0xffffffff80560de0][SLP] httpd
298 ffffff003d875680 0 1 298 0000000 [SLPQ select 0xffffffff80569bf0][SLP] rpcbind
283 ffffff003d856340 0 1 283 0000000 [SLPQ select 0xffffffff80569bf0][SLP] syslogd
248 ffffff003d8759c0 0 1 248 0000000 [SLPQ select 0xffffffff80569bf0][SLP] devd
37 ffffff003d8569c0 0 0 0 0000204 [SLPQ - 0xffffffffa5384be4][SLP] schedcpu
36 ffffff003d877000 0 0 0 0000204 [SLPQ - 0xffffffff8056f978][SLP] nfsiod 3
35 ffffff003dbff680 0 0 0 0000204 [SLPQ - 0xffffffff8056f970][SLP] nfsiod 2
34 ffffff003dbff9c0 0 0 0 0000204 [SLPQ - 0xffffffff8056f968][SLP] nfsiod 1
33 ffffff003d85c000 0 0 0 0000204 [SLPQ - 0xffffffff8056f960][SLP] nfsiod 0
32 ffffff003d85c340 0 0 0 0000204 [SLPQ sdflush 0xffffffff80570120][SLP] softdepflush
31 ffffff003d85c680 0 0 0 0000204 [SLPQ syncer 0xffffffff805609c0][SLP] syncer
30 ffffff003d85c9c0 0 0 0 0000204 [SLPQ vlruwt 0xffffff003d85c9c0][SLP] vnlru
29 ffffff003d85d000 0 0 0 0000204 [SLPQ psleep 0xffffffff8056a4b8][SLP] bufdaemon
28 ffffff003d85d340 0 0 0 000020c [SLPQ pgzero 0xffffffff80571aa0][SLP] pagezero
27 ffffff003d85d680 0 0 0 0000204 [SLPQ psleep 0xffffffff8057116c][SLP] vmdaemon
26 ffffff003dbcc680 0 0 0 0000204 [SLPQ psleep 0xffffffff8057111c][SLP] pagedaemon
25 ffffff003dbcc9c0 0 0 0 0000204 [IWAIT] swi0: sio
24 ffffff003dc1e000 0 0 0 0000204 [IWAIT] irq10: bge0
23 ffffff003dc1e340 0 0 0 0000204 [IWAIT] irq5: nve0
22 ffffff003dc1e680 0 0 0 0000204 [IWAIT] irq11: amr0
21 ffffff003dc1e9c0 0 0 0 0000204 [IWAIT] irq15: ata1
20 ffffff003dbff000 0 0 0 0000204 [IWAIT] irq14: ata0
19 ffffff003dbff340 0 0 0 0000204 [IWAIT] swi2: cambio
18 ffffff003dba99c0 0 0 0 0000204 [IWAIT] swi6: task queue
17 ffffff003db89000 0 0 0 0000204 [IWAIT] swi6: Giant taskq
9 ffffff003db89340 0 0 0 0000204 [SLPQ - 0xffffff0000758e00][SLP] thread taskq
16 ffffff003db89680 0 0 0 0000204 [IWAIT] swi5: +
8 ffffff003db899c0 0 0 0 0000204 [SLPQ - 0xffffff00007c8200][SLP] kqueue taskq
7 ffffff003dbcc000 0 0 0 0000204 [SLPQ - 0xffffff00007c8300][SLP] acpi_task_2
6 ffffff003dbcc340 0 0 0 0000204 [SLPQ - 0xffffff00007c8300][SLP] acpi_task_1
5 ffffff003dc1d340 0 0 0 0000204 [SLPQ - 0xffffff00007c8300][SLP] acpi_task_0
15 ffffff003dc1d680 0 0 0 0000204 [SLPQ - 0xffffffff8055b7c0][SLP] yarrow
4 ffffff003dc1d9c0 0 0 0 0000204 [SLPQ - 0xffffffff8055c488][SLP] g_down
3 ffffff003dba9000 0 0 0 0000204 [SLPQ - 0xffffffff8055c480][SLP] g_up
2 ffffff003dba9340 0 0 0 0000204 [SLPQ - 0xffffffff8055c470][SLP] g_event
14 ffffff003dba9680 0 0 0 0000204 [IWAIT] swi3: vm
13 ffffff003dbf8000 0 0 0 000020c [IWAIT] swi4: clock sio
12 ffffff003dbf8340 0 0 0 0000204 [LOCK vm page queue mutex ffffff00309b9680] swi1: net
11 ffffff003dbf8680 0 0 0 000020c [Can run] idle
1 ffffff003dbf89c0 0 0 1 0004200 [SLPQ wait 0xffffff003dbf89c0][SLP] init
10 ffffff003dc1d000 0 0 0 0000204 [SLPQ ktrace 0xffffffff8055d580][SLP] ktrace
0 ffffffff8055c5e0 0 0 0 0000200 [IWAIT] swapper
db> show reg
cs 0x8
ss 0x10
rax 0x28
rcx 0xfffffffe
rdx 0xb4
rbx 0xffffff003dc19be0
rsp 0xffffffffa52ee870
rbp 0xffffffffa52ee8b0
rsi 0x28
rdi 0xffffff003a5e94c0
r8 0xffffff00309b96c0
r9 0
r10 0xffffff003a5e94c2
r11 0
r12 0xffffff003a5e94c0
r13 0xffffff003dc19be0
r14 0xffffffff803fb79f vm_object_check_cmd+0x13f
r15 0
rip 0xffffffff8026d5f6 propagate_priority+0x66
rflags 0x10082
dr0 0
dr1 0
dr2 0
dr3 0
dr4 0xffff0ff0
dr5 0x400
dr6 0xffff0ff0
dr7 0x400
propagate_priority+0x66: movq 0x48(%r15),%rdi
db> panic
panic: from debugger
Uptime: 4h15m12s
Dumping 1023 MB (2 chunks)
chunk 0: 1MB (152 pages) ... ok
chunk 1: 1023MB (261888 pages) 1008 992 976 960 944 928 912 896 880 864 848 832 816 800 784 768 752 736 720 704 688 672 656k
Dump complete
Automatic reboot in 15 seconds - press a key on the console to abort
------=_Part_252_5751521.1142650865633
Content-Type: text/plain; name=kernel_config.txt; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Attachment-Id: f_ekxcr044
Content-Disposition: attachment; filename="kernel_config.txt"
machine amd64
cpu HAMMER
ident DEF_WEB
# To statically compile in device wiring instead of /boot/device.hints
#hints "GENERIC.hints" # Default places to look for devices.
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
options DDB
options KDB
#options SCHED_ULE # ULE scheduler
options SCHED_4BSD # 4BSD scheduler
options PREEMPTION # Enable kernel thread preemption
options INET # InterNETworking
#options INET6 # IPv6 communications protocols
options FFS # Berkeley Fast Filesystem
options SOFTUPDATES # Enable FFS soft updates support
options UFS_ACL # Support for access control lists
options UFS_DIRHASH # Improve performance on big directories
options MD_ROOT # MD is a potential root device
options NFSCLIENT # Network Filesystem Client
#options NFSSERVER # Network Filesystem Server
options NFS_ROOT # NFS usable as /, requires NFSCLIENT
#options NTFS # NT File System
options MSDOSFS # MSDOS Filesystem
options CD9660 # ISO 9660 Filesystem
options PROCFS # Process filesystem (requires PSEUDOFS)
options PSEUDOFS # Pseudo-filesystem framework
options GEOM_GPT # GUID Partition Tables.
options COMPAT_43 # Needed by COMPAT_LINUX32
options COMPAT_IA32 # Compatible with i386 binaries
#options COMPAT_FREEBSD4 # Compatible with FreeBSD4
options COMPAT_FREEBSD5 # Compatible with FreeBSD5
#options COMPAT_LINUX32 # Compatible with i386 linux binaries
options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI
options KTRACE # ktrace(1) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options KBD_INSTALL_CDEV # install a CDEV entry in /dev
options AHC_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~128k to driver.
options AHD_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~215k to driver.
options ADAPTIVE_GIANT # Giant mutex is adaptive.
# Workarounds for some known-to-be-broken chipsets (nVidia nForce3-Pro150)
device atpic # 8259A compatability
# Linux 32-bit ABI support
#options LINPROCFS # Cannot be a module yet.
# Bus support.
device acpi
device pci
# ATA and ATAPI devices
device ata
device atadisk # ATA disk drives
device ataraid # ATA RAID drives
device atapicd # ATAPI CDROM drives
device atapifd # ATAPI floppy drives
device atapist # ATAPI tape drives
options ATA_STATIC_ID # Static device numbering
# SCSI peripherals
device scbus # SCSI bus (required for SCSI)
device ch # SCSI media changers
device da # Direct Access (disks)
device sa # Sequential Access (tape etc)
device cd # CD
device pass # Passthrough device (direct SCSI access)
device ses # SCSI Environmental Services (and SAF-TE)
# RAID controllers interfaced to the SCSI subsystem
device amr # AMI MegaRAID
# atkbdc0 controls both the keyboard and the PS/2 mouse
device atkbdc # AT keyboard controller
device atkbd # AT keyboard
device psm # PS/2 mouse
device vga # VGA video card driver
device splash # Splash screen and screen saver support
# syscons is the default console driver, resembling an SCO console
device sc
device agp # support several AGP chipsets
# Serial (COM) ports
device sio # 8250, 16[45]50 based serial ports
# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device miibus # MII bus support
device bge # Broadcom BCM570xx Gigabit Ethernet
device nve # nVidia nForce MCP on-board Ethernet Networking
# Pseudo devices.
device loop # Network loopback
device random # Entropy device
device ether # Ethernet support
device sl # Kernel SLIP
device ppp # Kernel PPP
device tun # Packet tunnel.
device pty # Pseudo-ttys (telnet etc)
device md # Memory "disks"
# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device bpf # Berkeley packet filter
device nve
options CONSPEED=115200
------=_Part_252_5751521.1142650865633
Content-Type: text/plain; name=kgdb.txt; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Attachment-Id: f_ekxcr2qr
Content-Disposition: attachment; filename="kgdb.txt"
# kgdb kernel.debug /var/crash/vmcore.0
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd".
Unread portion of the kernel message buffer:
kernel trap 12 with interrupts disabled
Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x48
fault code = supervisor read, page not present
instruction pointer = 0x8:0xffffffff8026d5f6
stack pointer = 0x10:0xffffffffa52ee870
frame pointer = 0x10:0xffffffffa52ee8b0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = resume, IOPL = 0
current process = 12 (swi1: net)
panic: from debugger
Uptime: 4h15m12s
Dumping 1023 MB (2 chunks)
chunk 0: 1MB (152 pages) ... ok
chunk 1: 1023MB (261888 pages) 1008 992 976 960 944 928 912 896 880 864 848 832 816 800 784 768 752 736 720 704 688 672 656 640 624 608 592 576 560 544 528 512 496 480 464 448 432 416 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16
#0 doadump () at pcpu.h:172
172 __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) backtrace
#0 doadump () at pcpu.h:172
#1 0xffffffff802456c3 in boot (howto=260) at ../../../kern/kern_shutdown.c:402
#2 0xffffffff80245cf7 in panic (fmt=0xffffff003dc19be0 "@\203c=") at ../../../kern/kern_shutdown.c:558
#3 0xffffffff8017eaa2 in db_panic (addr=0, have_addr=0, count=0, modif=0x0) at ../../../ddb/db_command.c:438
#4 0xffffffff8017efe5 in db_command_loop () at ../../../ddb/db_command.c:350
#5 0xffffffff80180ef3 in db_trap (type=-1523653024, code=0) at ../../../ddb/db_main.c:221
#6 0xffffffff8026386b in kdb_trap (type=12, code=0, tf=0xffffffffa52ee7c0) at ../../../kern/subr_kdb.c:473
#7 0xffffffff803909cd in trap_fatal (frame=0xffffffffa52ee7c0, eva=18446742975234022368)
at ../../../amd64/amd64/trap.c:651
#8 0xffffffff80390f48 in trap (frame=
{tf_rdi = -1098532350784, tf_rsi = 40, tf_rdx = 180, tf_rcx = 4294967294, tf_r8 = -1098696124736, tf_r9 = 0, tf_rax = 40, tf_rbx = -1098475529248, tf_rbp = -1523652432, tf_r10 = -1098532350782, tf_r11 = 0, tf_r12 = -1098532350784, tf_r13 = -1098475529248, tf_r14 = -2143307873, tf_r15 = 0, tf_trapno = 12, tf_addr = 72, tf_flags = -1098685787648, tf_err = 0, tf_rip = -2144938506, tf_cs = 8, tf_rflags = 65666, tf_rsp = -1523652480, tf_ss = 16}) at ../../../amd64/amd64/trap.c:238
#9 0xffffffff8037ee2b in calltrap () at ../../../amd64/amd64/exception.S:168
#10 0xffffffff8026d5f6 in propagate_priority (td=0xffffff003a5e94c0) at ../../../kern/subr_turnstile.c:233
#11 0xffffffff8026de2f in turnstile_wait (lock=0xffffffff805710c0, owner=0x0) at ../../../kern/subr_turnstile.c:628
#12 0xffffffff8023b4a9 in _mtx_lock_sleep (m=0xffffffff805710c0, tid=18446742975234022368, opts=180,
file=0xfffffffe <Address 0xfffffffe out of bounds>, line=815503040) at ../../../kern/kern_mutex.c:565
#13 0xffffffff80293f03 in sf_buf_mext (addr=0xffffff003a5e94c0, args=0xffffff003f059328)
at ../../../kern/uipc_syscalls.c:1710
#14 0xffffffff80287aa4 in mb_free_ext (m=0xffffff003d909600) at ../../../kern/uipc_mbuf.c:272
#15 0xffffffff8028e328 in sbdrop_locked (sb=0xffffff000c8ce3c0, len=540) at mbuf.h:486
#16 0xffffffff8029099a in sbdrop (sb=0xffffff000c8ce3c0, len=1460) at ../../../kern/uipc_socket2.c:1208
#17 0xffffffff802f02de in tcp_input (m=0xffffff0029ebe300, off0=668661232) at ../../../netinet/tcp_input.c:1212
#18 0xffffffff802e7e70 in ip_input (m=0xffffff0029ebe300) at ../../../netinet/ip_input.c:786
#19 0xffffffff802d3778 in netisr_processqueue (ni=0xffffffff8056c290) at ../../../net/netisr.c:236
#20 0xffffffff802d3a4d in swi_net (dummy=0xffffff003a5e94c0) at ../../../net/netisr.c:349
#21 0xffffffff8022c262 in ithread_loop (arg=0xffffff00000246c0) at ../../../kern/kern_intr.c:673
#22 0xffffffff8022ad56 in fork_exit (callout=0xffffffff8022c100 <ithread_loop>, arg=0xffffff00000246c0,
frame=0xffffffffa52eec50) at ../../../kern/kern_fork.c:789
#23 0xffffffff8037f18e in fork_trampoline () at ../../../amd64/amd64/exception.S:394
#24 0x0000000000000000 in ?? ()
Previous frame identical to this frame (corrupt stack?)
(kgdb) list *0xffffffff8026d5f6
0xffffffff8026d5f6 is in propagate_priority (../../../kern/subr_turnstile.c:235).
230 /*
231 * Pick up the lock that td is blocked on.
232 */
233 ts = td->td_blocked;
234 MPASS(ts != NULL);
235 tc = TC_LOOKUP(ts->ts_lockobj);
236 mtx_lock_spin(&tc->tc_lock);
237
238 /* Resort td on the list if needed. */
239 if (!turnstile_adjust_thread(ts, td)) {
(kgdb)
------=_Part_252_5751521.1142650865633--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cd70c6810603171901t64cf8aectebd9c288164b32b0>