Date: Tue, 8 Jun 2004 11:18:10 +0700 From: Charlie Root <root@swp.bspu.secna.ru> To: freebsd-net@freebsd.org Subject: kernel panic by mars_nwe (nwserv -k) Message-ID: <20040608041810.GA1077@swp.bspu.secna.ru>
next in thread | raw e-mail | index | archive | help
helo.
kernel panic for shutdown of mars_nwe...
bash-2.05b# gdb -k kernel /var/crash/vmcore.0
GNU gdb 5.2.1 (FreeBSD)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-unknown-freebsd"...
(no debugging symbols found)...
panic: mutex rtentry not owned at ../../../net/route.c:225
panic messages:
---
panic: mutex rtentry not owned at ../../../net/route.c:225
Stack backtrace:
panic: from debugger
Uptime: 57m26s
Dumping 255 MB
16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
---
warning: cannot find file for module vesa.ko
warning: cannot find file for module if_ef.ko
warning: cannot find file for module miibus.ko
warning: cannot find file for module if_fxp.ko
warning: cannot find file for module snd_pcm.ko
warning: cannot find file for module snd_es137x.ko
warning: cannot find file for module random.ko
warning: cannot find file for module acpi.ko
warning: cannot find file for module fdc.ko
warning: cannot find file for module speaker.ko
warning: cannot find file for module vinum.ko
warning: cannot find file for module procfs.ko
warning: cannot find file for module pseudofs.ko
warning: cannot find file for module linprocfs.ko
warning: cannot find file for module linux.ko
warning: cannot find file for module sysvmsg.ko
warning: cannot find file for module sysvsem.ko
warning: cannot find file for module sysvshm.ko
warning: cannot find file for module ipl.ko
warning: cannot find file for module nfsclient.ko
warning: cannot find file for module nfsserver.ko
warning: cannot find file for module usb.ko
warning: cannot find file for module snake_saver.ko
warning: cannot find file for module vmmon_up.ko
warning: cannot find file for module vmnet.ko
warning: cannot find file for module if_tap.ko
warning: cannot find file for module netgraph.ko
warning: cannot find file for module ng_socket.ko
warning: cannot find file for module ng_ether.ko
warning: cannot find file for module ng_tee.ko
warning: cannot find file for module ng_bridge.ko
warning: cannot find file for module rtc.ko
Error while mapping shared library sections:
vesa.ko: No such file or directory.
Error while mapping shared library sections:
if_ef.ko: Unknown error: 0.
Error while mapping shared library sections:
miibus.ko: Unknown error: 0.
Error while mapping shared library sections:
if_fxp.ko: Unknown error: 0.
Error while mapping shared library sections:
snd_pcm.ko: Unknown error: 0.
Error while mapping shared library sections:
snd_es137x.ko: Unknown error: 0.
Error while mapping shared library sections:
random.ko: Unknown error: 0.
Error while mapping shared library sections:
acpi.ko: Unknown error: 0.
Error while mapping shared library sections:
fdc.ko: Unknown error: 0.
Error while mapping shared library sections:
speaker.ko: Unknown error: 0.
Error while mapping shared library sections:
vinum.ko: Unknown error: 0.
Error while mapping shared library sections:
procfs.ko: Unknown error: 0.
Error while mapping shared library sections:
pseudofs.ko: Unknown error: 0.
Error while mapping shared library sections:
linprocfs.ko: Unknown error: 0.
Error while mapping shared library sections:
linux.ko: Unknown error: 0.
Error while mapping shared library sections:
sysvmsg.ko: Unknown error: 0.
Error while mapping shared library sections:
sysvsem.ko: Unknown error: 0.
Error while mapping shared library sections:
sysvshm.ko: Unknown error: 0.
Error while mapping shared library sections:
ipl.ko: Unknown error: 0.
Error while mapping shared library sections:
nfsclient.ko: Unknown error: 0.
Error while mapping shared library sections:
nfsserver.ko: Unknown error: 0.
Error while mapping shared library sections:
usb.ko: Unknown error: 0.
Error while mapping shared library sections:
snake_saver.ko: Unknown error: 0.
Error while mapping shared library sections:
vmmon_up.ko: Unknown error: 0.
Error while mapping shared library sections:
vmnet.ko: Unknown error: 0.
Error while mapping shared library sections:
if_tap.ko: Unknown error: 0.
Error while mapping shared library sections:
netgraph.ko: Unknown error: 0.
Error while mapping shared library sections:
ng_socket.ko: Unknown error: 0.
Error while mapping shared library sections:
ng_ether.ko: Unknown error: 0.
Error while mapping shared library sections:
ng_tee.ko: Unknown error: 0.
Error while mapping shared library sections:
ng_bridge.ko: Unknown error: 0.
Error while mapping shared library sections:
rtc.ko: Unknown error: 0.
Error while reading shared library symbols:
vesa.ko: No such file or directory.
Error while reading shared library symbols:
if_ef.ko: No such file or directory.
Error while reading shared library symbols:
miibus.ko: No such file or directory.
Error while reading shared library symbols:
if_fxp.ko: No such file or directory.
Error while reading shared library symbols:
snd_pcm.ko: No such file or directory.
Error while reading shared library symbols:
snd_es137x.ko: No such file or directory.
Error while reading shared library symbols:
random.ko: No such file or directory.
Error while reading shared library symbols:
acpi.ko: No such file or directory.
Error while reading shared library symbols:
fdc.ko: No such file or directory.
Error while reading shared library symbols:
speaker.ko: No such file or directory.
Error while reading shared library symbols:
vinum.ko: No such file or directory.
Error while reading shared library symbols:
procfs.ko: No such file or directory.
Error while reading shared library symbols:
pseudofs.ko: No such file or directory.
Error while reading shared library symbols:
linprocfs.ko: No such file or directory.
Error while reading shared library symbols:
linux.ko: No such file or directory.
Error while reading shared library symbols:
sysvmsg.ko: No such file or directory.
Error while reading shared library symbols:
sysvsem.ko: No such file or directory.
Error while reading shared library symbols:
sysvshm.ko: No such file or directory.
Error while reading shared library symbols:
ipl.ko: No such file or directory.
Error while reading shared library symbols:
nfsclient.ko: No such file or directory.
Error while reading shared library symbols:
nfsserver.ko: No such file or directory.
Error while reading shared library symbols:
usb.ko: No such file or directory.
Error while reading shared library symbols:
snake_saver.ko: No such file or directory.
Error while reading shared library symbols:
vmmon_up.ko: No such file or directory.
Error while reading shared library symbols:
vmnet.ko: No such file or directory.
Error while reading shared library symbols:
if_tap.ko: No such file or directory.
Error while reading shared library symbols:
netgraph.ko: No such file or directory.
Error while reading shared library symbols:
ng_socket.ko: No such file or directory.
Error while reading shared library symbols:
ng_ether.ko: No such file or directory.
Error while reading shared library symbols:
ng_tee.ko: No such file or directory.
Error while reading shared library symbols:
ng_bridge.ko: No such file or directory.
Error while reading shared library symbols:
rtc.ko: No such file or directory.
#0 0xc04c567b in doadump ()
(kgdb) file kernel.debug
Reading symbols from kernel.debug...done.
(kgdb) where
#0 doadump () at ../../../kern/kern_shutdown.c:240
#1 0xc04c5ce5 in boot (howto=260) at ../../../kern/kern_shutdown.c:372
#2 0xc04c6090 in panic (fmt=0xc0657095 "from debugger")
at ../../../kern/kern_shutdown.c:550
#3 0xc043a732 in db_panic (addr=-1067460476, have_addr=0, count=-1,
modif=0xce67896c "") at ../../../ddb/db_command.c:450
#4 0xc043a692 in db_command (last_cmdp=0xc06aa180, cmd_table=0xce678940,
aux_cmd_tablep=0xc0657095, aux_cmd_tablep_end=0x0)
at ../../../ddb/db_command.c:346
#5 0xc043a7d5 in db_command_loop () at ../../../ddb/db_command.c:472
#6 0xc043d7e5 in db_trap (type=3, code=0) at ../../../ddb/db_trap.c:73
#7 0xc05fd5cc in kdb_trap (type=3, code=0, regs=0xce678abc)
at ../../../i386/i386/db_interface.c:171
#8 0xc060d92a in trap (frame=
{tf_fs = -832110568, tf_es = -1067515888, tf_ds = 16, tf_edi = 1, tf_esi = -1067046359, tf_ebp = -832075000, tf_isp = -832075032, tf_ebx = 0, tf_edx = 0, tf_ecx = -1066644000, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1067460476, tf_cs = 8, tf_eflags = 646, tf_esp = -1066959182, tf_ss = -1067043737})
at ../../../i386/i386/trap.c:580
#9 0xc05fef98 in calltrap () at {standard input}:94
#10 0xc04c6015 in panic (fmt=0xc0662a29 "mutex %s not owned at %s:%d")
at ../../../kern/kern_shutdown.c:534
#11 0xc04bc1b5 in _mtx_assert (m=0xc0662a29, what=0,
file=0xc066a143 "../../../net/route.c", line=1)
---Type <return> to continue, or q <return> to quit---
at ../../../kern/kern_mutex.c:654
#12 0xc0534fb3 in rtfree (rt=0xc0662a29) at ../../../net/route.c:225
#13 0xc05774a1 in ipx_pcbdetach (ipxp=0xc2554080)
at ../../../netipx/ipx_pcb.c:274
#14 0xc0577dbd in ipx_detach (so=0x0) at ../../../netipx/ipx_usrreq.c:492
#15 0xc04fe9f2 in soclose (so=0xc2138d20) at ../../../kern/uipc_socket.c:379
#16 0xc04f3a7b in soo_close (fp=0x0, td=0xc1d5e500)
at ../../../kern/sys_socket.c:244
#17 0xc04a8a59 in fdrop_locked (fp=0xc2149d8c, td=0x0)
at ../../../sys/file.h:292
#18 0xc04a7a5e in fdrop (fp=0xc2149d8c, td=0x0)
at ../../../kern/kern_descrip.c:1829
#19 0xc04a7a0c in closef (fp=0xc2149d8c, td=0xc1d5e500)
at ../../../kern/kern_descrip.c:1815
#20 0xc04a5b38 in close (td=0xc1d5e500, uap=0x0)
at ../../../kern/kern_descrip.c:862
#21 0xc060e2a0 in syscall (frame=
{tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 1, tf_esi = 0, tf_ebp = -1077941576, tf_isp = -832074380, tf_ebx = 2, tf_edx = -1077943056, tf_ecx = 672561504, tf_eax = 6, tf_trapno = 12, tf_err = 2, tf_eip = 672052015, tf_cs = 31, tf_eflags = 642, tf_esp = -1077941588, tf_ss = 47})
at ../../../i386/i386/trap.c:1010
#22 0xc05fefed in Xint0x80_syscall () at {standard input}:136
---Can't read userspace from dump, or kernel process---
(kgdb) where full
#0 doadump () at ../../../kern/kern_shutdown.c:240
No locals.
#1 0xc04c5ce5 in boot (howto=260) at ../../../kern/kern_shutdown.c:372
No locals.
#2 0xc04c6090 in panic (fmt=0xc0657095 "from debugger")
at ../../../kern/kern_shutdown.c:550
td = (struct thread *) 0xc1d5e500
bootopt = 260
newpanic = 0
ap = 0xce678940 "ø\211gÎ\222¦CÀ\204Ø_À"
buf = "mutex rtentry not owned at ../../../net/route.c:225", '\0' <repeats 204 times>
#3 0xc043a732 in db_panic (addr=-1067460476, have_addr=0, count=-1,
modif=0xce67896c "") at ../../../ddb/db_command.c:450
No locals.
#4 0xc043a692 in db_command (last_cmdp=0xc06aa180, cmd_table=0xce678940,
aux_cmd_tablep=0xc0657095, aux_cmd_tablep_end=0x0)
at ../../../ddb/db_command.c:346
cmd = (struct command *) 0xc0622e40
t = 0
modif = "\0ªjÀ\0\0\0\0\210\211gÎ\r\0\0\0àklÀ\r\0\0\0\001\0\0\0¨\211gÎÖ3_À`QlÀ\aK\0 dllÀàVkÀàªjÀx\0\0\0àªjÀ\0\0\0\0Ì\211gÎÁÅCÀ(¤eÀpÃCÀ\0\0\0\0\020\0\0\0\0\0\0\0àªjÀÖ¼CÀàªjÀ ¢jÀx\0\0\0\003\0\0"
addr = -1067460476
---Type <return> to continue, or q <return> to quit---
count = -1
have_addr = 0
result = 0
#5 0xc043a7d5 in db_command_loop () at ../../../ddb/db_command.c:472
No locals.
#6 0xc043d7e5 in db_trap (type=3, code=0) at ../../../ddb/db_trap.c:73
bkpt = 0
#7 0xc05fd5cc in kdb_trap (type=3, code=0, regs=0xce678abc)
at ../../../i386/i386/db_interface.c:171
ef = 70
ddb_mode = 1
#8 0xc060d92a in trap (frame=
{tf_fs = -832110568, tf_es = -1067515888, tf_ds = 16, tf_edi = 1, tf_esi = -1067046359, tf_ebp = -832075000, tf_isp = -832075032, tf_ebx = 0, tf_edx = 0, tf_ecx = -1066644000, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1067460476, tf_cs = 8, tf_eflags = 646, tf_esp = -1066959182, tf_ss = -1067043737})
at ../../../i386/i386/trap.c:580
td = (struct thread *) 0xc1d5e500
p = (struct proc *) 0xc2086a98
sticks = 0
i = 0
ucode = 0
type = 3
code = 0
---Type <return> to continue, or q <return> to quit---
eva = 0
#9 0xc05fef98 in calltrap () at {standard input}:94
No locals.
#10 0xc04c6015 in panic (fmt=0xc0662a29 "mutex %s not owned at %s:%d")
at ../../../kern/kern_shutdown.c:534
td = (struct thread *) 0xc1d5e500
bootopt = 256
newpanic = 1
ap = 0x0
buf = "mutex rtentry not owned at ../../../net/route.c:225", '\0' <repeats 204 times>
#11 0xc04bc1b5 in _mtx_assert (m=0xc0662a29, what=0,
file=0xc066a143 "../../../net/route.c", line=1)
at ../../../kern/kern_mutex.c:654
No locals.
#12 0xc0534fb3 in rtfree (rt=0xc0662a29) at ../../../net/route.c:225
rnh = (struct radix_node_head *) 0x1
#13 0xc05774a1 in ipx_pcbdetach (ipxp=0xc2554080)
at ../../../netipx/ipx_pcb.c:274
so = (struct socket *) 0x0
#14 0xc0577dbd in ipx_detach (so=0x0) at ../../../netipx/ipx_usrreq.c:492
ipxp = (struct ipxpcb *) 0x0
#15 0xc04fe9f2 in soclose (so=0xc2138d20) at ../../../kern/uipc_socket.c:379
error2 = 0
---Type <return> to continue, or q <return> to quit---
error = 0
#16 0xc04f3a7b in soo_close (fp=0x0, td=0xc1d5e500)
at ../../../kern/sys_socket.c:244
error = 0
so = (struct socket *) 0x0
#17 0xc04a8a59 in fdrop_locked (fp=0xc2149d8c, td=0x0)
at ../../../sys/file.h:292
lf = {l_start = 7523290432, l_len = 1051199940862,
l_pid = -1056791004, l_type = -18016, l_whence = -16125}
vp = (struct vnode *) 0x0
error = -1042946816
#18 0xc04a7a5e in fdrop (fp=0xc2149d8c, td=0x0)
at ../../../kern/kern_descrip.c:1829
No locals.
#19 0xc04a7a0c in closef (fp=0xc2149d8c, td=0xc1d5e500)
at ../../../kern/kern_descrip.c:1815
vp = (struct vnode *) 0x0
lf = {l_start = -4582963836952247460, l_len = -4590368889683604296,
l_pid = -1034551244, l_type = 1, l_whence = 0}
fdtol = (struct filedesc_to_leader *) 0xc2149d8c
fdp = (struct filedesc *) 0x155
#20 0xc04a5b38 in close (td=0xc1d5e500, uap=0x0)
at ../../../kern/kern_descrip.c:862
fdp = (struct filedesc *) 0xc2560000
---Type <return> to continue, or q <return> to quit---
fp = (struct file *) 0xc2149d8c
fd = 6
error = 6
holdleaders = 0
#21 0xc060e2a0 in syscall (frame=
{tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 1, tf_esi = 0, tf_ebp = -1077941576, tf_isp = -832074380, tf_ebx = 2, tf_edx = -1077943056, tf_ecx = 672561504, tf_eax = 6, tf_trapno = 12, tf_err = 2, tf_eip = 672052015, tf_cs = 31, tf_eflags = 642, tf_esp = -1077941588, tf_ss = 47})
at ../../../i386/i386/trap.c:1010
params = 0xbfbfeab0---Can't read userspace from dump, or kernel process---
(kgdb) up
No stack.
(kgdb) help
List of classes of commands:
aliases -- Aliases of other commands
breakpoints -- Making program stop at certain points
data -- Examining data
files -- Specifying and examining files
internals -- Maintenance commands
obscure -- Obscure features
running -- Running the program
stack -- Examining the stack
status -- Status inquiries
support -- Support facilities
tracepoints -- Tracing of program execution without stopping the program
user-defined -- User-defined commands
Type "help" followed by a class name for a list of commands in that class.
Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(kgdb) help stack
Examining the stack.
The stack is made up of stack frames. Gdb assigns numbers to stack frames
counting from zero for the innermost (currently executing) frame.
At any time gdb identifies one frame as the "selected" frame.
Variable lookups are done with respect to the selected frame.
When the program being debugged stops, gdb selects the innermost frame.
The commands below can be used to select other frames by number or address.
List of commands:
backtrace -- Print backtrace of all stack frames
bt -- Print backtrace of all stack frames
down -- Select and print stack frame called by this one
frame -- Select and print a stack frame
return -- Make selected stack frame return to its caller
select-frame -- Select a stack frame without printing anything
up -- Select and print stack frame that called this one
Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(kgdb) return
No selected frame.
(kgdb) help select-frame
Select a stack frame without printing anything.
An argument specifies the frame to select.
It can be a stack frame number or the address of the frame.
(kgdb) select-frame 10
(kgdb) up
#11 0xc04bc1b5 in _mtx_assert (m=0xc0662a29, what=0,
file=0xc066a143 "../../../net/route.c", line=1)
at ../../../kern/kern_mutex.c:654
654 panic("mutex %s not owned at %s:%d",
(kgdb) p m
$1 = (struct mtx *) 0xc0662a29
(kgdb) p *m
$2 = {mtx_object = {lo_class = 0x6574756d, lo_name = 0x73252078---Can't read userspace from dump, or kernel process---
(kgdb) list
649 switch (what) {
650 case MA_OWNED:
651 case MA_OWNED | MA_RECURSED:
652 case MA_OWNED | MA_NOTRECURSED:
653 if (!mtx_owned(m))
654 panic("mutex %s not owned at %s:%d",
655 m->mtx_object.lo_name, file, line);
656 if (mtx_recursed(m)) {
657 if ((what & MA_NOTRECURSED) != 0)
658 panic("mutex %s recursed at %s:%d",
(kgdb) up
#12 0xc0534fb3 in rtfree (rt=0xc0662a29) at ../../../net/route.c:225
225 RT_LOCK_ASSERT(rt);
(kgdb) p *rt
$3 = {rt_nodes = {{rn_mklist = 0x6574756d, rn_parent = 0x73252078,
rn_bit = 28192, rn_bmask = 111 'o', rn_flags = 116 't', rn_u = {
rn_leaf = {rn_Key = 0x6e776f20---Can't read userspace from dump, or kernel process---
(kgdb) list
220 struct radix_node_head *rnh = rt_tables[rt_key(rt)->sa_family];
221
222 if (rt == 0 || rnh == 0)
223 panic("rtfree");
224
225 RT_LOCK_ASSERT(rt);
226
227 /*
228 * decrement the reference count by one and if it reaches 0,
229 * and there is a close function defined, call the close function
(kgdb) up
#13 0xc05774a1 in ipx_pcbdetach (ipxp=0xc2554080)
at ../../../netipx/ipx_pcb.c:274
274 rtfree(ipxp->ipxp_route.ro_rt);
(kgdb) p *ipxp_route
No symbol "ipxp_route" in current context.
(kgdb) p *ipxp
$4 = {ipxp_next = 0xc2004680, ipxp_prev = 0xc1f52400, ipxp_head = 0x0,
ipxp_socket = 0xc2138d20, ipxp_faddr = {x_net = {c_net = "\0\0\0", s_net = {
0, 0}}, x_host = {c_host = "\0\0\0\0\0", s_host = {0, 0, 0}},
x_port = 0}, ipxp_laddr = {x_net = {c_net = " \004\006\005", s_net = {
1056, 1286}}, x_host = {c_host = "\0\0\0\0\0\001", s_host = {0, 0,
256}}, x_port = 832}, ipxp_pcb = 0x0, ipxp_route = {
ro_rt = 0xc1d60900, ro_dst = {sa_len = 16 '\020', sa_family = 23 '\027',
sa_data = " \004\006\005\0\0\0\0\0\001\0\0\0"}}, ipxp_lastdst = {
x_net = {c_net = " \004\006\005", s_net = {1056, 1286}}, x_host = {
c_host = "\0\0\0\0\0\001", s_host = {0, 0, 256}}, x_port = 1088},
ipxp_notify_param = 0, ipxp_flags = 2, ipxp_dpt = 17 '\021',
ipxp_rpt = 17 '\021'}
(kgdb) p *ipxp->ipxp_route.ro_rt
$5 = {rt_nodes = {{rn_mklist = 0x0, rn_parent = 0xc20f5218, rn_bit = -49,
rn_bmask = 0 '\0', rn_flags = 5 '\005', rn_u = {rn_leaf = {
rn_Key = 0xc1c7a980 "\020\027 \004\006\005",
rn_Mask = 0xc1d2acb0 "\006ÿÿÿÿÿ", rn_Dupedkey = 0x0}, rn_node = {
rn_Off = -1043879552, rn_L = 0xc1d2acb0, rn_R = 0x0}}}, {
rn_mklist = 0x0, rn_parent = 0xc1cea04c, rn_bit = 18, rn_bmask = 32 ' ',
rn_flags = 4 '\004', rn_u = {rn_leaf = {rn_Key = 0x2---Can't read userspace from dump, or kernel process---
(kgdb) list
269 struct socket *so = ipxp->ipxp_socket;
270
271 so->so_pcb = 0;
272 sotryfree(so);
273 if (ipxp->ipxp_route.ro_rt != NULL)
274 rtfree(ipxp->ipxp_route.ro_rt);
275 remque(ipxp);
276 FREE(ipxp, M_PCB);
277 }
278
(kgdb) up
#14 0xc0577dbd in ipx_detach (so=0x0) at ../../../netipx/ipx_usrreq.c:492
492 ipx_pcbdetach(ipxp);
(kgdb) list
487 struct ipxpcb *ipxp = sotoipxpcb(so);
488
489 if (ipxp == NULL)
490 return (ENOTCONN);
491 s = splnet();
492 ipx_pcbdetach(ipxp);
493 splx(s);
494 return (0);
495 }
496
(kgdb) quit
help me please...
/swp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040608041810.GA1077>