Date: Tue, 22 Sep 2009 13:46:54 -0500 (CDT) From: "Sean C. Farley" <scf@FreeBSD.org> To: freebsd-emulation@FreeBSD.org Subject: Panic with vboxnet drivers Message-ID: <alpine.BSF.2.00.0909221337580.13791@thor.farley.org>
next in thread | raw e-mail | index | archive | help
While I have had no trouble with two systems running with the tap patch,
I am experiencing some lockups when using the vboxnet* drivers.
I have five witness logs (attached):
1. The first one appeared when I tried to run VirtualBox with vboxnetadp
loaded via loader.conf. VirtualBox will not be able to find the
VirtualBox network drivers this way, so I unloaded and loaded
vboxnetadp from the command line for the following logs.
2. The next three logs are LOR's (sleepable after non-sleepable)
concerning VirtualBox's "IPRT Fast Mutex Semaphore" which is an sx.
I am not sure I am reading the backtrace correctly. It looks like
the call to RTSemFastMutexRequest(), which calls sx_xlock(), is the
effect. I just do not know where the cause, i.e.,
RTSemFastMutexRequest(), is being called.
3. The last log is a LOR, but it may not be related to VirtualBox.
Does _end() represent a special function called after any function
returns?
The easiest way to induce a panic is to scp a large file into a VM and
then start a download of a file over the same network device of the host
that vboxnet0 is bridged. It usually takes a few seconds. This occurs
on two different systems running 7-STABLE (r196739):
1. i386, em0, gmirror
2. amd64, em0, no gmirror
Here is the only panic that ever made it to the disk. Most of the time
it just locks the system.
Unread portion of the kernel message buffer:
Sleeping thread (tid 100048, pid 48) owns a non-sleepable lock
panic: sleeping thread
cpuid = 0
Sleeping thread (tid 100048, pid 48) owns a non-sleepable lock
panic: sleeping thread
cpuid = 0
Uptime: 27m12s
Physical memory: 3306 MB
Dumping 236 MB: 221 205 189 173 157 141 125 109 93 77 61 45 29 13
#0 doadump () at pcpu.h:196
#1 0xc057bf87 in boot (howto=260) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_shutdown.c:418
#2 0xc057c259 in panic (fmt=Variable "fmt" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_shutdown.c:574
#3 0xc05b3a7e in propagate_priority (td=0xc71b8b40) at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_turnstile.c:222
#4 0xc05b3b89 in turnstile_adjust (td=0xc7638b40, oldpri=128 '\200') at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_turnstile.c:442
#5 0xc059d6fb in sched_prio (td=0xc7638b40, prio=76 'L') at /usr/FreeBSD/RELENG_7/src/sys/kern/sched_ule.c:1726
#6 0xc058444d in _sleep (ident=0xc7486870, lock=0x0, priority=76, wmesg=0xc09ab880 "m:destroy", timo=200) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_synch.c:219
#7 0xc09a5441 in g_mirror_destroy (sc=0xc7486800, how=1) at /usr/FreeBSD/RELENG_7/src/sys/modules/geom/geom_mirror/../../../geom/mirror/g_mirror.c:2975
#8 0xc09a57e7 in g_mirror_shutdown_pre_sync (arg=0xc09ad3e0, howto=260) at /usr/FreeBSD/RELENG_7/src/sys/modules/geom/geom_mirror/../../../geom/mirror/g_mirror.c:3235
#9 0xc057b9f3 in boot (howto=260) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_shutdown.c:296
#10 0xc057c259 in panic (fmt=Variable "fmt" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_shutdown.c:574
#11 0xc05b3a7e in propagate_priority (td=0xc71b8b40) at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_turnstile.c:222
#12 0xc05b4948 in turnstile_wait (ts=0xc763a4b0, owner=0xc71b8b40, queue=Variable "queue" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_turnstile.c:740
#13 0xc057aa88 in _rw_wlock_hard (rw=0xc7a9f36c, tid=3345189696, file=0x0, line=0) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_rwlock.c:686
#14 0xc06aaadb in tcp_usr_rcvd (so=0xc7a16000, flags=0) at /usr/FreeBSD/RELENG_7/src/sys/netinet/tcp_usrreq.c:707
#15 0xc05d8f93 in soreceive_generic (so=0xc7a16000, psa=0xe9bafbe8, uio=0xe9bafbf4, mp0=0x0, controlp=0x0, flagsp=0xe9bafc78) at /usr/FreeBSD/RELENG_7/src/sys/kern/uipc_socket.c:1828
#16 0xc05d3788 in soreceive (so=0xc7a16000, psa=0xe9bafbe8, uio=0xe9bafbf4, mp0=0x0, controlp=0x0, flagsp=0xe9bafc78) at /usr/FreeBSD/RELENG_7/src/sys/kern/uipc_socket.c:2032
#17 0xc05da5a5 in kern_recvit (td=0xc7638b40, s=3, mp=0xe9bafc60, fromseg=UIO_USERSPACE, controlp=0x0) at /usr/FreeBSD/RELENG_7/src/sys/kern/uipc_syscalls.c:1002
#18 0xc05da7b1 in recvit (td=Variable "td" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/uipc_syscalls.c:1113
#19 0xc05da926 in recvfrom (td=0xc7638b40, uap=0xe9bafcfc) at /usr/FreeBSD/RELENG_7/src/sys/kern/uipc_syscalls.c:1157
#20 0xc07cb085 in syscall (frame=0xe9bafd38) at /usr/FreeBSD/RELENG_7/src/sys/i386/i386/trap.c:1094
#21 0xc07b0c80 in Xint0x80_syscall () at /usr/FreeBSD/RELENG_7/src/sys/i386/i386/exception.s:262
#22 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
Here is the output from the sleeping thread:
#0 sched_switch (td=0xc71b8b40, newtd=Variable "newtd" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/sched_ule.c:1944
#1 0xc0584046 in mi_switch (flags=Variable "flags" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_synch.c:444
#2 0xc05b080b in sleepq_switch (wchan=Variable "wchan" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_sleepqueue.c:497
#3 0xc05b0e56 in sleepq_wait (wchan=0xc7be02d4) at /usr/FreeBSD/RELENG_7/src/sys/kern/subr_sleepqueue.c:580
#4 0xc058388e in _sx_xlock_hard (sx=0xc7be02d4, tid=3340471104, opts=0, file=0xc0acb042 "/usr/ports/emulators/virtualbox/work/virtualbox-3.0.51r22902/src/VBox/Runtime/r0drv/freebsd/semfastmutex-r0drv-freebsd.c", line=103) at /usr/FreeBSD/RELENG_7/src/sys/kern/kern_sx.c:560
#5 0xc0583c28 in _sx_xlock (sx=0xc7be02d4, opts=0, file=0xc0acb042 "/usr/ports/emulators/virtualbox/work/virtualbox-3.0.51r22902/src/VBox/Runtime/r0drv/freebsd/semfastmutex-r0drv-freebsd.c", line=103) at sx.h:154
#6 0xc0ab8351 in RTSemFastMutexRequest () from /boot/modules/vboxdrv.ko
#7 0xc7f3d82a in ?? ()
#8 0xc7be02d0 in ?? ()
#9 0xc85b6c00 in ?? ()
#10 0x00000000 in ?? ()
#11 0xc8338a90 in ?? ()
#12 0xc8338a90 in ?? ()
#13 0xc85b6c00 in ?? ()
#14 0xe788f6e4 in ?? ()
#15 0xc7ee10bb in ng_vboxnetflt_rcvdata () from /boot/modules/vboxnetflt.ko
The thread itself is:
49 Thread 100048 (PID=48: em0 taskq) sched_switch (td=0xc71b8b40, newtd=Variable "newtd" is not available.) at /usr/FreeBSD/RELENG_7/src/sys/kern/sched_ule.c:1944
While em0 taskq is the thread, is it really the em driver's fault or
vboxnetflt? Does the panic message mean that the priority was changed
while holding a spinlock?
Sean
--
scf@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.0909221337580.13791>