Date: Mon, 1 Jul 2002 10:09:03 +0800 From: "David Xu" <davidx@viasoft.com.cn> To: "\"Matthew Dillon\"" <dillon@apollo.backplane.com> Cc: "Julian Elischer" <julian@elischer.org>, <sgk@troutmask.apl.washington.edu>, <wa1ter@hotmail.com>, <freebsd-current@FreeBSD.ORG> Subject: Re: KSE / interrupt panic (patch) Message-ID: <002801c220a4$4637c2a0$ef01a8c0@davidwnt>
next in thread | raw e-mail | index | archive | help
Now let me describe where the race is:
Thread A: | Thread B:
cv_timedwait() | softclock()
| cv_timedwait_end()
|
mtx_lock_spin(&sched_lock); | mtx_lock_spin(&sched_lock); /* suppose blocked!!! */
... |
callout_stop(&td->td_slpcallout) |
td->td_flags |= TDF_TIMEOUT; |
td->td_proc->p_stats->p_ru.ru_nivcsw++; |
// problem, does not set thread state to TDS_SLP!!! |
mi_switch(); |
// run at once again |
... |
mtx_unlock_spin(&sched_lock); | mtx_lock_spin() returned !!!
thread is still running | if (td->td_flags & TDF_TIMEOUT) {
... | td->td_flags &= ~TDF_TIMEOUT;
some place call mi_switch() and now on runqueue | setrunqueue(td); // crash
| }
----------------------------------------------------------
here is the patch:
--- /sys/kern/kern_condvar.c.old Mon Jul 1 09:06:01 2002
+++ /sys/kern/kern_condvar.c Mon Jul 1 09:32:50 2002
@@ -396,6 +396,7 @@
* between msleep and endtsleep.
*/
td->td_flags |= TDF_TIMEOUT;
+ td->td_state = TDS_SLP;
td->td_proc->p_stats->p_ru.ru_nivcsw++;
mi_switch();
}
@@ -472,6 +473,7 @@
* between msleep and endtsleep.
*/
td->td_flags |= TDF_TIMEOUT;
+ td->td_state = TDS_SLP;
td->td_proc->p_stats->p_ru.ru_nivcsw++;
mi_switch();
}
-------------------------------------------------------------------
bug is because cv_timedwait() detects timeout callout is running,
but it does not correctly wait callout to complete, so panic.
BTW, the bug seems also exists in msleep() and endtsleep(), please
fix it!
-David Xu
--- David Xu <bsddiy@yahoo.com> wrote:
> setrunqueue() call can be simply removed from cv_timedwait_end(), because
> there
> is a race in softclock() and callout_stop(), when cv_timedwait_end() losts a
> race, it means that that thread is already running(wokenup by another
> thread),
> when you setrunqueue() it, of course it will panic.
> in cv_timedwait_end(), sentence "if (td->td_flags & TDF_TIMEOUT) {...}"
> is to check this race condition.
>
> -David Xu
>
> ----- Original Message -----
> From: "Matthew Dillon" <dillon@apollo.backplane.com>
> To: "Julian Elischer" <julian@elischer.org>
> Cc: "Steve Kargl" <sgk@troutmask.apl.washington.edu>; "walt"
> <wa1ter@hotmail.com>; <freebsd-current@FreeBSD.ORG>
> Sent: Monday, July 01, 2002 4:43 AM
> Subject: KSE / interrupt panic
>
>
> > Got another one. Different panic, same place.
> >
> > panic: setrunqueue: bad thread state
> > cpuid = 0; lapic.id = 01000000
> > Debugger("panic")
> > Stopped at Debugger+0x46: xchgl %ebx,in_Debugger.0
> > db> trace
> > Debugger(c02ec2ba) at Debugger+0x46
> > panic(c02ec8a9,c6461d80,c6461d80,c6461d80,c01afa30) at panic+0xd6
> > setrunqueue(c6461d80) at setrunqueue+0x1dd
> > cv_timedwait_end(c6461d80) at cv_timedwait_end+0x36
> > softclock(0) at softclock+0x159
> > ithread_loop(c229c700,df3eed48,c22aec00,c01b9c6c,0) at ithread_loop+0x12c
> > fork_exit(c01b9c6c,c229c700,df3eed48) at fork_exit+0xa8
> > fork_trampoline() at fork_trampoline+0x37
> > db> gdb
> > ...
> >
> > #0 Debugger (msg=0xc02ec2ba "panic")
> > at /FreeBSD/FreeBSD-current/src/sys/i386/i386/db_interface.c:324
> > #1 0xc01c878a in panic (fmt=0xc02ec8a9 "setrunqueue: bad thread state")
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_shutdown.c:482
> > #2 0xc01cc6cd in setrunqueue (td=0xc6461d80)
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_switch.c:396
> > #3 0xc01afa66 in cv_timedwait_end (arg=0xc6461d80)
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_condvar.c:608
> > #4 0xc01d22c9 in softclock (dummy=0x0)
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_timeout.c:187
> > #5 0xc01b9d98 in ithread_loop (arg=0xc229c700)
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_intr.c:535
> > #6 0xc01b923c in fork_exit (callout=0xc01b9c6c <ithread_loop>,
> > arg=0xc229c700, frame=0xdf3eed48)
> > at /FreeBSD/FreeBSD-current/src/sys/kern/kern_fork.c:863
> >
> > I'm not sure why the panic was 'bad thread state' when gdb seems to
> > show it being stuck on 'unexpected ke present'. Maybe it was an
> > optimization and gdb is confused. The panic is definitely
> > 'bad thread state'.
> >
> > (gdb) print td->td_state
> > $2 = TDS_RUNQ
> >
> > setrunqueue() is being called on a thread which is already on the run
> > queue.
> >
> > -Matt
> >
> >
> > To Unsubscribe: send mail to majordomo@FreeBSD.org
> > with "unsubscribe freebsd-current" in the body of the message
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! - Official partner of 2002 FIFA World Cup
> http://fifaworldcup.yahoo.com
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?002801c220a4$4637c2a0$ef01a8c0>