Date: Thu, 31 Mar 2011 14:40:44 -0400 From: Attilio Rao <attilio@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-current@freebsd.org, Svatopluk Kraus <onwahe@gmail.com> Subject: Re: schedcpu() in /sys/kern/sched_4bsd.c calls thread_lock() on thread with un-initialized td_lock Message-ID: <AANLkTikDkiqofFuAT6PQuy8mZX7d4mxqzsO0v7ZvRmt=@mail.gmail.com> In-Reply-To: <201103311437.19682.jhb@freebsd.org> References: <AANLkTimEiOW%2BkSZD6n1MHiRou3UWibU6Oy3fr9RO4_O4@mail.gmail.com> <201103311418.31658.jhb@freebsd.org> <AANLkTin7jzwh2CVouWfK7j7576MSjDVLAY%2B--_LTJX_Z@mail.gmail.com> <201103311437.19682.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
2011/3/31 John Baldwin <jhb@freebsd.org>:
> On Thursday, March 31, 2011 2:20:11 pm Attilio Rao wrote:
>> 2011/3/31 John Baldwin <jhb@freebsd.org>:
>> > On Thursday, March 31, 2011 12:34:31 pm Attilio Rao wrote:
>> >> 2011/3/31 John Baldwin <jhb@freebsd.org>:
>> >> > On Thursday, March 31, 2011 7:32:26 am Svatopluk Kraus wrote:
>> >> >> Hi,
>> >> >>
>> >> >> =C2=A0 I've got a page fault (because of NULL td_lock) in
>> >> >> thread_lock_flags() called from schedcpu() in /sys/kern/sched_4bsd=
.c
>> >> >> file. During process fork, new thread is linked to new process whi=
ch
>> >> >> is linked to allproc list and both allproc_lock and new process lo=
ck
>> >> >> are unlocked before sched_fork() is called, where new thread td_lo=
ck
>> >> >> is initialized. Only PRS_NEW process status is on sentry but not
>> >> >> checked in schedcpu().
>> >> >
>> >> > I think this should fix it:
>> >> >
>> >> > Index: sched_4bsd.c
>> >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> > --- sched_4bsd.c =C2=A0 =C2=A0 =C2=A0 =C2=A0(revision 220190)
>> >> > +++ sched_4bsd.c =C2=A0 =C2=A0 =C2=A0 =C2=A0(working copy)
>> >> > @@ -463,6 +463,10 @@ schedcpu(void)
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0sx_slock(&allproc_lock);
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0FOREACH_PROC_IN_SYSTEM(p) {
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PROC_LOCK(p)=
;
>> >> > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (p->p_state =
=3D=3D PRS_NEW) {
>> >> > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 PROC_UNLOCK(p);
>> >> > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 continue;
>> >> > + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0FOREACH_THRE=
AD_IN_PROC(p, td) {
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0awake =3D 0;
>> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0thread_lock(td);
>> >> >
>> >>
>> >> I don't really think this fix is right because otherwise, when using
>> >> sched_4bsd anytime we are going to scan the thread list within a proc
>> >> we need to check for PRS_NEW.
>> >>
>> >> We likely need to change the init scheme for the td_lock by having a
>> >> scheduler primitive setting it and doing that on thread_init() UMA
>> >> constructor, or similar approach.
>> >
>> > But the thread state isn't valid anyway. =C2=A04BSD shouldn't be touch=
ing the
>> > thread since it is in an incomplete / undefined state.
>>
>> Yep, in this case I'd then want to just add the threads to proc once
>> they are fully initialized.
>>
>> It is pointless (and dangerous) to replicate this check all over,
>> besides we want scheduler agnostic code, which means every iterations
>> of p_threads will need to check for a valid state of threads.
>
> Yes, we do have to check for PRS_NEW in many places with the current appr=
oach,
> but we need some way to reserve the PID to avoid duplicates and unless we
> expand the scope of allproc in fork by a whole lot or stop using the allp=
roc
> list to track "pids in use", we will be stuck with some sort of "process
> is still being built" sentry.
Yes, you are right, I was assuming you wanted to work on a larger
patchset though.
If you are happy enough with the band-aid, for the moment, ok, but I
strongly raccomand to change this in the future (could be a nice task
to work through BSDCan, for example).
Attilio
--=20
Peace can only be achieved by understanding - A. Einstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikDkiqofFuAT6PQuy8mZX7d4mxqzsO0v7ZvRmt=>