Date: Mon, 27 Sep 2004 22:52:16 -0400 From: Stephan Uphoff <ups@tree.com> To: Peter Holm <peter@holm.cc> Cc: "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: scheduler (sched_4bsd) questions Message-ID: <1096339936.3733.279.camel@palm.tree.com> In-Reply-To: <20040926075218.GA85983@peter.osted.lan> References: <1095468747.31297.241.camel@palm.tree.com> <1095529353.31297.1192.camel@palm.tree.com> <1096135220.53798.17754.camel@palm.tree.com> <20040926075218.GA85983@peter.osted.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
--=-rj4c4DOGe8hfA+4+RE6p Content-Type: text/plain Content-Transfer-Encoding: 7bit On Sun, 2004-09-26 at 03:52, Peter Holm wrote: > On Sat, Sep 25, 2004 at 08:50:45PM -0700, Julian Elischer wrote: > > Stephan Uphoff wrote: > > > > >>Maybe something brutal like: > > >> if ((curthread->td_ksegrp == kg) && > > >> (td->td_priority > curthread->td_priority)) > > >> curthread->td_flags |= TDF_NEEDRESCHED; > > >> > > >>in setrunqueue for > > >>the else case of "if (kg->kg_avail_opennings > 0)" > > >>would do the trick (without preemption) for the easy but probably more > > >>common cases? > > >> > > >>Maybe I can find some time next week to think about a clean > > >>fix. I find it always helpful having a small task in mind while reading > > >>source code. > > > > > > > > >I wrote a fix that should cover all cases. > > >However I would like to test it a little bit before posting the patch. > > >Is there any multi-threaded kernel torture program that you can > > >recommend? > > > > > > Peter Holm (CC'd) has a really cool set of torture tests. > > he has also seen all sorts of failures others have not (yet) triggered. :-) > > > > I'm 'busy" for the next couple of weeks so you may want to communicate > > directly with him and see if you and he together can figure out some of the > > things he's > > been seeing :-) > > > > his tests are at: > > http://www.holm.cc/stress/src/stress.tgz > > > > > > > >Thanks > > > > > > Stephan > > > > > I'll be glad to test any patches. Great. Can you try the attached patch to see if it changes any of your previously observed behaviour? Thanks Stephan --=-rj4c4DOGe8hfA+4+RE6p Content-Disposition: attachment; filename=switch_patch Content-Type: text/x-patch; name=switch_patch; charset=ASCII Content-Transfer-Encoding: 7bit Index: sys/kern/kern_switch.c =================================================================== RCS file: /cvsroot/src/sys/kern/kern_switch.c,v retrieving revision 1.95 diff -u -r1.95 kern_switch.c --- sys/kern/kern_switch.c 19 Sep 2004 18:34:17 -0000 1.95 +++ sys/kern/kern_switch.c 28 Sep 2004 02:48:43 -0000 @@ -315,6 +315,94 @@ td->td_priority = newpri; setrunqueue(td, SRQ_BORING); } + + +/* + * This function is called when a thread is about to be put on a + * ksegrp run queue because it has been made runnable or its + * priority has been adjusted and the ksegrp does not have a + * free kse slot. It determines if a thread from the same ksegrp + * should be preempted. If so, it tries to switch threads + * if the thread is on the same cpu or notifies another cpu that + * it should switch threads. + */ + +static void +maybe_preempt_in_ksegrp(struct thread *td) +{ +#if defined(SMP) + int highest_pri; + struct ksegrp *kg; + cpumask_t cpumask,dontuse; + struct pcpu *pc; + struct pcpu *highest_pcpu; + + mtx_assert(&sched_lock, MA_OWNED); + +#if !defined(KSEG_PEEMPT_BEST_CPU) + if(curthread->td_ksegrp != td->td_ksegrp) +#endif + { + kg = td->td_ksegrp; + + /* Anyone waiting in front ? */ + if(td != TAILQ_FIRST(&kg->kg_runq)) { + return; /* Yes - wait your turn*/ + } + highest_pri = td->td_priority; + highest_pcpu = NULL; + dontuse = stopped_cpus | idle_cpus_mask; + + /* Find a cpu with the worst priority that runs at thread from the + * same ksegrp - if multiple exist give first the last run cpu and then + * the current cpu priority + */ + + SLIST_FOREACH(pc, &cpuhead, pc_allcpu) { + cpumask = pc->pc_cpumask; + if ( (cpumask & dontuse) == 0 && + pc->pc_curthread->td_ksegrp == kg) { + if (pc->pc_curthread->td_priority > highest_pri) { + highest_pri = pc->pc_curthread->td_priority; + highest_pcpu = pc; + } else if (pc->pc_curthread->td_priority == highest_pri && + highest_pcpu != NULL) { + if (td->td_lastcpu == pc->pc_cpuid || + (PCPU_GET(cpumask) == cpumask && + td->td_lastcpu != highest_pcpu->pc_cpuid)) { + highest_pcpu = pc; + } + } + } + } + + /* Check if we need to preempt someone */ + if (highest_pcpu == NULL) return; + + if (PCPU_GET(cpuid) != highest_pcpu->pc_cpuid) { + pc->pc_curthread->td_flags |= TDF_NEEDRESCHED; + ipi_selected(highest_pcpu->pc_cpumask, IPI_AST); + return; + } + } +#else + KASSERT(curthread->td_ksegrp == td->td_ksegrp,("maybe_preempt_in_ksegrp: No chance to run thread")); +#endif + + if (td->td_priority <= curthread->td_priority) + return; +#ifdef PREEMPTION + if (td->td_critnest > 1) { + td->td_pflags |= TDP_OWEPREEMPT; + } else { + mi_switch(SW_INVOL, NULL); + } +#else + curthread->td_flags |= TDF_NEEDRESCHED; +#endif + return; +} + int limitcount; void setrunqueue(struct thread *td, int flags) @@ -422,6 +510,7 @@ } else { CTR3(KTR_RUNQ, "setrunqueue: held: td%p kg%p pid%d", td, td->td_ksegrp, td->td_proc->p_pid); + maybe_preempt_in_ksegrp(td); } } --=-rj4c4DOGe8hfA+4+RE6p--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1096339936.3733.279.camel>