Date: Mon, 02 Mar 2015 13:53:13 -0500 From: John Baldwin <jhb@freebsd.org> To: freebsd-arch@freebsd.org Cc: Harrison Grundy <harrison.grundy@astrodoggroup.com> Subject: Re: Minor ULE changes and optimizations Message-ID: <5490895.NN1ciTh6gZ@ralph.baldwin.cx> In-Reply-To: <54F1E25F.5040905@astrodoggroup.com> References: <54EF2C54.7030207@astrodoggroup.com> <1547642.s3cC06khRt@ralph.baldwin.cx> <54F1E25F.5040905@astrodoggroup.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Saturday, February 28, 2015 07:44:31 AM Harrison Grundy wrote: > On 02/28/15 04:24, John Baldwin wrote: > > On Friday, February 27, 2015 07:50:55 AM Harrison Grundy wrote: > >> On 02/27/15 06:14, John Baldwin wrote: > >>> On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy > >>> > >>> wrote: > >>>> https://reviews.freebsd.org/D1969 This allows a > >>>> non-migratable thread to pin itself to a CPU if it is already > >>>> running on that CPU. > >>>> > >>>> I've been running these patches for the past week or so > >>>> without issue. Any additional testing or comments would be > >>>> greatly appreciated. > >>> > >>> Can you explain the reason / use case for this? This seems to > >>> be allowing an API violation. sched_pin() was designed to be > >>> a lower-level API than sched_bind(), so you wouldn't call > >>> sched_bind() if you were already pinned. In addition, > >>> sched_pin() is sometimes used by code that assumes it won't > >>> migrate until sched_unpin() (e.g. temporary mappings inside an > >>> sfbuf). If you allow sched_bind() to move a thread that is > >>> pinned you will allow someone to unintentionally break those > >>> sort of things instead of getting an assertion failure panic. > >> > >> For a pinned thread, the underlying idea is that if you're > >> already on the CPU you pinned to, calling sched_bind with that > >> CPU specified allows you to set TSF_BOUND without calling > >> sched_unpin first. > >> > >> If a pinned thread were to call sched_bind for a CPU it isn't > >> pinned to, it would still hit the assert and fail. > >> > >> For any unpinned thread, if you're already running on the correct > >> CPU, you can skip the THREAD_CAN_MIGRATE check and the call to > >> mi_switch. > > > > Ah, ok, so you aren't allowing migration in theory. However, I'm > > still curious as to why you want/need this. This makes the API > > usage a bit more complex to reason about (sched_bind() can > > sometimes be called while pinned but not always after this change), > > so I think that extra complexity needs a reason to exist. > > Primarily, it allows those threads already on a CPU to skip the call > to mi_switch and get out of sched_bind a bit faster. sched_bind() already does this. Internally it skips the call to mi_switch() if the thread is already on the correct CPU: void sched_bind(struct thread *td, int cpu) { ... ts->ts_flags |= TSF_BOUND; sched_pin(); if (PCPU_GET(cpuid) == cpu) return; ... } Calling sched_pin() before sched_bind() isn't going to really change that. Once you do thread_lock(td) your thread is effectively pinned until you do a thread_unlock() since the spin lock blocks preemption (and thus migration as well), so in a sequence of: thread_lock(td); sched_bind(td, cpu); The thread is effectively pinned once thread_lock() returns and will not need to use mi_switch() if it is already on the correct CPU. > Additionally, it allows a driver to call sched_pin, then bind to that > same cpu later without having to write something like > "critical_enter(); sched_unpin(); sched_bind(foo, bar); > critical_exit();", since otherwise it could be migrated/preempted > between unpin and bind. But why would a driver want to do that? This code: sched_pin(td); /* do something */ thread_lock(td); sched_unpin(td); sched_bind(td, PCPU_GET(cpuid)); thread_unlock(td); /* do something else */ thread_lock(td); sched_unbind(td); thread_unlock(td); Is equivalent to: sched_pin(td); /* do something */ /* do something else */ sched_unpin(td); But the latter form is lighter weight and easier to read / understand. Letting you sched_bind() to the current CPU while you are pinned doesn't enable any new functionality than you can already achieve by just using sched_pin() and sched_unpin(). -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5490895.NN1ciTh6gZ>