Date: Sun, 25 Jun 2000 09:58:27 -0400 (EDT) From: Daniel Eischen <eischen@vigrid.com> To: Jason Evans <jasone@canonware.com> Cc: smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <Pine.SUN.3.91.1000625091445.2784A-100000@pcnet1.pcnet.com> In-Reply-To: <20000624235605.D8965@blitz.canonware.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 24 Jun 2000, Jason Evans wrote:
> 3. The BSD/OS solution.
>
> - The BGL remains, but becomes increasingly meaningless. In particular,
> it is not always necessary to obtain it in order to enter the kernel.
>
> - Instead the system protects shared data structures with mutexes.
> These mutexes replace calls to tsleep() when waiting on shared
> resources (the involuntary process synchronization mentioned above).
> In contrast to traditional UNIX, mutexes will be used much more
> frequently in order to protect data structures which were previously
> implicitly protected by the non-preemptive nature of the kernel. This
> mechanism will replace calls to tsleep() for involuntary context
> switches.
>
> Compared with the use of tsleep(), mutexes have a number of
> advantages:
>
> - Each mutex has its own wait (sleep) queue. When a process releases
> a mutex, it automatically schedules the next process waiting on the
> queue. This is more efficient than searching a possibly very long,
> linear sleep queue. It also avoids the flooding when multiple
> processes get scheduled, and most of them have to go back to sleep
> again.
>
> - Mutexes can be a combination of spin and sleep mutexes: for a
> resource which may be held only for a very short period of time,
> even the overhead of sleeping and rescheduling may be higher than
> waiting in a tight loop. A spin/sleep lock might first wait in a
> tight loop for 2 microseconds and then sleep if the lock is still
> not available at that time. This is an issue which Sun has
> investigated in great detail with Solaris. BSDi has not pursued
> this yet, though the BSD/OS threading primitives make this an easy
> extention to add. It's possibly an area for us to investigate once
> the system is up and limping again.
If anyone is interested...
All high-level interrupts (levels 11-15, mostly PIO serial interrupts)
in Solaris use spin mutexes and don't use an interrupt thread. They
execute in the context of the thread that was currently running. All
other interrupts below level 11 (clock, network, disk, etc) use interrupt
threads.
A Solaris (non-spinning) mutex will only spin while the owning thread is
running. Since BSDi mutexes have owners (correct me if I'm wrong), this
seems to be better than arbitrarily spinning.
>
> - Interrupt lockout (spl()s) go away completely. Instead, interrupt
> functions use mutexes for synchronization. This means that an
> interrupt function must be capable of blocking, which is currently
> impossible. In order to block, the function must have a "process"
> context (a stack and a process control block). In particular, this
> could include kernel threads.
>
> BSD/OS on Intel currently uses light-weight interrupt threads to
> process interrupts, while on SPARC uses normal ("heavyweight")
> processes. Chuck indicated that the decision to implement
> light-weight threads initially was probably the wrong one, since it
> gave rise to a large number of problems, and although the heavyweight
> process model would give lousy performance, it would probably make it
> easier to develop the kernel while the light-weight processes were
> being debugged. There is also the possibility of building a kernel
> with one or the other support, so that in case of problems during
> development it would be possible to revert to the heavy-weight
> processes while searching for the bug.
>
> Other details we discussed included:
>
> - BSD/OS has not implemented condition variables. We didn't go into
> details. The opinion was expressed that they would be very useful for
> synchronization, but that they require coding discipline somewhat
> different than the tsleep() mechanism. Converting all use of tsleep()
> is a lot of work, and of dubious value. However, condition variables
> can live along with tsleep(), so a full changeover is not necessary.
For a lot of drivers, it seems pretty straight forward to convert
splXXX() ... tsleep() ... splx() to mtx_enter() ... cv_wait()/cv_wait_sig()
... mtx_exit().
>
> - BSD/OS also does not implement read/write locks, a thing that Linux
> has recently introduced. We didn't discuss this further, but the
> concepts are well understood, and it should be relatively simple to
> implement them if we find a need.
Mutexes are only used in Solaris when they will be held for very small
amounts of time. Read/write locks and semaphores are used for all
other instances. While we are modifying the kernel to add mutexes,
it would probably be worthwhile to comment those sections of code
that could hold mutexes for something other than a very short period
of time. Or even use a different naming convention for those mutexes.
--
Dan Eischen
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SUN.3.91.1000625091445.2784A-100000>