Date: Thu, 29 Nov 2012 07:38:30 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Jeff Roberson <jroberson@jroberson.net> Cc: "attilio@freebsd.org" <attilio@freebsd.org>, Mark Linimon <linimon@lonesome.com>, Alfred Perlstein <bright@mu.org>, Oleksandr Tymoshenko <gonzo@bluezbox.com>, "arch@freebsd.org" <arch@freebsd.org> Subject: Re: [RFC] sema_wait_sig Message-ID: <alpine.BSF.2.00.1211290726020.59662@fledge.watson.org> In-Reply-To: <alpine.BSF.2.00.1211262055150.4081@desktop> References: <20121124193010.GB1627@lonesome.com> <50B12520.7040508@mu.org> <CAJ-FndBeLvsgXQ4fskRwdZh2qaWbn7-LrCOmJTjPcfnbmD7aYg@mail.gmail.com> <50B145C5.8070503@mu.org> <CAJ-FndC%2BjHO2uKg%2Bd8GmzWGkQ8nZ6_iefgyGAeno0VqprR84Wg@mail.gmail.com> <50B16E7A.60900@mu.org> <CAJ-FndAetPqiZ0nCQTY1xAcqBJuuaq9dZfUhP9YXXw669o0WNQ@mail.gmail.com> <50B178A3.4070305@mu.org> <46D582BB-1EB9-4080-9733-7558D6D87FA8@bsdimp.com> <50B17BF7.7020609@mu.org> <CD4350B2-40E3-4964-B403-78825A00C75A@vigrid.com> <50B187F2.2040506@mu.org> <alpine.BSF.2.00.1211262055150.4081@desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 26 Nov 2012, Jeff Roberson wrote:
> I think that's the underlying issue here. We don't use semaphores, they
> aren't implemented well on FreeBSD. They are there as a token effort at
> providing an API that we felt was better served in other ways. If we want
> to support this API then we should support it well and it should be written
> to the standards of the other primitives.
>
> Attilio wants to solve the problem by eliminating the API entirely because
> he knows how long it takes to write a good primitive. Those who are calling
> to support it should consider the effort it takes. It's not free. Calling
> it supported rather than deprecated means driver authors will expect it to
> be up to the quality standards of the rest of the kernel. This could have
> unintended consequences. Idioms will be developed. Code will be copied and
> propagated. All while using something terribly inefficient. Users will
> wonder why their driver performs so much better on Linux.
>
> I am responsible for one use of this code and I only did so knowing that it
> wasn't for anything performance critical and I was being lazy with a
> considerable porting effort ahead of me. I think my approach at this point
> would be either to entirely eliminate sema or rewrite it appropriately.
> Adding new consumers and hacking in a new feature is just another half
> measure. Which of the other solutions is best is up to individual bias.
I had originally followed up on Jeff's comment privately, but probably would
have been more useful to do so publically -- I'm a bit late to the discussion,
but:
FWIW, I quite agree with these points -- I'd hoped sema(9) would go away years
ago, since its underuse has a number of implications:
(1) Poor integration with debugging support -- e.g., WITNESS, profiling, etc.
These features make a huge difference in our development model,
effectively preventing us from shipping many common forms of deadlock in
the field at all, but come at significant initial cost in developing the
underlying locking primitives. sema(9) is an abandoned step child and
should not persist in its current state.
(2) Poor optimisation -- the great work folk like John and Attilio have done
on adaptive locking support, etc, massively improved performance in the
6.x and 7.x timeframe, addressing necessary contention much more
efficiently. The reality is that contention occurs frequently because
communication occurs frequently; obviously, we try to use work
distribution, improved lock granularity, etc, where we can, but sometimes
you must communicate and you can't use lockless primitives, so locks need
to be fast under contention, not just when uncontended.
(3) Not fitting our general design choice of aligning synchronisation
primitives with POSIX -- a particularly important design choice in
pthreads is not conflating mutual exclusion (locks) with process
synchronisation (condition variables). This semantic distinction improves
our ability to use static and dynamic analysis (e.g., WITNESS, static lock
order checking), but also affects our ability to offer priority
propagation. We could try and fix this in semaphores by providing more
intent information (SEM_THIS_IS_A_MUTEX, SEM_THIS_IS_A_CONDVAR) but at
that point I think we're much better off using the existing mutex, rwlock,
and condition variable primitives directly, which gives us the information
via C types.
The main reason sema(9) didn't go away was its use in our old ATA code; this
is something that can and should be fixed, and the new ATA code does not take
that perspective. While I have no plans to do so myself, I'd be quite
supportive of garbage collecting sema(9) in favour of requiring minor
modifications to driver ports in order to better distinguish mutual exclusion
and synchronisation, allowing WITNESS and other tools to properly analyse the
behavior of imported code, and also to support future use of static analysis
(e.g., the maturing Clang static lock analyser).
The only recent code I'm aware of to start using sema(9) is the Infiniband
work you did; if you feel that it's not a significant burden to adapt it to
use the native synchronisation primitives, then the main structural obstacle
to moving away from sema(9) is eliminated.
Robert
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1211290726020.59662>