Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 27 Dec 2012 04:55:22 -0800
From:      Attilio Rao <attilio@freebsd.org>
To:        Gleb Smirnoff <glebius@freebsd.org>
Cc:        svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org
Subject:   Re: svn commit: r244732 - head/sys/sys
Message-ID:  <CAJ-FndD9aDfPprwBYC%2B3T1WsfE1b4aZJENRAjo%2BhFEL1NLBKmw@mail.gmail.com>
In-Reply-To: <20121227124657.GX80310@FreeBSD.org>
References:  <201212271236.qBRCawuU078203@svn.freebsd.org> <20121227124657.GX80310@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Dec 27, 2012 at 4:46 AM, Gleb Smirnoff <glebius@freebsd.org> wrote:
>   Attilio,
>
> On Thu, Dec 27, 2012 at 12:36:58PM +0000, Attilio Rao wrote:
> A> Author: attilio
> A> Date: Thu Dec 27 12:36:58 2012
> A> New Revision: 244732
> A> URL: http://svnweb.freebsd.org/changeset/base/244732
> A>
> A> Log:
> A>   br_prod_tail and br_cons_tail members are used as barrier to
> A>   signal bug_ring ownership. However, instructions can be reordered
> A>   around members write leading to stale values for ie. br_prod_bufs.
> A>
> A>   Use correct memory barriers to ensure proper ordering of the
> A>   ownership tokens updates.
> A>
> A>   Sponsored by:      EMC / Isilon storage division
> A>   MFC after: 2 weeks
>
> Have you profiled this?
>
> After this change the buf_ring actually gains its own hand-rolled
> mutex:
>
>         while (atomic_load_acq_32(&br->br_prod_tail) != prod_head)
>                 cpu_spinwait();
>
> The only difference with mutex(9) is that this one isn't monitored
> by WITNESS.

I think you are not correct. It doesn't disable interrupts (as
spinlock do) and it doesn't sleep.
So your analogy is completely off.

Also, on x86 atomic_store_rel_*() is a simple write. The only thing
that really changes is the atomic_load_acq_*() that introduces a
locked instruction.

> The idea behind buf_ring was lockless storing and lockless fetching
> from a ring and now this vanished.
>
> What does your change actually fixes except precise accounting of
> br_prod_bufs that are actually unused and should be better garbage
> collected rather than fixed?

The write of br_prod_tail must happens as very last thing, also after
the whole buf setup. The only way you can enforce this is with a
memory barrier. I can double-check if we can garbage collect
br_prod_bufs but this should not be enough yet.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-FndD9aDfPprwBYC%2B3T1WsfE1b4aZJENRAjo%2BhFEL1NLBKmw>