Date: Sat, 24 Sep 2011 10:56:08 +0200 From: "K. Macy" <kmacy@freebsd.org> To: Arnaud Lacombe <lacombar@gmail.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: buf_ring(9) API precisions Message-ID: <CAHM0Q_Pt3GKnBKns4O4LQrQEPsSKZWugQoYYX3UWaKeaCd6Zjw@mail.gmail.com> In-Reply-To: <CACqU3MXJJeF0HnqQSQQAAANhR_cnB3hF9qF2xb3GnU=J5xiaVA@mail.gmail.com> References: <CACqU3MXQ6tD804fKymeFeKDnHndSXVvHJwepYztB4DsnNmtMiw@mail.gmail.com> <CACqU3MWwOw_otd0sJ-c4OXedeeJtchwiX9Xpx7V0zNW%2BcNZ7Yw@mail.gmail.com> <CAHM0Q_NfoSoa52rAAF8iUPQoqardbgSsq0PDnfh%2BmUFN993ZVA@mail.gmail.com> <CACqU3MWMeAMcrDZ2NF_OytYgiAFxmHvYRKcCVk=-=_VVYAcExQ@mail.gmail.com> <CAHM0Q_NbOGj4rEpHWBJooyrzYi2rehbxd5LChTga1DzWW6P44g@mail.gmail.com> <CACqU3MXJJeF0HnqQSQQAAANhR_cnB3hF9qF2xb3GnU=J5xiaVA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
You're right. A write memory barrier is needed there.
Thanks
On Thu, Sep 22, 2011 at 12:43 AM, Arnaud Lacombe <lacombar@gmail.com> wrote=
:
> Hi,
>
> On Mon, Sep 19, 2011 at 8:46 AM, K. Macy <kmacy@freebsd.org> wrote:
>> If the value lags next by one then it is ours. This rule applies to
>> all callers so the rule holds consistently.
>>
> I think you do not understand what I mean, which is that the following:
>
> =A0 =A0 =A0 while (br->br_prod_tail !=3D prod_head)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 cpu_spinwait();
> =A0 =A0 =A0 br->br_prod_bufs++;
> =A0 =A0 =A0 br->br_prod_bytes +=3D nbytes;
> =A0 =A0 =A0 br->br_prod_tail =3D prod_next;
> =A0 =A0 =A0 critical_exit();
>
> at runtime, can be seen, memory-wise as:
>
> =A0 =A0 =A0 while (br->br_prod_tail !=3D prod_head)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 cpu_spinwait();
> =A0 =A0 =A0 br->br_prod_tail =3D prod_next;
> =A0 =A0 =A0 br->br_prod_bufs++;
> =A0 =A0 =A0 br->br_prod_bytes +=3D nbytes;
> =A0 =A0 =A0 critical_exit();
>
> That is, there is no memory barrier to enforce completion of the
> load/increment/store/load/load/addition/store operations before
> updating what other thread spin on. Yes, `br_prod_tail' is marked
> `volatile', but there is no guarantee that it will not be re-ordered
> wrt. non-volatile write (to `br_prod_bufs' and `br_prod_bytes').
>
> =A0- Arnaud
>
>> On Mon, Sep 19, 2011 at 5:53 AM, Arnaud Lacombe <lacombar@gmail.com> wro=
te:
>>> Hi,
>>>
>>> On Fri, Sep 16, 2011 at 10:41 AM, K. Macy <kmacy@freebsd.org> wrote:
>>>> On Fri, Sep 16, 2011 at 3:02 AM, Arnaud Lacombe <lacombar@gmail.com> w=
rote:
>>>>> Hi,
>>>>>
>>>>> On Wed, Sep 14, 2011 at 10:53 PM, Arnaud Lacombe <lacombar@gmail.com>=
wrote:
>>>>>> Hi Kip,
>>>>>>
>>>>>> I've got a few question about the buf_ring(9) API.
>>>>>>
>>>>>> 1) what means the 'drbr_' prefix. I can guess the two last letter, '=
b'
>>>>>> and 'r', for Buffer Ring, but what about 'd' and 'r' ?
>>>>>>
>>>>>> 2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
>>>>>>
>>>>>> struct buf_ring {
>>>>>> =A0 =A0 =A0 =A0volatile uint32_t =A0 =A0 =A0 br_prod_head;
>>>>>> =A0 =A0 =A0 =A0volatile uint32_t =A0 =A0 =A0 br_prod_tail;
>>>>>> =A0 =A0 =A0 =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 br_prod_s=
ize;
>>>>>> =A0 =A0 =A0 =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 br_prod_m=
ask;
>>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_drops;
>>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_prod_bufs;
>>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_prod_bytes=
;
>>>>> shouldn't those 3 fields be updated atomically, especially on 32bits
>>>>> platforms ? That might pose a problem as, AFAIK, FreeBSD do not have
>>>>> MI 64bits atomics operations...
>>>>
>>>> Between the point at which br_prod_tail =3D=3D prod_head and when we
>>>> update br_prod_tail to point to prod_next we are the exclusive owners
>>>> of the fields in buf_ring. That is why we wait for any other
>>>> enqueueing threads to update br_prod_tail to point to prod_head before
>>>> continuing.
>>>>
>>> How do you enforce ordering ? I do not see anything particular
>>> forbidding the `br->br_prod_tail' to be committed first, leading other
>>> thread to believe they have access to the statistics, while the other
>>> thread has not yet committed its change.
>>>
>>> Thanks,
>>> =A0- Arnaud
>>>
>>>> Cheers
>>>>
>>>> =A0 =A0 =A0 =A0/*
>>>> =A0 =A0 =A0 =A0 * If there are other enqueues in progress
>>>> =A0 =A0 =A0 =A0 * that preceeded us, we need to wait for them
>>>> =A0 =A0 =A0 =A0 * to complete
>>>> =A0 =A0 =A0 =A0 */
>>>> =A0 =A0 =A0 =A0while (br->br_prod_tail !=3D prod_head)
>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0cpu_spinwait();
>>>> =A0 =A0 =A0 =A0br->br_prod_bufs++;
>>>> =A0 =A0 =A0 =A0br->br_prod_bytes +=3D nbytes;
>>>> =A0 =A0 =A0 =A0br->br_prod_tail =3D prod_next;
>>>> =A0 =A0 =A0 =A0critical_exit();
>>>>
>>>
>>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHM0Q_Pt3GKnBKns4O4LQrQEPsSKZWugQoYYX3UWaKeaCd6Zjw>