Date: Wed, 21 Sep 2011 18:43:45 -0400 From: Arnaud Lacombe <lacombar@gmail.com> To: "K. Macy" <kmacy@freebsd.org> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: buf_ring(9) API precisions Message-ID: <CACqU3MXJJeF0HnqQSQQAAANhR_cnB3hF9qF2xb3GnU=J5xiaVA@mail.gmail.com> In-Reply-To: <CAHM0Q_NbOGj4rEpHWBJooyrzYi2rehbxd5LChTga1DzWW6P44g@mail.gmail.com> References: <CACqU3MXQ6tD804fKymeFeKDnHndSXVvHJwepYztB4DsnNmtMiw@mail.gmail.com> <CACqU3MWwOw_otd0sJ-c4OXedeeJtchwiX9Xpx7V0zNW%2BcNZ7Yw@mail.gmail.com> <CAHM0Q_NfoSoa52rAAF8iUPQoqardbgSsq0PDnfh%2BmUFN993ZVA@mail.gmail.com> <CACqU3MWMeAMcrDZ2NF_OytYgiAFxmHvYRKcCVk=-=_VVYAcExQ@mail.gmail.com> <CAHM0Q_NbOGj4rEpHWBJooyrzYi2rehbxd5LChTga1DzWW6P44g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
On Mon, Sep 19, 2011 at 8:46 AM, K. Macy <kmacy@freebsd.org> wrote:
> If the value lags next by one then it is ours. This rule applies to
> all callers so the rule holds consistently.
>
I think you do not understand what I mean, which is that the following:
while (br->br_prod_tail !=3D prod_head)
cpu_spinwait();
br->br_prod_bufs++;
br->br_prod_bytes +=3D nbytes;
br->br_prod_tail =3D prod_next;
critical_exit();
at runtime, can be seen, memory-wise as:
while (br->br_prod_tail !=3D prod_head)
cpu_spinwait();
br->br_prod_tail =3D prod_next;
br->br_prod_bufs++;
br->br_prod_bytes +=3D nbytes;
critical_exit();
That is, there is no memory barrier to enforce completion of the
load/increment/store/load/load/addition/store operations before
updating what other thread spin on. Yes, `br_prod_tail' is marked
`volatile', but there is no guarantee that it will not be re-ordered
wrt. non-volatile write (to `br_prod_bufs' and `br_prod_bytes').
- Arnaud
> On Mon, Sep 19, 2011 at 5:53 AM, Arnaud Lacombe <lacombar@gmail.com> wrot=
e:
>> Hi,
>>
>> On Fri, Sep 16, 2011 at 10:41 AM, K. Macy <kmacy@freebsd.org> wrote:
>>> On Fri, Sep 16, 2011 at 3:02 AM, Arnaud Lacombe <lacombar@gmail.com> wr=
ote:
>>>> Hi,
>>>>
>>>> On Wed, Sep 14, 2011 at 10:53 PM, Arnaud Lacombe <lacombar@gmail.com> =
wrote:
>>>>> Hi Kip,
>>>>>
>>>>> I've got a few question about the buf_ring(9) API.
>>>>>
>>>>> 1) what means the 'drbr_' prefix. I can guess the two last letter, 'b=
'
>>>>> and 'r', for Buffer Ring, but what about 'd' and 'r' ?
>>>>>
>>>>> 2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
>>>>>
>>>>> struct buf_ring {
>>>>> =A0 =A0 =A0 =A0volatile uint32_t =A0 =A0 =A0 br_prod_head;
>>>>> =A0 =A0 =A0 =A0volatile uint32_t =A0 =A0 =A0 br_prod_tail;
>>>>> =A0 =A0 =A0 =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 br_prod_si=
ze;
>>>>> =A0 =A0 =A0 =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 br_prod_ma=
sk;
>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_drops;
>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_prod_bufs;
>>>>> =A0 =A0 =A0 =A0uint64_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0br_prod_bytes;
>>>> shouldn't those 3 fields be updated atomically, especially on 32bits
>>>> platforms ? That might pose a problem as, AFAIK, FreeBSD do not have
>>>> MI 64bits atomics operations...
>>>
>>> Between the point at which br_prod_tail =3D=3D prod_head and when we
>>> update br_prod_tail to point to prod_next we are the exclusive owners
>>> of the fields in buf_ring. That is why we wait for any other
>>> enqueueing threads to update br_prod_tail to point to prod_head before
>>> continuing.
>>>
>> How do you enforce ordering ? I do not see anything particular
>> forbidding the `br->br_prod_tail' to be committed first, leading other
>> thread to believe they have access to the statistics, while the other
>> thread has not yet committed its change.
>>
>> Thanks,
>> =A0- Arnaud
>>
>>> Cheers
>>>
>>> =A0 =A0 =A0 =A0/*
>>> =A0 =A0 =A0 =A0 * If there are other enqueues in progress
>>> =A0 =A0 =A0 =A0 * that preceeded us, we need to wait for them
>>> =A0 =A0 =A0 =A0 * to complete
>>> =A0 =A0 =A0 =A0 */
>>> =A0 =A0 =A0 =A0while (br->br_prod_tail !=3D prod_head)
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0cpu_spinwait();
>>> =A0 =A0 =A0 =A0br->br_prod_bufs++;
>>> =A0 =A0 =A0 =A0br->br_prod_bytes +=3D nbytes;
>>> =A0 =A0 =A0 =A0br->br_prod_tail =3D prod_next;
>>> =A0 =A0 =A0 =A0critical_exit();
>>>
>>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACqU3MXJJeF0HnqQSQQAAANhR_cnB3hF9qF2xb3GnU=J5xiaVA>