Date: Wed, 12 Apr 2017 01:17:02 +0800 From: Yubin Ruan <ablacktshirt@gmail.com> To: Chris Torek <torek@elf.torek.net>, imp@bsdimp.com Cc: ed@nuxi.nl, freebsd-hackers@freebsd.org, rysto32@gmail.com Subject: Re: Understanding the FreeBSD locking mechanism Message-ID: <04b3328f-7bfb-bb70-c665-b43038cdd768@gmail.com> In-Reply-To: <4768e26a-cdec-6f40-1463-ece9847ca34d@gmail.com> References: <201704100426.v3A4QR9Q042761@elf.torek.net> <4768e26a-cdec-6f40-1463-ece9847ca34d@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017/4/12 1:15, Yubin Ruan wrote:
>
> Thanks for your reply. I have read your mails and your discussion with
> Konstantin Belousov
>
> On 2017/4/10 12:26, Chris Torek wrote:
>>>>> Is it true that a thread holding a MTX_DEF mutex can be descheduled?
>>
>>>> Yes, they can be descheduled. But that's not a problem. No other
>>>> thread can acquire the MTX_DEF lock. ...
>>
>>> Does that imply that MTX_DEF should not be used in something like
>>> interrupt handler? Putting an interrupt handler into sleep doesn't
>>> make so much sense.
>>
>> Go back to the old top-half / bottom-half model, and consider that
>> now that there are interrupt *threads*, your ithread is also in the
>> "top half". It's therefore OK to suspend. ("Sleep" is not quite
>> correct here: a mutex wait is not a "sleep" state but instead is
>> just a waiting, not-scheduled-to-run state. The precise difference
>> is irrelevant at this level though.)
>
> I don't truely understand the "top-half/bottom-half" model you proposed,
> but I think I get the idea of how things work now. Basically, we can
> assume that if a thread is in the "bottom-half", then it should never
> suspend(or, in the other words, be preempted). This is the case of the
> "interrupt filter" in FreeBSD. On the other hand, if a thread is in the
> "top-half", then it is safe to suspend/block. This is the case of the
> "ithread".
>
> The difference between the "ithread" and "interrupt filter" things is
> that ithread has its own thread context, while interrupt handling
> through interrupt filter shares the same kernel stack.
>
> So, for ithread, we should use the MTX_DEF, which don't disable
> interrupt, and for "interrupt filter", we should use the MTX_SPIN, which
> disable interrupt.
>
> What really confuses me is that I don't really see how owning an
> "independent" thread context(i.e ithread) makes a thread run in the
> "top-half" and how sharing the same kernel stack makes a thread run in
> the "bottom-half".
>
> I did read your long explanation in the previous mail. For the non-SMP
> case, the "top-half/bottom-half" model goes well and I understand how
> the *code* path/*data* path things go. But I cannot still fully
> understand the model for the SMP case. Maybe you can draw something like
>
> ----- -----
> | |<-- top-half | | <-- top-half
> | | | |
> | | | |
> | | | |
> | |<-- bottom-half | | <-- bottom-half
> ----- -----
> CPU1 CPU2
>
> to make things less abstract.
>
> Thanks,
> Yubin Ruan
>
>> It's not *great* to suspend here, but all your alternatives are
>> *also* bad:
>>
>> * You may grab incoming data and stuff it into a ring buffer, and
>> schedule some other thread to handle it later. But if the ring
>> buffer is full you have a problem, and all you have done is push
>> the actual processing off to another thread, adding more overhead.
>>
>> * You may put the device itself on hold so that no more data can
>> come in (if it's that kind of device).
>>
>> On the other hand, if you are handling an interrupt but not in an
>> interrupt thread, you are running in the "bottom half". It is
>> therefore *not OK* to suspend. You must now use one of those
>> alternatives.
>>
>> Note that if you suspend on an MTX_DEF mutex, and your priority is
>> *higher* than the priority of whatever thread actually holds that
>> mutex now, that other thread gets a priority boost to your level
>> (priority propagation, to prevent priority inversion). So letting
>> your ithread suspend, assuming you have an ithread, is probably your
>> best bet.
>>
>> Chris
>>
>
Sorry for the ugly format. The mail client sucks.
Yubin Ruan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?04b3328f-7bfb-bb70-c665-b43038cdd768>