From owner-freebsd-hackers@freebsd.org Tue Apr 11 17:17:11 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9B67D3AEE9 for ; Tue, 11 Apr 2017 17:17:11 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-pg0-x241.google.com (mail-pg0-x241.google.com [IPv6:2607:f8b0:400e:c05::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98D0D1320 for ; Tue, 11 Apr 2017 17:17:11 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-pg0-x241.google.com with SMTP id o123so579717pga.1 for ; Tue, 11 Apr 2017 10:17:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=S0ugCMLYngrYkykv/9b4twhqQF1hRsfGxiB1t9VdqkU=; b=VqL5kkpFzIb3aEh8SXgrf3YwpwUGlk2XIhNzgIVsJf/Fiz+DCLvro/kMOBxMbpi5I1 X6GKEmFZupM2ZBeFbsGD1/QX6tpqF16MuKAIFrEoC0+j3otgPCR92auUIYHdA0mjRLsC jnM+VaaEVcxXoBmIRCz7zRl3Crbp7qrzqUaTf8e44V+2zRaL2LWqntqBu34aTGcKhZSI xvWOgRqbZfZmGnKF1M+ZnTsx0F+V16NDLvQCBgY3HvMMYCtJpjlMEQraUMIoWHykNfLB 5lfH/4KKRJB6sV7CnSFrpAy+D1kAf7afOvIrF5/RiAxsE2GLj6g2nnfMGP5nWLEFQpoJ 2rew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=S0ugCMLYngrYkykv/9b4twhqQF1hRsfGxiB1t9VdqkU=; b=oFtWPWoJ2KvN3AZC/YUNDLi4LsPk0wC22PezqDIQcXlMtBLXzFIaW4Xfu0taw+He78 s8SW3KwA+mRtqr4wiM/3RGI9FWjItyatPQzMk9+RrfERaLWrjnbUbtVfkqkLMgYmnyJs LnL3cAYpDH+oCXOJWnN893J6TEMoU/3ONOU5JJRnbwb+eBX94WLSrbTBXlG8SYwVh/DX +e01GWICsNMk57fquIrdF/qwxKMNbuTZC0ryRJAEhj7N64Ji4MS+ha2O2NObEcrAUY9h OODlRdWkS4IYGi+jjaQwZgNdsi1lIwyJ6PcYFL3TmkgkDqyqJpDRci7HIIfKXfZ0UGnV pWYA== X-Gm-Message-State: AFeK/H2X85yYknyQkyGG8MiypSu5QDQM5rXH5DVrjAPfY4ZnImPqqmFz7DHH0pXMMcB9Nw== X-Received: by 10.99.212.69 with SMTP id i5mr62137301pgj.36.1491931031107; Tue, 11 Apr 2017 10:17:11 -0700 (PDT) Received: from [192.168.0.100] ([110.64.91.54]) by smtp.gmail.com with ESMTPSA id v17sm31868381pgc.20.2017.04.11.10.17.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Apr 2017 10:17:09 -0700 (PDT) Subject: Re: Understanding the FreeBSD locking mechanism To: Chris Torek , imp@bsdimp.com References: <201704100426.v3A4QR9Q042761@elf.torek.net> <4768e26a-cdec-6f40-1463-ece9847ca34d@gmail.com> Cc: ed@nuxi.nl, freebsd-hackers@freebsd.org, rysto32@gmail.com From: Yubin Ruan Message-ID: <04b3328f-7bfb-bb70-c665-b43038cdd768@gmail.com> Date: Wed, 12 Apr 2017 01:17:02 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <4768e26a-cdec-6f40-1463-ece9847ca34d@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2017 17:17:11 -0000 On 2017/4/12 1:15, Yubin Ruan wrote: > > Thanks for your reply. I have read your mails and your discussion with > Konstantin Belousov > > On 2017/4/10 12:26, Chris Torek wrote: >>>>> Is it true that a thread holding a MTX_DEF mutex can be descheduled? >> >>>> Yes, they can be descheduled. But that's not a problem. No other >>>> thread can acquire the MTX_DEF lock. ... >> >>> Does that imply that MTX_DEF should not be used in something like >>> interrupt handler? Putting an interrupt handler into sleep doesn't >>> make so much sense. >> >> Go back to the old top-half / bottom-half model, and consider that >> now that there are interrupt *threads*, your ithread is also in the >> "top half". It's therefore OK to suspend. ("Sleep" is not quite >> correct here: a mutex wait is not a "sleep" state but instead is >> just a waiting, not-scheduled-to-run state. The precise difference >> is irrelevant at this level though.) > > I don't truely understand the "top-half/bottom-half" model you proposed, > but I think I get the idea of how things work now. Basically, we can > assume that if a thread is in the "bottom-half", then it should never > suspend(or, in the other words, be preempted). This is the case of the > "interrupt filter" in FreeBSD. On the other hand, if a thread is in the > "top-half", then it is safe to suspend/block. This is the case of the > "ithread". > > The difference between the "ithread" and "interrupt filter" things is > that ithread has its own thread context, while interrupt handling > through interrupt filter shares the same kernel stack. > > So, for ithread, we should use the MTX_DEF, which don't disable > interrupt, and for "interrupt filter", we should use the MTX_SPIN, which > disable interrupt. > > What really confuses me is that I don't really see how owning an > "independent" thread context(i.e ithread) makes a thread run in the > "top-half" and how sharing the same kernel stack makes a thread run in > the "bottom-half". > > I did read your long explanation in the previous mail. For the non-SMP > case, the "top-half/bottom-half" model goes well and I understand how > the *code* path/*data* path things go. But I cannot still fully > understand the model for the SMP case. Maybe you can draw something like > > ----- ----- > | |<-- top-half | | <-- top-half > | | | | > | | | | > | | | | > | |<-- bottom-half | | <-- bottom-half > ----- ----- > CPU1 CPU2 > > to make things less abstract. > > Thanks, > Yubin Ruan > >> It's not *great* to suspend here, but all your alternatives are >> *also* bad: >> >> * You may grab incoming data and stuff it into a ring buffer, and >> schedule some other thread to handle it later. But if the ring >> buffer is full you have a problem, and all you have done is push >> the actual processing off to another thread, adding more overhead. >> >> * You may put the device itself on hold so that no more data can >> come in (if it's that kind of device). >> >> On the other hand, if you are handling an interrupt but not in an >> interrupt thread, you are running in the "bottom half". It is >> therefore *not OK* to suspend. You must now use one of those >> alternatives. >> >> Note that if you suspend on an MTX_DEF mutex, and your priority is >> *higher* than the priority of whatever thread actually holds that >> mutex now, that other thread gets a priority boost to your level >> (priority propagation, to prevent priority inversion). So letting >> your ithread suspend, assuming you have an ithread, is probably your >> best bet. >> >> Chris >> > Sorry for the ugly format. The mail client sucks. Yubin Ruan