From owner-freebsd-hackers@freebsd.org Tue Apr 11 17:15:44 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20CD4D3AD63 for ; Tue, 11 Apr 2017 17:15:44 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-pf0-x242.google.com (mail-pf0-x242.google.com [IPv6:2607:f8b0:400e:c00::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4578F01 for ; Tue, 11 Apr 2017 17:15:43 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-pf0-x242.google.com with SMTP id o126so553844pfb.1 for ; Tue, 11 Apr 2017 10:15:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=ovrJSqlRWmu3DimgzZ0CfywoGCZcEcUX70IYYI41pg0=; b=nKoeaKSTcgqgsIODVW/jfh0qH+8dPKqF6+fqPkGDr5LiSC2GPelJW55Az7aQsMfpHs 8YKkUtwA+n8USAeq9hCvYwg1VFZWECvW8Hrh932XKay2mrYRnNEGXZgDqQsdTvn68XOF 39Pd80+6YWl+idLWvF9O4Y/uuf5lFt9J9POKGZg0wTdwDY/UXqwSuXzJrSiNzhgkvtFi xiTxdbhVxyfago2oMgJNOp3738zmSzTbVWAhSiezWPwKU10kVqPTs/vZh8zzxP5nx0la omb2r/vbHICWqWXf1KPnUJy8dySMRVrpuytw1uo9GCL2foitYMmUEpSjm6owyFXsVnPU WCsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=ovrJSqlRWmu3DimgzZ0CfywoGCZcEcUX70IYYI41pg0=; b=oQwTWrQ6Jz7o//joCdz4IOjcemni1ZYtlMgoHEPCqY6ysxs1sYo0c6nrAP7RKmxMqJ SXjkjorDme55rdUtC3vuiP9sJ77o9ktP3a2n10VqzTVtPAXt1owJ/p/kLElewxZMT3uC xgw4w5ehbibj1D86wFBNZg60sKiYZbIYXNgZ1mFy/JMSbM29uk2b0UBZd2a7swfZHA0C uw3uMdMlqClaQ6lkKQ3VVQRJBOMcufGj0kQzSfWFRv09MEiizzD/vYMssRI1P9DEqaQl Q8JGhditGBfDMt3GMvhoixmp1J3NBCXh/apRTmRjIthqC9Yh9BfXt5CFZGXiwj4qIQmW N7Jw== X-Gm-Message-State: AFeK/H32wSbxEjrcvIwOAg/Zy92SUe2bGVCY9dfnXiWs4+QXarFQD9rd1E+XZM/a1/340Q== X-Received: by 10.84.218.68 with SMTP id f4mr76690785plm.146.1491930943443; Tue, 11 Apr 2017 10:15:43 -0700 (PDT) Received: from [192.168.0.100] ([110.64.91.54]) by smtp.gmail.com with ESMTPSA id 133sm25559138pfy.106.2017.04.11.10.15.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Apr 2017 10:15:42 -0700 (PDT) Subject: Re: Understanding the FreeBSD locking mechanism To: Chris Torek , imp@bsdimp.com References: <201704100426.v3A4QR9Q042761@elf.torek.net> Cc: ed@nuxi.nl, freebsd-hackers@freebsd.org, rysto32@gmail.com From: Yubin Ruan Message-ID: <4768e26a-cdec-6f40-1463-ece9847ca34d@gmail.com> Date: Wed, 12 Apr 2017 01:15:34 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <201704100426.v3A4QR9Q042761@elf.torek.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2017 17:15:44 -0000 Thanks for your reply. I have read your mails and your discussion with Konstantin Belousov On 2017/4/10 12:26, Chris Torek wrote: >>>> Is it true that a thread holding a MTX_DEF mutex can be descheduled? > >>> Yes, they can be descheduled. But that's not a problem. No other >>> thread can acquire the MTX_DEF lock. ... > >> Does that imply that MTX_DEF should not be used in something like >> interrupt handler? Putting an interrupt handler into sleep doesn't >> make so much sense. > > Go back to the old top-half / bottom-half model, and consider that > now that there are interrupt *threads*, your ithread is also in the > "top half". It's therefore OK to suspend. ("Sleep" is not quite > correct here: a mutex wait is not a "sleep" state but instead is > just a waiting, not-scheduled-to-run state. The precise difference > is irrelevant at this level though.) I don't truely understand the "top-half/bottom-half" model you proposed, but I think I get the idea of how things work now. Basically, we can assume that if a thread is in the "bottom-half", then it should never suspend(or, in the other words, be preempted). This is the case of the "interrupt filter" in FreeBSD. On the other hand, if a thread is in the "top-half", then it is safe to suspend/block. This is the case of the "ithread". The difference between the "ithread" and "interrupt filter" things is that ithread has its own thread context, while interrupt handling through interrupt filter shares the same kernel stack. So, for ithread, we should use the MTX_DEF, which don't disable interrupt, and for "interrupt filter", we should use the MTX_SPIN, which disable interrupt. What really confuses me is that I don't really see how owning an "independent" thread context(i.e ithread) makes a thread run in the "top-half" and how sharing the same kernel stack makes a thread run in the "bottom-half". I did read your long explanation in the previous mail. For the non-SMP case, the "top-half/bottom-half" model goes well and I understand how the *code* path/*data* path things go. But I cannot still fully understand the model for the SMP case. Maybe you can draw something like ----- ----- | |<-- top-half | | <-- top-half | | | | | | | | | | | | | |<-- bottom-half | | <-- bottom-half ----- ----- CPU1 CPU2 to make things less abstract. Thanks, Yubin Ruan > It's not *great* to suspend here, but all your alternatives are > *also* bad: > > * You may grab incoming data and stuff it into a ring buffer, and > schedule some other thread to handle it later. But if the ring > buffer is full you have a problem, and all you have done is push > the actual processing off to another thread, adding more overhead. > > * You may put the device itself on hold so that no more data can > come in (if it's that kind of device). > > On the other hand, if you are handling an interrupt but not in an > interrupt thread, you are running in the "bottom half". It is > therefore *not OK* to suspend. You must now use one of those > alternatives. > > Note that if you suspend on an MTX_DEF mutex, and your priority is > *higher* than the priority of whatever thread actually holds that > mutex now, that other thread gets a priority boost to your level > (priority propagation, to prevent priority inversion). So letting > your ithread suspend, assuming you have an ithread, is probably your > best bet. > > Chris >