From owner-freebsd-stable@FreeBSD.ORG Thu Feb 23 13:22:15 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B36C71065670; Thu, 23 Feb 2012 13:22:15 +0000 (UTC) (envelope-from Kashyap.Desai@lsi.com) Received: from na3sys009aog110.obsmtp.com (na3sys009aog110.obsmtp.com [74.125.149.203]) by mx1.freebsd.org (Postfix) with ESMTP id B7C358FC13; Thu, 23 Feb 2012 13:22:14 +0000 (UTC) Received: from paledge01.lsi.com ([192.19.193.42]) (using TLSv1) by na3sys009aob110.postini.com ([74.125.148.12]) with SMTP ID DSNKT0Y9hlkV3GddnhIt39PwVHD848jt0Sma@postini.com; Thu, 23 Feb 2012 05:22:15 PST Received: from PALCAS01.lsi.com (128.94.213.117) by PALEDGE01.lsi.com (192.19.193.42) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 23 Feb 2012 08:26:57 -0500 Received: from inbexch01.lsi.com (135.36.98.37) by PALCAS01.lsi.com (128.94.213.117) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 23 Feb 2012 08:22:11 -0500 Received: from inbmail01.lsi.com ([135.36.98.64]) by inbexch01.lsi.com ([135.36.98.37]) with mapi; Thu, 23 Feb 2012 18:52:08 +0530 From: "Desai, Kashyap" To: Konstantin Belousov Date: Thu, 23 Feb 2012 18:52:07 +0530 Thread-Topic: mpslsi0 : Trying sleep, but thread marked as sleeping prohibited Thread-Index: AczyDROYsJKb9H2BQMOCoaNj4iTZKAAIFAPA Message-ID: References: <20120222191519.GT55074@deviant.kiev.zoral.com.ua> <20120223092457.GB55074@deviant.kiev.zoral.com.ua> In-Reply-To: <20120223092457.GB55074@deviant.kiev.zoral.com.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-scsi@freebsd.org" , "Kenneth D. Merry" , "Justin T. Gibbs" , freebsd-stable , "McConnell, Stephen" Subject: RE: mpslsi0 : Trying sleep, but thread marked as sleeping prohibited X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 13:22:15 -0000 > -----Original Message----- > From: Konstantin Belousov [mailto:kostikbel@gmail.com] > Sent: Thursday, February 23, 2012 2:55 PM > To: Desai, Kashyap > Cc: freebsd-scsi@freebsd.org; freebsd-stable; Justin T. Gibbs; Kenneth > D. Merry; McConnell, Stephen > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping > prohibited >=20 > On Thu, Feb 23, 2012 at 05:52:12AM +0530, Desai, Kashyap wrote: > > > > > > > -----Original Message----- > > > From: Konstantin Belousov [mailto:kostikbel@gmail.com] > > > Sent: Thursday, February 23, 2012 12:45 AM > > > To: Desai, Kashyap > > > Cc: freebsd-scsi@freebsd.org; freebsd-stable; Justin T. Gibbs; > > > Kenneth D. Merry; McConnell, Stephen > > > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping > > > prohibited > > > > > > On Wed, Feb 22, 2012 at 07:36:42PM +0530, Desai, Kashyap wrote: > > > > Hi, > > > > > > > > I am doing some code changes in mps dirver. While working on those > > > changes, I come to know about something which is new to me. > > > > Some expert help is required to clarify my doubt. > > > > > > > > 1. When any irq is register with FreeBSD OS, it sets " > TDP_NOSLEEPING" > > > > pflag. It means though irq in freebsd is treated as thread, We > > > > cannot > > > sleep in IRQ because of " "TDP_NOSLEEPING " set. > > > > 2. In mps driver we have below code snippet in ISR routine. > > > > > > > > > > > > mps_dprint(sc, MPS_TRACE, "%s\n", __func__); > > > > mps_lock(sc); > > > > mps_intr_locked(data); > > > > mps_unlock(sc); > > > > > > > > I wonder why there is no issue with above code ? Theoretical we > > > > cannot sleep in ISR. (as explained in #1) Any thoughts ? > > > > > > > > > > > > 3. I recently added few place msleep() instead of DELAY in ISR > > > > context and I see " Trying sleep, but thread marked as sleeping > prohibited". > > > > > > > FreeBSD has several basic ways to prevent a thread from executing on > > > CPU. > > > They mostly fall into two categories: bounded sleep, sometimes > > > called blocking, and unbounded sleep, usually abbreviated as sleep. > > > The bounded there refers to amount of code executed by other thread > > > that hold resource preventing blocked thread from making a progress. > > > > > > Examples of the blocking primitives are mutexes, rw locks and rm > locks. > > > The blocking is not counted as sleeping, so interrupt threads, which > > > are designated as non-sleeping, still can lock mutexes. > > Thanks for the tech help. . > > > > As per you comment, So now I understood as "TDP_NOSLEEPING" is only > > for unbounded sleep restriction. Just curious to know, What is a > > reason that thread can do blocking sleep but can't do unbounded sleep > > ? Since technically we introduced sleeping restriction on interrupt > > thread is to avoid starvation and that can be fit with either of the > > sleep type. Is this not true ? > No, not to avoid starvation. >=20 > The intent of the blocking primitives is to acquire resources for > limited amount of time. In other words, you never take a mutex for > undefinitely long computation process. On the other hand, msleep sleep > usually has no limitations. I got same reply from Ed Schouten. I agree and understood your note. Thanks= for poring knowledge on this area. _but_ only query is when thread take mutex, we don't know when it will rele= ase. So holding time of mutex is really not known. In case of some bad code, where thread took mutex and not release within sh= ort time. This can eventually match upto msleep restriction as well. Do we have any checks that thread took long time holding mutext ? Similar = to linux where spinlock has been not release in some specific time, they du= mp warnings with backtrace. ~ Kashyap >=20 > You do not want the interrupt thread to be put off the processor for > undefined time, so sleep is prohibited. >=20 > Another issue is that sleeping locks do not do priority propagation to > the resource owners, while turnstiles used for blocking do. This way, > interrupt thread waiting for mutex donates its priority to the current > mutex owner, or at least it shall do. >=20 > > > > I will be able to progress on my work based on your comment. A much > thanks for correcting my doubt. > > > > ~ Kashyap > > > > > > > > Examples of the sleeping primitives are msleep(), sx locks, lockmgr > > > locks and conditional variables. > > > > > > In essence, the locking facilities are split into several classes > > > that form the hierarchy, and you cannot legally obtain the lock of > > > higher class while holding a lock of lower class: > > > spin mutexes -> blocking locks -> sleeping locks. > > > It establishes some meta-order on the all locks. > > > > > > Does this make sense ? > >