From owner-svn-src-head@FreeBSD.ORG Thu May 21 16:23:23 2009 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2EDD106567F; Thu, 21 May 2009 16:23:23 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 466AD8FC29; Thu, 21 May 2009 16:23:23 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local (pooker.samsco.org [168.103.85.57]) by pooker.samsco.org (8.14.2/8.14.2) with ESMTP id n4LGNFj9080750; Thu, 21 May 2009 10:23:16 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4A157FF3.8020408@samsco.org> Date: Thu, 21 May 2009 09:23:15 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Kostik Belousov References: <3bbf2fe10905210629p46c7a204v6863aaba77354462@mail.gmail.com> <20090521.094100.70797067.imp@bsdimp.com> <4A157919.7040103@samsco.org> <200905211211.00168.jhb@freebsd.org> <20090521161535.GQ1927@deviant.kiev.zoral.com.ua> In-Reply-To: <20090521161535.GQ1927@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.6 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: src-committers@freebsd.org, John Baldwin , svn-src-all@freebsd.org, attilio@freebsd.org, rwatson@freebsd.org, svn-src-head@freebsd.org, "M. Warner Losh" Subject: Re: svn commit: r192535 - head/sys/kern X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2009 16:23:25 -0000 Kostik Belousov wrote: > On Thu, May 21, 2009 at 12:10:59PM -0400, John Baldwin wrote: >> On Thursday 21 May 2009 11:54:01 am Scott Long wrote: >>> M. Warner Losh wrote: >>>> In message: >>>> Robert Watson writes: >>>> : On Thu, 21 May 2009, John Baldwin wrote: >>>> : >>>> : >>>> Move the M_WAITOK flag in notify() into an M_NOWAIT one in order >> to >>>> : > match >>>> : >>>> the behaviour alredy present with the further malloc() call in >>>> : >>>> devctl_notify(). >>>> : >>>> This fixes a bug in the CAM layer where the camisr handler >> finished to >>>> : >>>> call camperiphfree() (and subsequently destroy_dev() resulting in >> a new >>>> : >>>> dev notify) while the xpt lock is held. >>>> : >>> This is wrong. You cannot call destroy_dev() while holding any >> mutex. >>>> : >>> Taking this into account, it makes no sense to use M_NOWAIT in >> notify(). >>>> : >> >>>> : >> As long as devctl_notify() also calls M_NOWAIT and if not available >> skips >>>> : >> "silently" it just does the same thing, I think this approach is more >>>> : >> consistent. >>>> : >> >>>> : >> It remains, though, the fact to fix CAM when calling destroy_dev(). >> Maybe >>>> : >> we should add a witness_warn() there? >>>> : > >>>> : > I agree with kib, this should be reverted and CAM fixed instead. I >> also >>>> : > agree that M_NOWAIT use should be limited where possible. >>>> : >>>> : devctl_notify() probably needs to grow a sleepable flag, or perhaps we >> need >>>> : two variations, one that can sleep. >>>> >>>> devctl_notify() has expanded well beyond its original needs. Having >>>> an extra case for sleeping is the wrong way to solve this problem. >>>> Really. We're adding hacks on hacks on hacks here and we need to step >>>> back and think. >>>> >>>> I specifically didn't put in CDEV notifications into devd when I >>>> originally did it because one can get the same notification via >>>> kevents on /dev. Maybe the right answer is to remove this stuff >>>> entirely and update devd to do that instead? It isn't a lot of code, >>>> and should provide equivalent functionality without needing to change >>>> the rules of the game when it comes to destroy_dev(). Especially this >>>> close to the code slush... >>>> >>>> Comments? >>>> >>>> Warner >>> Very much in agreement here. I would also love to have destroy_dev() >>> and make_dev() be locking-neutral. Having sleepable locks in leaf APIs >>> is unpleasant for consumers of those APIs. >> destroy_dev() does not use a sleepable lock, the problem is it drains so it >> can provide sane semantics to a caller who wants to ensure that all outside >> references to a cdev are gone when it returns. You can't provide that >> without doing some sort of synchronization with the other threads trying to >> call d_open(), etc. And you most certainly can't do it if you call >> destroy_dev() while holding your driver's mutex as you then have the problem >> that some other thread could be blocked on that mutex already in your >> d_open() routine when you call destroy_dev(). These sane semantics are >> needed so drivers can do things like safely free softcs and destroy locks, >> etc. > > Another thing done inside destroy_dev is the call to the destructors > of the cdevpriv data, that never had any restrictions on the sleepable > context. > > We do have the KPI for the callers that cannot drop the locks and need > to do destroy_dev, destroy_dev_sched(9). Good to know, I'll look at destroy_dev_sched(). I'd rather not have to roll my own decoupled version. And I understand the argument about destroy_dev being a drain point for the API. However, what about create_dev()? Making that non-blocking would help a lot. Scott