Date: Thu, 21 May 2009 09:23:15 -0700 From: Scott Long <scottl@samsco.org> To: Kostik Belousov <kostikbel@gmail.com> Cc: src-committers@freebsd.org, John Baldwin <jhb@freebsd.org>, svn-src-all@freebsd.org, attilio@freebsd.org, rwatson@freebsd.org, svn-src-head@freebsd.org, "M. Warner Losh" <imp@bsdimp.com> Subject: Re: svn commit: r192535 - head/sys/kern Message-ID: <4A157FF3.8020408@samsco.org> In-Reply-To: <20090521161535.GQ1927@deviant.kiev.zoral.com.ua> References: <3bbf2fe10905210629p46c7a204v6863aaba77354462@mail.gmail.com> <20090521.094100.70797067.imp@bsdimp.com> <4A157919.7040103@samsco.org> <200905211211.00168.jhb@freebsd.org> <20090521161535.GQ1927@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Kostik Belousov wrote: > On Thu, May 21, 2009 at 12:10:59PM -0400, John Baldwin wrote: >> On Thursday 21 May 2009 11:54:01 am Scott Long wrote: >>> M. Warner Losh wrote: >>>> In message: <alpine.BSF.2.00.0905211610140.18790@fledge.watson.org> >>>> Robert Watson <rwatson@FreeBSD.org> writes: >>>> : On Thu, 21 May 2009, John Baldwin wrote: >>>> : >>>> : >>>> Move the M_WAITOK flag in notify() into an M_NOWAIT one in order >> to >>>> : > match >>>> : >>>> the behaviour alredy present with the further malloc() call in >>>> : >>>> devctl_notify(). >>>> : >>>> This fixes a bug in the CAM layer where the camisr handler >> finished to >>>> : >>>> call camperiphfree() (and subsequently destroy_dev() resulting in >> a new >>>> : >>>> dev notify) while the xpt lock is held. >>>> : >>> This is wrong. You cannot call destroy_dev() while holding any >> mutex. >>>> : >>> Taking this into account, it makes no sense to use M_NOWAIT in >> notify(). >>>> : >> >>>> : >> As long as devctl_notify() also calls M_NOWAIT and if not available >> skips >>>> : >> "silently" it just does the same thing, I think this approach is more >>>> : >> consistent. >>>> : >> >>>> : >> It remains, though, the fact to fix CAM when calling destroy_dev(). >> Maybe >>>> : >> we should add a witness_warn() there? >>>> : > >>>> : > I agree with kib, this should be reverted and CAM fixed instead. I >> also >>>> : > agree that M_NOWAIT use should be limited where possible. >>>> : >>>> : devctl_notify() probably needs to grow a sleepable flag, or perhaps we >> need >>>> : two variations, one that can sleep. >>>> >>>> devctl_notify() has expanded well beyond its original needs. Having >>>> an extra case for sleeping is the wrong way to solve this problem. >>>> Really. We're adding hacks on hacks on hacks here and we need to step >>>> back and think. >>>> >>>> I specifically didn't put in CDEV notifications into devd when I >>>> originally did it because one can get the same notification via >>>> kevents on /dev. Maybe the right answer is to remove this stuff >>>> entirely and update devd to do that instead? It isn't a lot of code, >>>> and should provide equivalent functionality without needing to change >>>> the rules of the game when it comes to destroy_dev(). Especially this >>>> close to the code slush... >>>> >>>> Comments? >>>> >>>> Warner >>> Very much in agreement here. I would also love to have destroy_dev() >>> and make_dev() be locking-neutral. Having sleepable locks in leaf APIs >>> is unpleasant for consumers of those APIs. >> destroy_dev() does not use a sleepable lock, the problem is it drains so it >> can provide sane semantics to a caller who wants to ensure that all outside >> references to a cdev are gone when it returns. You can't provide that >> without doing some sort of synchronization with the other threads trying to >> call d_open(), etc. And you most certainly can't do it if you call >> destroy_dev() while holding your driver's mutex as you then have the problem >> that some other thread could be blocked on that mutex already in your >> d_open() routine when you call destroy_dev(). These sane semantics are >> needed so drivers can do things like safely free softcs and destroy locks, >> etc. > > Another thing done inside destroy_dev is the call to the destructors > of the cdevpriv data, that never had any restrictions on the sleepable > context. > > We do have the KPI for the callers that cannot drop the locks and need > to do destroy_dev, destroy_dev_sched(9). Good to know, I'll look at destroy_dev_sched(). I'd rather not have to roll my own decoupled version. And I understand the argument about destroy_dev being a drain point for the API. However, what about create_dev()? Making that non-blocking would help a lot. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A157FF3.8020408>