Date: Sat, 5 Dec 2009 03:46:03 +0000 From: Brandon Gooch <jamesbrandongooch@gmail.com> To: Benjamin Kaduk <kaduk@mit.edu>, freebsd-net@freebsd.org, Bernhard Schmidt <bschmidt@techwires.net> Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock Message-ID: <179b97fb0912041946j13760914ycc2c5145d8df9ee@mail.gmail.com> In-Reply-To: <200912050250.nB52o4P0035345@freefall.freebsd.org> References: <200912050250.nB52o4P0035345@freefall.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Dec 5, 2009 at 2:50 AM, Benjamin Kaduk <kaduk@mit.edu> wrote:
> The following reply was made to PR kern/140036; it has been noted by GNAT=
S.
>
> From: Benjamin Kaduk <kaduk@MIT.EDU>
> To: Bernhard Schmidt <bschmidt@techwires.net>, sam@freebsd.org
> Cc: bug-followup@freebsd.org
> Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_l=
ock
> =A0and iwn0 softc lock
> Date: Fri, 4 Dec 2009 21:49:42 -0500 (EST)
>
> =A0Okay, I've been getting these lockups fairly frequently -- in fact,
> =A0there was this one room where I was getting them every five minutes
> =A0or so. =A0I hypothesized that dissociation/association events might
> =A0be triggers, so I set dev.iwn.0.debug=3D-1 and started wandering aroun=
d
> =A0this building (which is chock full of APs).
> =A0I get:
> =A0panic: lock (sleep mutex) iwn0_com_lock not locked @
> =A0/usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254
>
> =A0Unfortunately, it appears to have messed up my machine pretty badly,
> =A0as only numlock blinks the LED, and I don't have a db> prompt ...
>
> =A0Trying again without the debugging spew gets a useful debugger, though=
.
> =A0"show alllocks" produces empty output (?!).
> =A0backtrace is:
> =A0kdb_enter
> =A0panic
> =A0witness_unlock
> =A0_mtx_unlock_flags
> =A0iwn_raw_xmit
> =A0ieee80211_send_probereq
> =A0beacom_miss
> =A0taskqueue_run
> =A0taskqueue_thread_loop
> =A0fork_exit
>
> =A0Looking at the coredump, I'm inclined to suspect this block of
> =A0ieee80211_proto.c (in beacon_miss() ):
> =A01.46 =A0 =A0 =A0sam =A0 1379: =A0 =A0 =A0 =A0/* XXX locking */
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01380: =A0 =A0 =A0 =A0TAILQ_FOREACH(vap=
, &ic->ic_vaps, iv_next) {
> =A01.38 =A0 =A0 =A0sam =A0 1381: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/*
> =A01.46 =A0 =A0 =A0sam =A0 1382: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * We onl=
y pass events through for sta vap's in RUN state;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01383: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
* may be too restrictive but for now this saves all the
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01384: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
* handlers duplicating these checks.
> =A01.38 =A0 =A0 =A0sam =A0 1385: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */
> =A01.46 =A0 =A0 =A0sam =A0 1386: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (vap->=
iv_opmode =3D=3D IEEE80211_M_STA &&
> =A01.64 =A0 =A0 =A0sam =A0 1387: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0v=
ap->iv_state >=3D IEEE80211_S_RUN &&
> =A01.46 =A0 =A0 =A0sam =A0 1388: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0v=
ap->iv_bmiss !=3D NULL)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01389: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0vap->iv_bmiss(vap);
>
>
> =A0Sam, do you have any thoughts as to why throwing a
> =A0IEEE80211_LOCK(ic)/IEEE80211_UNLOCK(ic) block around the TAILQ_FOREACH
> =A0might not be a good idea?
> =A0I'm currently running with that, which gives me a new LOR
> =A0(iwn_com_lock @ /usr/src/sys/net80211/ieee80211_scan.c:683 and
> =A0iwn0 @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3=
289)
> =A0but it hasn't locked up or panic()ed on me, yet.
>
>
> =A0I am happy to pull more information from the coredump if needed.
>
Ben, have you tried Bernhard Schmidt's driver? He's recently updated
it with a reordering of locking code:
http://svn.techwires.net/viewvc/viewvc.cgi/svnrepos/projects/freebsd/sys/de=
v/iwn/if_iwn.c?view=3Dlog
I've been using the code from his repository for a while, and it's
much more stable than what's in the tree currently.
-Brandon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?179b97fb0912041946j13760914ycc2c5145d8df9ee>