Date: Wed, 28 Dec 2016 10:06:41 -0800 From: John Baldwin <jhb@freebsd.org> To: freebsd-net@freebsd.org Cc: Luigi Rizzo <rizzo@iet.unipi.it>, Adrian Chadd <adrian.chadd@gmail.com>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, Luigi Rizzo <luigi@freebsd.org>, Navdeep Parhar <np@freebsd.org>, Vincenzo Maffione <v.maffione@gmail.com> Subject: Re: cxgbe's native netmap support broken since r307394 Message-ID: <2698793.0b4KyHrqck@ralph.baldwin.cx> In-Reply-To: <CA%2BhQ2%2BjBJXs2gGQaiJVEkFQx=qiecQ=P637epvtex9XcnXzBEQ@mail.gmail.com> References: <20161217222812.GA4979@ox> <CAJ-VmomTTPfr85E1CqbYZ2dj1R7ZPkTFP03Z-AAd_u=5GjNb4g@mail.gmail.com> <CA%2BhQ2%2BjBJXs2gGQaiJVEkFQx=qiecQ=P637epvtex9XcnXzBEQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Why are you using MTX_SPIN? Changing the lock type to MTX_DEF would seem to be a smaller patch and probably more correct for FreeBSD. On Thursday, December 22, 2016 05:29:41 PM Luigi Rizzo wrote: > sure go ahead and thank you! > > On Thu, Dec 22, 2016 at 5:15 PM, Adrian Chadd <adrian.chadd@gmail.com> wrote: > > ok, does anyone mind if I commit it as-is? > > > > > > -a > > > > > > On 21 December 2016 at 13:37, Vincenzo Maffione <v.maffione@gmail.com> wrote: > >> Hi Luigi, > >> I attached a minimal change containing two fixes: > >> > >> - change IFNET_WLOCK into IFNET_RLOCK, to fix the cxgbe issue related > >> to this thread > >> - use the proper locking functions for the "worker_lock", unrelated > >> but needed to avoid the O.S. to trap because of a mismatch between > >> MTX_SPIN and MTX_DEF. > >> > >> Cheers, > >> Vincenzo > >> > >> 2016-12-21 20:30 GMT+01:00 Luigi Rizzo <rizzo@iet.unipi.it>: > >>> On Wed, Dec 21, 2016 at 11:15 AM, Vincenzo Maffione > >>> <v.maffione@gmail.com> wrote: > >>>> Hi, > >>>> There is no commit related to that in the FreeBSD svn or git. > >>>> > >>>> The fix has been published to the github netmap repository here > >>>> (branch master): https://github.com/luigirizzo/netmap > >>>> > >>>> What we should do is to import all the recent updates from the github > >>>> into HEAD. I can prepare a patch for HEAD, if you wish. Just let me > >>>> know. > >>> > >>> I just checked and the diff between FreeBSD head and netmap head > >>> in github is almost 3k lines due to a lot of recent refactoring. > >>> So, if there is an easy way to extract just the locking change that would > >>> be preferable as an interim solution. > >>> > >>> cheers > >>> luigi > >>> > >>>> > >>>> Cheers, > >>>> Vincenzo > >>>> > >>>> > >>>> 2016-12-20 21:45 GMT+01:00 Adrian Chadd <adrian.chadd@gmail.com>: > >>>>> hi, > >>>>> > >>>>> What's the commit? We should get it into -HEAD asap. > >>>>> > >>>>> > >>>>> -adrian > >>>>> > >>>>> > >>>>> On 20 December 2016 at 01:25, Vincenzo Maffione <v.maffione@gmail.com> wrote: > >>>>>> Ok, applied to the netmap github repo. > >>>>>> This fix will be published when Luigi does the next commit on FreeBSD. > >>>>>> > >>>>>> Cheers, > >>>>>> Vincenzo > >>>>>> > >>>>>> 2016-12-19 20:05 GMT+01:00 Navdeep Parhar <np@freebsd.org>: > >>>>>>> IFNET_RLOCK will work, thanks. > >>>>>>> > >>>>>>> Navdeep > >>>>>>> > >>>>>>> On Mon, Dec 19, 2016 at 3:21 AM, Vincenzo Maffione <v.maffione@gmail.com> wrote: > >>>>>>>> Hi Navdeep, > >>>>>>>> > >>>>>>>> Indeed, we have reviewed the code, and we think it is ok to > >>>>>>>> implement nm_os_ifnet_lock() with IFNET_RLOCK(), instead of using > >>>>>>>> IFNET_WLOCK(). > >>>>>>>> Since IFNET_RLOCK() results into sx_slock(), this should fix the issue. > >>>>>>>> > >>>>>>>> On FreeBSD, this locking is needed to protect a flag read by nm_iszombie(). > >>>>>>>> However, on Linux the same lock is also needed to protect the call to > >>>>>>>> the nm_hw_register() callback, so we prefer to have an "unified" > >>>>>>>> locking scheme, i.e. always calling nm_hw_register under the lock. > >>>>>>>> > >>>>>>>> Does this make sense to you? Would it be easy for you to make a quick > >>>>>>>> test by replacing IFNET_WLOCK with IFNET_RLOCK? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Vincenzo > >>>>>>>> > >>>>>>>> 2016-12-17 23:28 GMT+01:00 Navdeep Parhar <np@freebsd.org>: > >>>>>>>>> Luigi, Vincenzo, > >>>>>>>>> > >>>>>>>>> The last major update to netmap (r307394 and followups) broke cxgbe's > >>>>>>>>> native netmap support. The problem is that netmap_hw_reg now holds an > >>>>>>>>> rw_lock around the driver's netmap_on/off routines. It has always been > >>>>>>>>> safe for the driver to sleep during these operations but now it panics > >>>>>>>>> instead. > >>>>>>>>> > >>>>>>>>> Why is IFNET_WLOCK needed here? It seems like a regression to disallow > >>>>>>>>> sleep on the control path. > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> Navdeep > >>>>>>>>> > >>>>>>>>> begin_synchronized_op with the following non-sleepable locks held: > >>>>>>>>> exclusive rw ifnet_rw (ifnet_rw) r = 0 (0xffffffff8271d680) locked @ > >>>>>>>>> /root/ws/head/sys/dev/netmap/netmap_freebsd.c:95 > >>>>>>>>> stack backtrace: > >>>>>>>>> #0 0xffffffff810837a5 at witness_debugger+0xe5 > >>>>>>>>> #1 0xffffffff81084d88 at witness_warn+0x3b8 > >>>>>>>>> #2 0xffffffff83ef2bcc at begin_synchronized_op+0x6c > >>>>>>>>> #3 0xffffffff83f14beb at cxgbe_netmap_reg+0x5b > >>>>>>>>> #4 0xffffffff809846f1 at netmap_hw_reg+0x81 > >>>>>>>>> #5 0xffffffff809806de at netmap_do_regif+0x19e > >>>>>>>>> #6 0xffffffff8098121d at netmap_ioctl+0x7ad > >>>>>>>>> #7 0xffffffff8098682f at freebsd_netmap_ioctl+0x5f > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Vincenzo Maffione > >>>>>>>> _______________________________________________ > >>>>>>>> freebsd-net@freebsd.org mailing list > >>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Vincenzo Maffione > >>>>>> _______________________________________________ > >>>>>> freebsd-net@freebsd.org mailing list > >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >>>> > >>>> > >>>> > >>>> -- > >>>> Vincenzo Maffione > >>> > >>> > >>> > >>> -- > >>> -----------------------------------------+------------------------------- > >>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > >>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >>> TEL +39-050-2217533 . via Diotisalvi 2 > >>> Mobile +39-338-6809875 . 56122 PISA (Italy) > >>> -----------------------------------------+------------------------------- > >> > >> > >> > >> -- > >> Vincenzo Maffione > > > > -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2698793.0b4KyHrqck>