Date: Mon, 5 Aug 2013 10:17:27 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Navdeep Parhar <np@freebsd.org> Cc: Bryan Venteicher <bryanv@daemoninthecloset.org>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, Luigi Rizzo <rizzo@iet.unipi.it>, current@freebsd.org, net@freebsd.org Subject: Re: [net] protecting interfaces from races between control and data ? Message-ID: <CAJ-Vmo=Q9AqdBJ0%2B4AiX4%2BWreYuZx6VGGYw=MZ4XhMB1P2yMww@mail.gmail.com> In-Reply-To: <51FFDD1E.1000206@FreeBSD.org> References: <20130805082307.GA35162@onelab2.iet.unipi.it> <2034715395.855.1375714772487.JavaMail.root@daemoninthecloset.org> <CAJ-VmokT6YKPR7CXsoCavEmWv3W8urZu4eBVgKWaj9iMaVJFZg@mail.gmail.com> <CA%2BhQ2%2BhuoCCweq7fjoYmH3nyhmhb5DzukEdPSMtaJEWa8Ft0JQ@mail.gmail.com> <51FFDD1E.1000206@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I'm travelling back to San Jose today; poke me tomorrow and I'll brain dump what I did in ath(4) and the lessons learnt. The TL;DR version - you don't want to grab an extra lock in the read/write paths as that slows things down. Reuse the same per-queue TX/RX lock and have: * a reset flag that is set when something is resetting; that says to the queue "don't bother processing anything, just dive out"; * 'i am doing Tx / Rx' flags per queue that is set at the start of TX/RX servicing and finishes at the end; that way the reset code knows if there's something pending; * have the reset path grab each lock, set the 'reset' flag on each, then walk each queue again and make sure they're all marked as 'not doing TX/RX'. At that point the reset can occur, then the flag cna be cleared, then TX/RX can resume. -adrian On 5 August 2013 10:13, Navdeep Parhar <np@freebsd.org> wrote: > On 08/05/13 09:15, Luigi Rizzo wrote: >> On Mon, Aug 5, 2013 at 5:46 PM, Adrian Chadd <adrian@freebsd.org> wrote: >> >>> On 5 August 2013 07:59, Bryan Venteicher <bryanv@daemoninthecloset.org> >>> wrote: >>> >>>> What I've done in my drivers is: >>>> * Lock the core mutex >>>> * Clear IFF_DRV_RUNNING >>>> * Lock/unlock each queue's lock >>> >>> .. and I think that's the only sane way of doing it. >>> >> >> yeah, this was also the solution we had in mind, i was surprised >> not find this pattern in the drivers i have looked at. >> >> Also there are drivers (chelsio ?) which do not seem to have locks on the >> receive interrupt handlers ? > > This is correct. cxgbe(4) does not have any locks on rx, just a "state" > for each rx queue that's maintained with atomic ops. > > Regards, > Navdeep > > >> >> Does anyone know how linux copes with the same problem ? >> >> They seem to have an rtnl_lock() which is a global lock for all >> configuration >> of netdevices (would replace our per-interface 'core lock' above), >> but i am totally unclear on how individual tx threads and interrupt handlers >> acknowledge that they have read the change in status. >> >> cheers >> luigi >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=Q9AqdBJ0%2B4AiX4%2BWreYuZx6VGGYw=MZ4XhMB1P2yMww>