Date: Mon, 20 Jun 2011 22:55:43 +0900 From: Takuya ASADA <syuu@dokukino.com> To: soc-status@freebsd.org Cc: "Robert N. M. Watson" <rwatson@freebsd.org>, George Neville-Neil <gnn@freebsd.org>, Kazuya Goda <gockzy@gmail.com> Subject: Re: Weekly status report (14th June) Message-ID: <BANLkTikPNjj_m=KPtm-4KRjpmSw4wiVHxA@mail.gmail.com> In-Reply-To: <BANLkTik14bLzt1Z8-ABqnOMYPwMvJW1B9Q@mail.gmail.com> References: <BANLkTik14bLzt1Z8-ABqnOMYPwMvJW1B9Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I just realized last report is not completed. Just following one line: 2011/6/14 Takuya ASADA <syuu@dokukino.com>: > Here's status update from last week: > > *Multi-queue NIC emulation support on bpf > http://p4web.freebsd.org/@@194710?ac=3D10 > > I was considering how to implement/test multi-queue tap device, but I > didn't get good idea to do that. > But I get alternate idea to emulate multi-queue network device and the > way to support it on bpf, using "RPS" which is Kazuya GODA's project. > The benefit of it is The benefit of it is, we can use any mono-queue NIC for Multiqueue BPF testing, and we won't need any additional application that tun/tap needed. > I re-factored his implementation and renamed it "SOFTRSS". > On ethernet layer, it calculates flowid on m2cpuid function, then > selects cpuid. That is the idea of RPS, but I added few more things to > support multiqueue bpf. > I added > m->m_pkthdr.rxqid =3D *cpuid; > on m2cpuid function described above, also added > ifp->if_rxq_num =3D netisr_get_cpucount(); > on BIOCENAQMASK ioctl in bpf.c. > It's bit strange to set if_rxq_num in bpf, but until we don't have > driver independent struct ifnet initialize code probably it's the best > place to do so. > Otherwise we have to change all device drivers. > > To prevent rewriting all device drivers, and also to determine RSS > enabled device, I added IFCAP_MULTIQUEUE. > I added the capability on igb, and added following code on > BIOCENAQMASK ioctl in bpf: > #ifdef SOFTRSS > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0if (!(ifp->if_capenable & IFCAP_MULTIQUEUE)) { > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifp->if_rxq_num =3D netisr_get_cpucou= nt(); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifp->if_capabilities |=3D IFCAP_MULTI= QUEUE; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifp->if_capenable |=3D IFCAP_MULTIQUE= UE; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0} > #else > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0if (!(ifp->if_capenable & IFCAP_MULTIQUEUE)) { > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0error =3D EINVAL; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0} > #endif > > This means, > - If the NIC supports RSS, multi-queue bpf just works. > - If the NIC doesn't supports RSS, and SOFTRSS is enabled, bpf sets > rxq_num and adds capability on the if, multi-queue bpf works. > - If the NIC doesn't supports RSS, and SOFTRSS is disabled, > BIOCENAQMASK fails, multi-queue bpf doesn't work. > > I think the code is still dirty, maybe we shouldn't modify struct > ifnet in bpf, etc. > Also, I suppose there're few more missing features, ex) TX packet handlin= g. > > *replace mtx with rmlock > Haven't looking into it yet, it stacks somewhere, possibly deadlock occur= red. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BANLkTikPNjj_m=KPtm-4KRjpmSw4wiVHxA>