Date: Wed, 30 Oct 2013 12:50:47 +0100 From: Andre Oppermann <andre@freebsd.org> To: Navdeep Parhar <np@freebsd.org>, Luigi Rizzo <rizzo@iet.unipi.it>, Randall Stewart <rrs@lakerest.net>, "freebsd-net@freebsd.org" <net@freebsd.org> Subject: Re: MQ Patch. Message-ID: <5270F297.4090001@freebsd.org> In-Reply-To: <20131030064105.GV58155@funkthat.com> References: <40948D79-E890-4360-A3F2-BEC34A389C7E@lakerest.net> <526FFED9.1070704@freebsd.org> <CA%2BhQ2%2BgTc87M0f5pvFeW_GCZDogrLkT_1S2bKHngNcDEBUeZYQ@mail.gmail.com> <52701D8B.8050907@freebsd.org> <527022AC.4030502@FreeBSD.org> <527027CE.5040806@freebsd.org> <20131030064105.GV58155@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 30.10.2013 07:41, John-Mark Gurney wrote: > Andre Oppermann wrote this message on Tue, Oct 29, 2013 at 22:25 +0100: >> b) the driver assigns the DMA rings to particular cores which by that, >> through >> a critnest++ can drive them lockless. The drivers (*if_transmit) will >> look >> up the core it got called on and push the traffic out on that DMA ring. >> The problem is the actual upper stacks affinity which is not guaranteed. >> This has to consequences: there may be reordering of packets of the same >> flow because the protocols send function happens to be called from a >> different core the second time. Or the drivers (*if_transmit) has to >> switch to the right core to complete the transmit for this flow if the >> upper stack migrated/bounced around. It is rather difficult to assure >> full affinity from userspace down through the upper stack and then to >> the driver. > > I'll point you to the paper: > http://arxiv.org/abs/1106.0443 > > Please don't reorder packets. > > Binding TX queues to cores seems not very useful, sure you can do a > lockless implementation, but is running the scheduler to change cpu's > really cheaper than paying the cost of migrating the lock? > > I'll admit I haven't run benchmarks, but I doubt it. Don't worry. My list was about the possible ways of dealing with it and their constrains/disadvantage. Packet reordering is one part of it that pretty much makes approach b) non-viable as you correctly point out. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5270F297.4090001>