Date: Fri, 5 Mar 2010 13:55:39 -0800 From: Pyun YongHyeon <pyunyh@gmail.com> To: Ian FREISLICH <ianf@clue.co.za> Cc: current@freebsd.org Subject: Re: dev.bce.X.com_no_buffers increasing and packet loss Message-ID: <20100305215539.GG14818@michelle.cdnetworks.com> In-Reply-To: <E1Nnesz-00040L-AQ@clue.co.za> References: <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <E1NnVaT-0003Ft-3p@clue.co.za> <E1Nnc4d-0003mB-6e@clue.co.za> <E1Nne0Q-0003uZ-OR@clue.co.za> <E1Nnesz-00040L-AQ@clue.co.za>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Mar 05, 2010 at 11:16:41PM +0200, Ian FREISLICH wrote: > Pyun YongHyeon wrote: > > Thanks for the info. Frankly, I have no idea how to explain the > > issue given that you have no heavy load. > > How many cores would be involved in handling the traffic and runnig > PF rules on this machine? There are 4x > CPU: Quad-Core AMD Opteron(tm) Processor 8354 (2194.51-MHz K8-class CPU) > In this server. I'm also using carp extensively. > pf(4) uses a single lock for processing, number of core would have no much benefit. > > I have a bce(4) patch which fixes a couple of bus_dma(9) issues as > > well as fixing some minor bugs. However I don't know whether the > > patch can fix the RX issue you're suffering from. Anyway, would you > > give it try the patch at the following URL? > > http://people.freebsd.org/~yongari/bce/bce.20100305.diff > > The patch was generated against CURRENT and you may see a message > > like "Disabling COAL_NOW timedout!" during interface up. You can > > ignore that message. > > Thanks. I'll give the patch a go on Monday when there are people > nearby if something goes wrong during the boot. I don't want to > loose the redundancy over the week end. > >From my testing on quad-port BCM5709 controller, it was stable. But I agree that your plan would be better. > Otherwise, is there another interface chip we can try? It's got I guess bce(4) and igb(4) would be one of the best controller. > an igb(4) quad port in there as well, but the performance is worse > on that chip than the bce(4) interface. It's also riddled with Yeah, I also noticed that. I think bce(4) seems to give more better performance numbers than igb(4). > vlan and other hardware offload bugs. I had good success in the > past with em(4), but it looks like igb is the PCI-e version. > It may depend on specific workloads. Last time I tried igb(4), the driver had a couple of bugs and after patching it, igb(4) also seemed to work well even though the performance was slightly slower than I initially expected. One thing I saw was using LRO on igb(4) showed slightly worse performance. Another thing for igb(4) case, it began to support multi-TX queues as well as RSS. Theoretically current multi-TX queue implementation can reorder packets such that it can give negative effects. bce(4) still lacks multi-TX queue support as well as RSS. bce(4) controllers also supports MSI-X as well as RSS so I have plan to implement it in future but it's hard to tell when I can find time to implement that.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100305215539.GG14818>