Date: Tue, 21 Dec 2010 20:00:20 +0600 From: Eugene Grosbein <egrosbein@rdtc.ru> To: Fabien Thomas <fabien.thomas@netasq.com> Cc: net@freebsd.org Subject: Re: lagg/lacp poor traffic distribution Message-ID: <4D10B2F4.1060404@rdtc.ru> In-Reply-To: <84530C06-AC2E-4E2B-BFD4-693902BB0FA6@netasq.com> References: <4D0CFEFF.3000902@rdtc.ru> <1292844095.1917.136.camel@stormi> <4D1083D6.6010707@rdtc.ru> <84530C06-AC2E-4E2B-BFD4-693902BB0FA6@netasq.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21.12.2010 19:11, Fabien Thomas wrote: >>> I had this problem with igb driver, and I found, that lagg selects >>> outgoing interface based on packet header flowid field if M_FLOWID field >>> is set. And in the igb driver code flowid is set as >>> >>> #if __FreeBSD_version >= 800000 >>> <------><------><------>rxr->fmp->m_pkthdr.flowid = que->msix; >>> <------><------><------>rxr->fmp->m_flags |= M_FLOWID; >>> #endif >>> >>> The same thing in em driver with MULTIQUEUE >>> >>> That does not give enough number of flows to balance traffic well, so I >>> commented check in if_lagg.c >>> >>> lagg_lb_start(struct lagg_softc *sc, struct mbuf *m) >>> { >>> <------>struct lagg_lb *lb = (struct lagg_lb *)sc->sc_psc; >>> <------>struct lagg_port *lp = NULL; >>> <------>uint32_t p = 0; >>> >>> //<---->if (m->m_flags & M_FLOWID) >>> //<----><------>p = m->m_pkthdr.flowid; >>> //<---->else >>> >>> and with this change I have much better load distribution across interfaces. >>> >>> Hope it helps. >> >> You are perfectly right. By disabling flow usage I've obtained load sharing >> close to even (final patch follows). Two questions: >> >> 1. Is it a bug or design problem? > > How many queues have you with igb? If it's one it will explain why the flowid is bad for load balancing with lagg. How do I know? I've read igb(4) manual page and found no words about queues within igb, nor I have any knowledge about them. > The problem is that flowid is good if the number of queue is = or a multiple of the number of lagg ports. Now I see, thanks. Eugene Grosbein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4D10B2F4.1060404>