Date: Fri, 27 Jul 2012 09:14:55 -0700 From: Adarsh Joshi <adarsh.joshi@qlogic.com> To: Andrew Boyer <aboyer@averesystems.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: RE: lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance Message-ID: <5E4F49720D0BAD499EE1F01232234BA87742F8ABF8@AVEXMB1.qlogic.org> In-Reply-To: <9BC66C40-6F4B-48A0-AEC0-DCAD0C6863EF@averesystems.com> References: <5E4F49720D0BAD499EE1F01232234BA877435B2E28@AVEXMB1.qlogic.org> <4C5FC147-2F49-4AE0-ADF3-C5381DE6580F@averesystems.com> <5E4F49720D0BAD499EE1F01232234BA877435B2F2E@AVEXMB1.qlogic.org> <5E4F49720D0BAD499EE1F01232234BA877435B2F66@AVEXMB1.qlogic.org>, <9BC66C40-6F4B-48A0-AEC0-DCAD0C6863EF@averesystems.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Andrew, Thanks for the response. Yes, the link is working fine. I am able to see LA= CPDUs on tcpdump. regards Adarsh ________________________________________ From: Andrew Boyer [aboyer@averesystems.com] Sent: Friday, July 27, 2012 8:11 AM To: Adarsh Joshi Cc: freebsd-net@freebsd.org Subject: Re: lacp lagg port flags do not show correctly resulting in poor t= raffic distribution/performance Adarsh, Sorry for the delay. I'm not an LACP protocol expert, but looking at your logs I don't see ql1 o= n either node receiving a lacpdu response. Are you certain that link is wo= rking? -Andrew On Jul 10, 2012, at 1:53 PM, Adarsh Joshi wrote: > Andrew, > > Here are the logs with LACP_DEBUG defined in ieee802.3ad_lacp.c, > > after typing > > Ifconfig lagg0 create > ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 net= mask 255.255.255.0 > > I compiled it as a standalone driver by the way. > > System 1: > > # ifconfig -v lagg0 > lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu = 1500 > options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4= > > ether 00:0e:1e:08:05:20 > inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255 > nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp > lag id: [(8000,00-0E-1E-08-05-20,01D3,0000,0000), > (8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D > [(8000,00-0E-1E-08-05-20,01D3,8000,000D), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D3= D > [(8000,00-0E-1E-08-05-20,01D3,8000,000C), > (8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > > > System 2: > > # ifconfig -v lagg0 > lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu = 1500 > options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4= > > ether 00:0e:1e:04:2c:f0 > inet 192.168.100.2 netmask 0xffffff00 broadcast 192.168.100.255 > nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp > lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000), > (FFFF,00-00-00-00-00-00,0000,0000,0000)] > laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D7= D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000E), > (8000,00-0E-1E-08-05-20,01D3,8000,000C)] > > > System 1 logs : > > Jul 10 10:38:49 bsd-14 kernel: lacp_attach[738] : lacp attached > Jul 10 10:38:49 bsd-14 kernel: lacp_attach[740] : lacp_defined > Jul 10 10:38:49 bsd-14 kernel: lagg0: link state changed to UP > Jul 10 10:38:49 bsd-14 kernel: ql0: media changed 0x0 -> 0x100033, ether = =3D 1, fdx =3D 1, link =3D 1 > Jul 10 10:38:49 bsd-14 kernel: ql0: -> UNSELECTED > Jul 10 10:38:49 bsd-14 kernel: ql1: media changed 0x0 -> 0x100033, ether = =3D 1, fdx =3D 1, link =3D 1 > Jul 10 10:38:49 bsd-14 kernel: ql1: -> UNSELECTED > Jul 10 10:38:49 bsd-14 kernel: lacp_select_tx_port: no active aggregator > Jul 10 10:38:50 bsd-14 kernel: ql1: port lagid=3D[(8000,00-0E-1E-08-05-20= ,01D3,8000,000D),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator created > Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-08= -05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql1: mux_state 0 -> 1 > Jul 10 10:38:50 bsd-14 kernel: ql0: port lagid=3D[(8000,00-0E-1E-08-05-20= ,01D3,8000,000C),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator created > Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-08= -05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql0: mux_state 0 -> 1 > Jul 10 10:38:51 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000D) > Jul 10 10:38:51 bsd-14 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPI= RED> > Jul 10 10:38:51 bsd-14 kernel: partner=3D(0000,00-00-00-00-00-00,0000,000= 0,0000) > Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 10:38:51 bsd-14 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPI= RED> > Jul 10 10:38:51 bsd-14 kernel: partner=3D(0000,00-00-00-00-00-00,0000,000= 0,0000) > Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 10:38:51 bsd-14 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:51 bsd-14 kernel: ql0: old pstate 2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: ql0: partner timeout changed > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 10:38:51 bsd-14 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTE= D > Jul 10 10:38:52 bsd-14 kernel: ql1: partner timeout changed > Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)],= pending 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: collecting disabled > Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-= 0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], re= fcnt 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: mux_state 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:52 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000D) > Jul 10 10:38:52 bsd-14 kernel: actor.state=3D45<ACTIVITY,AGGREGATION,DEFA= ULTED> > Jul 10 10:38:52 bsd-14 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFF= F,0000) > Jul 10 10:38:52 bsd-14 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLEC= TING,DISTRIBUTING> > Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:52 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)],= pending 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: collecting disabled > Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-= 0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], re= fcnt 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: mux_state 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:52 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 10:38:52 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 10:38:52 bsd-14 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,800= 0,000E) > Jul 10 10:38:52 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:53 bsd-14 kernel: ql1: port lagid=3D[(8000,00-0E-1E-08-05-20= ,01D3,8000,000D),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator created > Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-08= -05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql1: mux_state 0 -> 1 > Jul 10 10:38:53 bsd-14 kernel: ql0: port lagid=3D[(8000,00-0E-1E-08-05-20= ,01D3,8000,000C),(8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator created > Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-08= -05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql0: mux_state 0 -> 1 > Jul 10 10:38:54 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:54 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 10:38:54 bsd-14 kernel: actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC> > Jul 10 10:38:54 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 10:38:54 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 10:38:54 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:54 bsd-14 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION> > Jul 10 10:38:54 bsd-14 kernel: ql0: new pstate d<ACTIVITY,AGGREGATION,SYN= C> > Jul 10 10:38:55 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)],= pending 1 -> 0 > Jul 10 10:38:55 bsd-14 kernel: ql1: collecting disabled > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 1 -> 2 > Jul 10 10:38:55 bsd-14 kernel: ql1: collecting enabled > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 2 -> 3 > Jul 10 10:38:55 bsd-14 kernel: ql1: enable distributing on aggregator [(8= 000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,000= 0)], nports 0 -> 1 > Jul 10 10:38:55 bsd-14 kernel: lacp_select_active_aggregator: > Jul 10 10:38:55 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(= FFFF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 10:38:55 bsd-14 kernel: active aggregator changed > Jul 10 10:38:55 bsd-14 kernel: old (none) > Jul 10 10:38:55 bsd-14 kernel: new [(8000,00-0E-1E-08-05-20,01D3,0000,000= 0),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:55 bsd-14 kernel: Set table 1 with 1 ports > Jul 10 10:38:55 bsd-14 kernel: lacp_suppress_distributing > Jul 10 10:38:55 bsd-14 kernel: ql1: marker transmit, port=3D13, sys=3D00:= 0e:1e:08:05:20, id=3D1 > Jul 10 10:38:55 bsd-14 kernel: ql0: marker transmit, port=3D12, sys=3D00:= 0e:1e:08:05:20, id=3D1 > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 3 -> 4 > Jul 10 10:38:55 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:55 bsd-14 kernel: ql0: marker response, port=3D12, sys=3D00:= 0e:1e:08:05:20, id=3D1 > Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000D) > Jul 10 10:38:55 bsd-14 kernel: actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING,DISTRIBUTING,DEFAULTED> > Jul 10 10:38:55 bsd-14 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFF= F,0000) > Jul 10 10:38:55 bsd-14 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLEC= TING,DISTRIBUTING> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)],= pending 1 -> 0 > Jul 10 10:38:55 bsd-14 kernel: ql0: collecting disabled > Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 1 -> 2 > Jul 10 10:38:55 bsd-14 kernel: ql0: collecting enabled > Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 2 -> 3 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 10:38:55 bsd-14 kernel: actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING> > Jul 10 10:38:55 bsd-14 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,800= 0,000E) > Jul 10 10:38:55 bsd-14 kernel: partner.state=3Dd<ACTIVITY,AGGREGATION,SYN= C> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 10:38:55 bsd-14 kernel: actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING,DISTRIBUTING> > Jul 10 10:38:55 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 10:38:55 bsd-14 kernel: partner.state=3D1d<ACTIVITY,AGGREGATION,SY= NC,COLLECTING> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0 > Jul 10 10:38:55 bsd-14 kernel: ql0: old pstate d<ACTIVITY,AGGREGATION,SYN= C> > Jul 10 10:38:55 bsd-14 kernel: ql0: new pstate 3d<ACTIVITY,AGGREGATION,SY= NC,COLLECTING,DISTRIBUTING> > Jul 10 10:38:56 bsd-14 kernel: ql0: enable distributing on aggregator [(8= 000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,000= 0)], nports 0 -> 1 > Jul 10 10:38:56 bsd-14 kernel: lacp_select_active_aggregator: > Jul 10 10:38:56 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(= FFFF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 10:38:56 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(= 8000,00-0E-1E-04-2C-F0,0213,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 10:38:56 bsd-14 kernel: active aggregator changed > Jul 10 10:38:56 bsd-14 kernel: old [(8000,00-0E-1E-08-05-20,01D3,0000,000= 0),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:56 bsd-14 kernel: new [(8000,00-0E-1E-08-05-20,01D3,0000,000= 0),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > Jul 10 10:38:56 bsd-14 kernel: Set table 0 with 1 ports > Jul 10 10:38:56 bsd-14 kernel: lacp_suppress_distributing > Jul 10 10:38:56 bsd-14 kernel: ql1: marker transmit, port=3D13, sys=3D00:= 0e:1e:08:05:20, id=3D2 > Jul 10 10:38:56 bsd-14 kernel: ql0: marker transmit, port=3D12, sys=3D00:= 0e:1e:08:05:20, id=3D2 > Jul 10 10:38:56 bsd-14 kernel: ql0: mux_state 3 -> 4 > Jul 10 10:38:56 bsd-14 kernel: ql0: marker response, port=3D12, sys=3D00:= 0e:1e:08:05:20, id=3D2 > Jul 10 10:38:59 bsd-14 kernel: lacp_transit_expire > ^C > > > > > System 2 logs : > > Jul 10 02:38:24 bsd-15 kernel: lacp_attach[738] : lacp attached > Jul 10 02:38:24 bsd-15 kernel: lacp_attach[740] : lacp_defined > Jul 10 02:38:24 bsd-15 kernel: lagg0: link state changed to UP > Jul 10 02:38:24 bsd-15 kernel: ql0: media changed 0x0 -> 0x100033, ether = =3D 1, fdx =3D 1, link =3D 1 > Jul 10 02:38:24 bsd-15 kernel: ql0: -> UNSELECTED > Jul 10 02:38:24 bsd-15 kernel: ql1: media changed 0x0 -> 0x100033, ether = =3D 1, fdx =3D 1, link =3D 1 > Jul 10 02:38:24 bsd-15 kernel: ql1: -> UNSELECTED > Jul 10 02:38:24 bsd-15 kernel: lacp_select_tx_port: no active aggregator > Jul 10 02:38:25 bsd-15 kernel: ql1: port lagid=3D[(8000,00-0E-1E-04-2C-F0= ,0213,8000,000F),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator created > Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-04= -2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql1: mux_state 0 -> 1 > Jul 10 02:38:25 bsd-15 kernel: ql0: port lagid=3D[(8000,00-0E-1E-04-2C-F0= ,0213,8000,000E),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator created > Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-04= -2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql0: mux_state 0 -> 1 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 02:38:26 bsd-15 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPI= RED> > Jul 10 02:38:26 bsd-15 kernel: partner=3D(0000,00-00-00-00-00-00,0000,000= 0,0000) > Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacp_sm_rx_update_ntt: assert ntt > Jul 10 02:38:26 bsd-15 kernel: ql0: old pstate 2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: ql0: new pstate 85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 02:38:26 bsd-15 kernel: ql0: partner timeout changed > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 02:38:26 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 02:38:26 bsd-15 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:26 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000F) > Jul 10 02:38:26 bsd-15 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPI= RED> > Jul 10 02:38:26 bsd-15 kernel: partner=3D(0000,00-00-00-00-00-00,0000,000= 0,0000) > Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:26 bsd-15 kernel: ql0: collecting disabled > Jul 10 02:38:26 bsd-15 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-= 0E-1E-04-2C-F0,0213, 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], r= efcnt 1 -> 0 > > Jul 10 02:38:26 bsd-15 kernel: ql0: mux_state 1 -> 0 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 02:38:26 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 02:38:26 bsd-15 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:27 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:27 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 02:38:27 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,800= 0,000E) > Jul 10 02:38:27 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:27 bsd-15 kernel: ql0: old pstate 85<ACTIVITY,AGGREGATION,EX= PIRED> > Jul 10 02:38:27 bsd-15 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTE= D > Jul 10 02:38:27 bsd-15 kernel: ql1: partner timeout changed > Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-04-2C-F0, 0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)]= , pending 1 -> 0 > Jul 10 02:38:27 bsd-15 kernel: ql1: collecting disabled > Jul 10 02:38:27 bsd-15 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-= 0E-1E-04-2C-F0,0213, 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], r= efcnt 1 -> 0 > > Jul 10 02:38:27 bsd-15 kernel: ql1: mux_state 1 -> 0 > Jul 10 02:38:27 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:27 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000F) > Jul 10 02:38:27 bsd-15 kernel: actor.state=3D45<ACTIVITY,AGGREGATION,DEFA= ULTED> > Jul 10 02:38:27 bsd-15 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFF= F,0000) > Jul 10 02:38:27 bsd-15 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLEC= TING,DISTRIBUTING> > Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:27 bsd-15 kernel: ql0: port lagid=3D[(8000,00-0E-1E-04-2C-F0= ,0213,8000,000E),(8000,00-0E-1E-08-05-20,01D3,8000,000C)] > Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator created > Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-04= -2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)] > Jul 10 02:38:27 bsd-15 kernel: ql0: mux_state 0 -> 1 > Jul 10 02:38:28 bsd-15 kernel: ql1: port lagid=3D[(8000,00-0E-1E-04-2C-F0= ,0213,8000,000F),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator created > Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-04= -2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:28 bsd-15 kernel: ql1: mux_state 0 -> 1 > Jul 10 02:38:29 bsd-15 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-04-2C-F0, 0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)]= , pending 1 -> 0 > Jul 10 02:38:29 bsd-15 kernel: ql0: collecting disabled > Jul 10 02:38:29 bsd-15 kernel: ql0: mux_state 1 -> 2 > Jul 10 02:38:29 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:29 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 02:38:29 bsd-15 kernel: actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC> > Jul 10 02:38:29 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 02:38:29 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION> > Jul 10 02:38:29 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,= 000C) > Jul 10 02:38:30 bsd-15 kernel: actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,800= 0,000E) > Jul 10 02:38:30 bsd-15 kernel: partner.state=3Dd<ACTIVITY,AGGREGATION,SYN= C> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:30 bsd-15 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION> > Jul 10 02:38:30 bsd-15 kernel: ql0: new pstate 1d<ACTIVITY,AGGREGATION,SY= NC,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,= 00-0E-1E-04-2C-F0, 0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)]= , pending 1 -> 0 > Jul 10 02:38:30 bsd-15 kernel: ql1: collecting disabled > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 1 -> 2 > Jul 10 02:38:30 bsd-15 kernel: ql1: collecting enabled > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 2 -> 3 > Jul 10 02:38:30 bsd-15 kernel: ql1: enable distributing on aggregator [(8= 000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,000= 0)], nports 0 -> 1 > Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator: > Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(= FFFF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 02:38:30 bsd-15 kernel: active aggregator changed > Jul 10 02:38:30 bsd-15 kernel: old (none) > Jul 10 02:38:30 bsd-15 kernel: new [(8000,00-0E-1E-04-2C-F0,0213,0000,000= 0),(FFFF,00-00-00 00-00-00,0000,0000,0000)] > Jul 10 02:38:30 bsd-15 kernel: Set table 1 with 1 ports > Jul 10 02:38:30 bsd-15 kernel: lacp_suppress_distributing > Jul 10 02:38:30 bsd-15 kernel: ql1: marker transmit, port=3D15, sys=3D00:= 0e:1e:04:2c:f0, id=3D1 > Jul 10 02:38:30 bsd-15 kernel: ql0: marker transmit, port=3D14, sys=3D00:= 0e:1e:04:2c:f0, id=3D1 > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 3 -> 4 > Jul 10 02:38:30 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:30 bsd-15 kernel: ql0: marker response, port=3D14, sys=3D00:= 0e:1e:04:2c:f0, id=3D1 > Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000F) > Jul 10 02:38:30 bsd-15 kernel: actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING,DISTRIBUTING,DEFAULTED> > Jul 10 02:38:30 bsd-15 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFF= F,0000) > Jul 10 02:38:30 bsd-15 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLEC= TING,DISTRIBUTING> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:30 bsd-15 kernel: ql0: collecting enabled > Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 2 -> 3 > Jul 10 02:38:30 bsd-15 kernel: ql0: enable distributing on aggregator [(8= 000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,000= 0)], nports 0 -> 1 > Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator: > Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(= 8000,00-0E-1E-08-05-20,01D3,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(= FFFF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1 > Jul 10 02:38:30 bsd-15 kernel: active aggregator not changed > Jul 10 02:38:30 bsd-15 kernel: new [(8000,00-0E-1E-04-2C-F0,0213,0000,000= 0),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 3 -> 4 > Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,= 000E) > Jul 10 02:38:30 bsd-15 kernel: actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC= ,COLLECTING,DISTRIBUTING> > Jul 10 02:38:30 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,800= 0,000C) > Jul 10 02:38:30 bsd-15 kernel: partner.state=3D1d<ACTIVITY,AGGREGATION,SY= NC,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0 > Jul 10 02:38:33 bsd-15 kernel: lacp_transit_expire > ^C > > Let me know if you need more info. > > Thanks > Adarsh > > > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org= ] On Behalf Of Adarsh Joshi > Sent: Tuesday, July 10, 2012 10:08 AM > To: Andrew Boyer > Cc: freebsd-net@freebsd.org > Subject: RE: lacp lagg port flags do not show correctly resulting in poor= traffic distribution/performance > > Andrew, > > Thanks for the reply. > > The reason for my suspicion on the portflags is thus (extracted from the = ifconfig output in my previous mail): > > System 1: > Laggport: ql1 flags =3D 18 state =3D 7D > Laggport: ql0 flags =3D 1c state =3D 3D > > System 2: > Laggport: ql1 flags =3D 1c state =3D 7D > Laggport: ql0 flags =3D 18 state =3D 3D > > I should have explained my setup to you before. Here it is. > Both the ql0 interfaces of the 2 systems are connected using a single cab= le and ql1 interfaces of the 2 systems are connected using a single cable. > > System 1 System 2 > ql0 <=3D=3D=3D=3D=3D=3D=3D> ql0 > ql1 <=3D=3D=3D=3D=3D=3D=3D> ql1 > > With this setup, I don't think it is possible for ports ql0 to talk to th= eir partners (each other) and ql1 ports not getting a response from their p= artner and still get the lagg configuration I have posted. > > I thought the portflags are dependent on the LACP state. But I see differ= ent flags for the same LACP state (For the state 7D, ql1 on system 1 shows = flags =3D 18 and ql1 on system 2 shows flags =3D 1c). > > Or is my understanding totally wrong? > > I will send the LACP_DEBUG logs within the hour. > > Thanks > Adarsh > > From: Andrew Boyer [mailto:aboyer@averesystems.com] > Sent: Tuesday, July 10, 2012 5:57 AM > To: Adarsh Joshi > Cc: freebsd-net@freebsd.org > Subject: Re: lacp lagg port flags do not show correctly resulting in poor= traffic distribution/performance > > > On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote: > > > Hi, > > I am trying to configure lacp lagg interfaces with 2 systems connected ba= ck to back as follows: > > Ifconfig lagg0 create > Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 net= mask 255.255.255.0 > > Sometimes, the lag interface comes up correctly but sometimes the laggpor= t flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>= , it shows values of 18. I have seen similar issues reported on various for= ums with no solution. > Looking at the lagg driver code and reading the standard, I thought the l= aggport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in f= ile ieee8023ad_lacp.h. But the following ifconfig -v output does not make a= ny sense to me. > > My concern is that when all the interfaces show flags as 1c, the traffic = is distributed across both the interfaces uniformly and I get aggregated th= roughput. If not, the traffic flows only on 1 interface. > > Is this a bug? How do I solve this? Or am I doing something wrong? > > I am using Free-BSD 9.0 release. > > System 1: > # ifconfig -v lagg0 > lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000), > (8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D > [(8000,00-0E-1E-08-05-20,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D3D > [(8000,00-0E-1E-08-05-20,0213,8000,000E), > (8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > > System 2: > > # ifconfig -v lagg0 > lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000), > (FFFF,00-00-00-00-00-00,0000,0000,0000)] > laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D7D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000E), > (8000,00-0E-1E-08-05-20,0213,8000,000E)] > > > thanks > Adarsh > > I don't think you have a port flags problem per se; the flags are correct= ly displaying the state of the lagg. Your problem is that your systems are= n't negotiating the correct lagg configuration. Each tuple after the laggp= ort represents the [(actor state),(partner state)]. Ports ql0 have been ab= le to talk to their partners (each other). Neither ql1 port has seen a res= ponse from a partner, though. > > You could try restarting the state machine on one box with 'ifconfig lagg= 0 laggproto lacp'. To see the negotiation you'll need to rebuild your kern= el with '#define LACP_DEBUG 1' added to the top of sys/net/ieee802.3ad_lacp= .c. Or upgrade to a newer stable snapshot that has the net.lacp_debug sysc= tl and turn it on. > > Or just turn off LACP. What does it get you in this configuration? > > Hope this helps, > Andrew > > -------------------------------------------------- > Andrew Boyer aboyer@averesystems.com<mailto:aboyer@averesystems.com= > > > > > > > ________________________________ > This message and any attached documents contain information from QLogic C= orporation or its wholly-owned subsidiaries that may be confidential. If yo= u are not the intended recipient, you may not read, copy, distribute, or us= e this information. If you have received this transmission in error, please= notify the sender immediately by reply e-mail and then delete this message= . > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > This message and any attached documents contain information from QLogic C= orporation or its wholly-owned subsidiaries that may be confidential. If yo= u are not the intended recipient, you may not read, copy, distribute, or us= e this information. If you have received this transmission in error, please= notify the sender immediately by reply e-mail and then delete this message= . > -------------------------------------------------- Andrew Boyer aboyer@averesystems.com This message and any attached documents contain information from QLogic Cor= poration or its wholly-owned subsidiaries that may be confidential. If you = are not the intended recipient, you may not read, copy, distribute, or use = this information. If you have received this transmission in error, please n= otify the sender immediately by reply e-mail and then delete this message.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5E4F49720D0BAD499EE1F01232234BA87742F8ABF8>