Date: Wed, 27 Jan 2010 15:43:54 +0200 From: Shteryana Shopova <syrinx@FreeBSD.org> To: Steven Hartland <killing@multiplay.co.uk> Cc: freebsd-net@freebsd.org Subject: Re: Problems getting lagg to balance using lacp Message-ID: <61b573981001270543k6514a5g8d4f679777b69b49@mail.gmail.com> In-Reply-To: <8E2B6E11A79B4958B94B44A41C7D4298@multiplay.co.uk> References: <8E2B6E11A79B4958B94B44A41C7D4298@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 27, 2010 at 2:40 PM, Steven Hartland <killing@multiplay.co.uk> wrote: > I've just setup lagg on one of our servers and run some > basic tests and although everything seems to be reporting > correctly no balancing appears to be happening. > > The following is the setup: > > FreeBSD side: > ifconfig_em0=3D"up" > ifconfig_em1=3D"up" > cloned_interfaces=3D"lagg0" > ifconfig_lagg0=3D"laggproto lacp laggport em0 laggport em1" > ipv4_addrs_lagg0=3D"10.10.1.66/24" > > Cisco side: > interface Port-channel2 > description SERVER:: test - lagg0 > switchport > switchport access vlan 96 > switchport mode access > no ip address > end > > interface GigabitEthernet3/35 > description SERVER:: test - em0 > switchport > switchport access vlan 96 > switchport mode access > no ip address > channel-protocol lacp > channel-group 2 mode active > end > > interface GigabitEthernet3/43 > description SERVER:: test - em1 > switchport > switchport access vlan 96 > switchport mode access > no ip address > channel-protocol lacp > channel-group 2 mode active > end > > Config:- > em0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 15= 00 > =C2=A0 =C2=A0 =C2=A0 options=3D19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,= VLAN_HWCSUM,TSO4> > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT <full-duplex>) > =C2=A0 =C2=A0 =C2=A0 status: active > em1: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 15= 00 > =C2=A0 =C2=A0 =C2=A0 options=3D19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,= VLAN_HWCSUM,TSO4> > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT <full-duplex>) > =C2=A0 =C2=A0 =C2=A0 status: active > lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 > =C2=A0 =C2=A0 =C2=A0 options=3D3<RXCSUM,TXCSUM> > =C2=A0 =C2=A0 =C2=A0 inet 127.0.0.1 netmask 0xff000000 lagg0: > flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > =C2=A0 =C2=A0 =C2=A0 options=3D19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,= VLAN_HWCSUM,TSO4> > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 inet 10.10.1.66 netmask 0xffffff00 broadcast 10.10.1= .255 > =C2=A0 =C2=A0 =C2=A0 inet 10.10.1.83 netmask 0xffffffff broadcast 10.10.1= .83 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect > =C2=A0 =C2=A0 =C2=A0 status: active > =C2=A0 =C2=A0 =C2=A0 laggproto lacp > =C2=A0 =C2=A0 =C2=A0 laggport: em1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUT= ING> > =C2=A0 =C2=A0 =C2=A0 laggport: em0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUT= ING> > > show lacp neighbor =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags: =C2=A0S - Device= is > requesting Slow LACPDUs =C2=A0 =C2=A0 =C2=A0 F - Device is requesting Fas= t LACPDUs > =C2=A0 =C2=A0 =C2=A0 A - Device is in Active mode =C2=A0 =C2=A0 =C2=A0 P = - Device is in Passive mode > Channel group 2 neighbors > > Partner's information: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 Partner Partner =C2=A0 LACP Partner =C2=A0Par= tner =C2=A0 Partner =C2=A0Partner > Partner > Port =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 State =C2=A0 =C2=A0 Port Priority A= dmin Key Oper Key Port Number > Port State > Gi3/35 =C2=A0 =C2=A0SA =C2=A0 =C2=A0 =C2=A0bndl =C2=A0 =C2=A0 =C2=A032768= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0x0 =C2=A0 =C2=A0 =C2=A0 0x90 =C2=A0 =C2=A0 0x= 1 > 0x3D =C2=A0Gi3/43 =C2=A0 =C2=A0SA =C2=A0 =C2=A0 =C2=A0bndl =C2=A0 =C2=A0 = =C2=A032768 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0x0 =C2=A0 =C2=A0 =C2=A0 0x90 =C2= =A0 =C2=A0 0x2 > =C2=A0 0x3D > > Stats: > > =C2=A0 =C2=A0 Interface =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Traffic =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Peak =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0Total > =C2=A0 =C2=A0 =C2=A0 =C2=A0 lagg0 =C2=A0in =C2=A0 =C2=A0 =C2=A00.351 KB/s= =C2=A0 =C2=A0 =C2=A0 =C2=A0117.349 MB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A02.695 GB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.345 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0114.183 MB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A08.103 GB > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 em1 =C2=A0in =C2=A0 =C2=A0 =C2=A00.233= KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0117.349 MB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A02.695 GB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.000 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0132.535 MB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A08.890 GB > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 em0 =C2=A0in =C2=A0 =C2=A0 =C2=A00.116= KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02.701 KB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0331.405 KB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.345 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A09.093 KB/s =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0631.883 KB > > > Name =C2=A0 =C2=A0Mtu Network =C2=A0 =C2=A0 =C2=A0 Address =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Ipkts Ierrs =C2=A0 =C2=A0Opkts Oerrs > =C2=A0Coll > em0 =C2=A0 =C2=A01500 <Link#1> =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2= =A0 =C2=A0 5381 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 5188 =C2=A0 =C2=A0 0 > =C2=A0 0 > em1 =C2=A0 =C2=A01500 <Link#2> =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2= =A05987608 =C2=A0 =C2=A0 0 =C2=A04015584 =C2=A0 =C2=A0 0 > =C2=A0 0 > lo0 =C2=A0 16384 <Link#3> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0= 0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 0 > =C2=A0 0 > lo0 =C2=A0 16384 your-net =C2=A0 =C2=A0 =C2=A0localhost =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 =C2= =A0 =C2=A00 =C2=A0 =C2=A0 - > =C2=A0 - > lagg0 =C2=A01500 <Link#4> =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2=A0599= 2849 =C2=A0 =C2=A0 0 =C2=A04016823 =C2=A0 =C2=A0 8 > =C2=A0 0 > lagg0 =C2=A01500 10.10.1.0 =C2=A0 test =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 5985503 =C2=A0 =C2=A0 - =C2=A06475013 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 - > lagg0 =C2=A01500 10.10.1.83/ jail1 =C2=A0 =C2=A0 =C2=A0 19 =C2=A0 =C2=A0 = - =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 - > > > Anyone got any ideas, as far as I can see lacp is negotiating correctly j= ust > no balancing is happening :( > > =C2=A0 Regards > =C2=A0 Steve > What types of traffic streams are you testing this with? if_lagg will use the SRC/DST MACs and IP addresses for IP traffic to decide which member port of the lagg to sent the traffic out to, balancing of incoming traffic should be done by the Cisco on the same principle. Try a different laggproto for the if_lagg interface e.g. roundrobin with channel-group <number> mode on set on the switchports. cheers, Shteryana
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?61b573981001270543k6514a5g8d4f679777b69b49>