From owner-freebsd-net@FreeBSD.ORG Mon Feb 3 14:02:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96BDFFE0 for ; Mon, 3 Feb 2014 14:02:40 +0000 (UTC) Received: from mail.niessen.ch (btx02.niessen.ch [85.10.192.239]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0C1BF1B8B for ; Mon, 3 Feb 2014 14:02:39 +0000 (UTC) Received: from mail.niessen.ch (mail.niessen.ch [127.0.10.3]) by mail.niessen.ch (Postfix) with ESMTP id DD4DB139351 for ; Mon, 3 Feb 2014 15:02:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=niessen.ch; h=message-id :date:from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; s=dkim-2012; bh=W9o/MNr Nx86wsYv1AN/Pm1UjlwPxBNSPrlB0M/CBOBI=; b=pJ0QbSHHIt3HQ4Oz8QMKOYT plYamMU446l/HI3EzQG5LwfH8Gkz5CDIIhKPpThSSmC7R3lR+SkhXnDUuCMz12r6 yn1Y0po4rVifKAeCEwScNL+v+g11JWxTN+pqw2JfPYcyS6tK27Ucz3IE7Fvne+zH Uu+xfI5xveF5bZNNfs6M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=niessen.ch; h=message-id :date:from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=dkim-2012; b=F lk1DW2WhX35CCZDu4/3DLKDu5bZChyVKK/lJgTFn9dvYZqABHjLk7oM1+HlSIdy7 oeSCz3FuRKukd4lzjm1WC2Op2PVeeErOruSZAQ4/jQhOvmwHoJnFSEOpmsRRl1+d pxNK7qugquAkSb1/GwG0zfVU6M4/B+W4+t+GYbP+q8= Received: from [172.20.10.3] (unknown [178.197.236.128]) by mail.niessen.ch (Postfix) with ESMTPSA id 8BAA1139350 for ; Mon, 3 Feb 2014 15:02:21 +0100 (CET) Message-ID: <52EFA157.9080007@niessen.ch> Date: Mon, 03 Feb 2014 15:01:59 +0100 From: Ben User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in 10.0 References: <52EF50A7.1050205@niessen.ch> <1C608452-6F29-486D-BC0F-CCC7853665C7@yahoo.com> <52EF55FE.8030901@niessen.ch> <1798FE17-5718-4125-8B00-1B00DC44B828@yahoo.com> <52EF5D1E.2000306@niessen.ch> <52EF6194.5060305@niessen.ch> <8585EA2E-116E-45A6-877D-DC8D4460C965@yahoo.com> <52EF6690.3010509@niessen.ch> <202BD17C-E68A-4B27-B7EF-E5D84AA89176@yahoo.com> In-Reply-To: <202BD17C-E68A-4B27-B7EF-E5D84AA89176@yahoo.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Feb 2014 14:02:40 -0000 On 03.02.2014 10:58, Scott Long wrote: > Hi, > > If you can, please test the patch I sent and let me know the results. = I=92ll check it into FreeBSD 11 and 10 if it works for you. > > Thanks, > Scott > > On Feb 3, 2014, at 2:51 AM, Ben wrote: > >> Thank you for your detailed explanation. >> >> If I understand correctly the switch is probably not set up correctly,= right? >> >> I will try to have it configured correctly first. >> >> Thanks a lot for your help! >> >> Regards >> Ben >> >> On 03.02.2014 10:45, Scott Long wrote: >>> Ok, please try the patch I emailed earlier. Since you=92re not seein= g any receive messages, it means that your switch isn=92t generating any = LACP heartbeats. The difference between FreeBSD 9.x and 10 is that in 9.= x, it ran in =93optimistic=94 mode, meaning that it didn=92t rely on gett= ing receive messages from the switch, and only took a channel down if the= link state went down. In strict mode, it looks for the receive messages= and only transitions to a full operational state if it gets them. So wh= ile I know it=92s easy to point at the problem being FreeBSD 10, seeing a= s FreeBSD 9 worked for you, please check to make sure that your switch is= set up correctly. >>> >>> I authored the original change that went into FreeBSD 10, and I tried= to make it so that strict_mode=3D0 would keep everything working as it d= id in 9. I guess that since you=92re getting no receive messages from th= e switch at all that we need to disable strict mode on setup, not afterwa= rds. Apply the patch and everything should work as it did in FreeBSD 9. >>> >>> Scott >>> >>> On Feb 3, 2014, at 2:29 AM, Ben wrote: >>> >>>> Yes, via sysctl and /etc/sysctl.conf >>>> >>>> I waited now roughly 20 minutes without touching it but no differenc= e. >>>> >>>> No, I only see these transmit messages, no receive. >>>> >>>> Thanks >>>> Ben >>>> >>>> On 03.02.2014 10:25, Scott Long wrote: >>>>> Did you set it to 0 via the sysctl? You might need to wait for sev= eral minutes if you set it after setting up the links. >>>>> >>>>> Also, the message that you=92re seeing is from your machine transmi= tting PDU packets. Are you seeing any "lacpdu receive=94 messages on the= console? >>>>> >>>>> Thanks, >>>>> Scott >>>>> >>>>> On Feb 3, 2014, at 2:10 AM, Ben wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I set strict mode to 0 but no use. I do receive PDU messages. >>>>>> >>>>>> igb0: lacpdu transmit >>>>>> actor=3D(...) >>>>>> actor.state=3D4d >>>>>> partner=3D(...) >>>>>> partner.state=3D0 >>>>>> maxdelay=3D0 >>>>>> >>>>>> Thanks >>>>>> Ben >>>>>> >>>>>> On 03.02.2014 10:03, Scott Long wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Unfortunately, you can=92t control the strict mode globally. My = apologies for this mess, I=92ll make sure that it=92s fixed for FreeBSD 1= 0.1. If the sysctl doesn=92t help then maybe consider compiling a custom = kernel with it defaulted to 0. You=92ll need to open /sys/net/ieee802ad_= lacp.c and look for the function lacp_attach(). You=92ll see the strict_= mode assign underneath that. I=92ll also send you a patch in a few minut= es. Until then, try enabling net.link.lagg.lacp.debug=3D1 and see if you= =92re receiving heartbeat PDU=92s from your switch. >>>>>>> >>>>>>> Scott >>>>>>> >>>>>>> On Feb 3, 2014, at 1:40 AM, Ben wrote: >>>>>>> >>>>>>>> Hi Scott, >>>>>>>> >>>>>>>> I had tried to set it in /etc/sysctl.conf but seems it didnt wor= k. But will I try again and report back. >>>>>>>> >>>>>>>> The settings of the switch have not been changed and are set to = LACP. It worked before so I guess the switch should not be the problem. M= aybe some incompatibility between FreeBSD + igb-driver + switch (Juniper = EX3300-48T). >>>>>>>> >>>>>>>> I will update you after setting the sysctl setting. It seems to = be "dynamic", I guess 0 reflects the index of LACP lagg devices. Can I sw= itch off the strict mode globally in /etc/sysctl.conf? >>>>>>>> >>>>>>>> Thanks for your help. >>>>>>>> >>>>>>>> Regards >>>>>>>> Ben >>>>>>>> >>>>>>>> On 03.02.2014 09:31, Scott Long wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> You=92re probably running into the consequences of r253687. Ch= eck to see the value of =91sysctl net.link.lagg.0.lacp.lacp_strict_mode=92= . If it=92s =911=92 then set it to 0. My original intention was for this= to default to 0, but apparently that didn=92t happen. However, the fact= that strict mode doesn=92t seem to work at all for you might hint that y= our switch either isn=92t configured correctly for LACP, or doesn=92t act= ually support LACP at all. You might want to investigate that. >>>>>>>>> >>>>>>>>> Scott >>>>>>>>> >>>>>>>>> On Feb 3, 2014, at 1:17 AM, Ben wrote= : >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I upgraded from FreeBSD 9.2-RELEASE to 10.0-RELEASE. FreeBSD 9= .2 was configured to use LACP with two igb devices. >>>>>>>>>> >>>>>>>>>> Now it stopped working after the upgrade. >>>>>>>>>> >>>>>>>>>> This is a screenshot of ifconfig -a after the upgrade to FreeB= SD 10..0-RELEASE: http://tinypic.com/view.php?pic=3D28jvgpw&s=3D5#.Uu9PXT= 1dVPM >>>>>>>>>> >>>>>>>>>> A PR is currently open: http://www.freebsd.org/cgi/query-pr.cg= i?pr=3Dkern/185967 >>>>>>>>>> >>>>>>>>>> It is set to low, but I would like somebody to have a look int= o it as it obviously has a great influence on our infrastructure. The onl= y way to "solve" it is currently switching back to FreeBSD 9.2. >>>>>>>>>> >>>>>>>>>> The suggested fix "use failover" seems not to work. >>>>>>>>>> >>>>>>>>>> Thank you for your help. >>>>>>>>>> >>>>>>>>>> Best regards >>>>>>>>>> Ben >>>>>>>>>> _______________________________________________ >>>>>>>>>> freebsd-net@freebsd.org mailing list >>>>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@free= bsd..org" >>>>>>>>> _______________________________________________ >>>>>>>>> freebsd-net@freebsd.org mailing list >>>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freeb= sd.org" >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> freebsd-net@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebs= d.org" >>>>>>> _______________________________________________ >>>>>>> freebsd-net@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd= .org" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.= org" >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>> >>> >>> >>> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > !DSPAM:1,52ef691c888821141715696! > > Hi, I tried your patch and it works. The strict mode is now set to 0. One thing I saw: There is a message at the login prompt: igb0: Interface stopped=20 DISTRIBUTING, possible flapping igb0 and igb1 are used for the lagg device. I still get the following messages when I restart netif: can't re-use a leaf (lacp_strict_mode)! can't re-use a leaf (rx_test)! can't re-use a leaf (tx_test)! sysctl says strcit_mode is off at that time. I hope this helps. Best regards Ben