From owner-freebsd-stable@FreeBSD.ORG Sat Oct 1 16:18:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BDFFC106566C for ; Sat, 1 Oct 2011 16:18:35 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 862588FC13 for ; Sat, 1 Oct 2011 16:18:35 +0000 (UTC) Received: by ywp17 with SMTP id 17so2965614ywp.13 for ; Sat, 01 Oct 2011 09:18:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.236.185.4 with SMTP id t4mr79477892yhm.121.1317485914753; Sat, 01 Oct 2011 09:18:34 -0700 (PDT) Received: by 10.146.86.8 with HTTP; Sat, 1 Oct 2011 09:18:34 -0700 (PDT) In-Reply-To: <4E86D787.9050207@gmail.com> References: <4E71C059.5060404@hi-media.com> <4E84627B.2050609@my.gd> <4E848592.5050300@my.gd> <4E86D787.9050207@gmail.com> Date: Sat, 1 Oct 2011 18:18:34 +0200 Message-ID: From: Damien Fleuriot To: johan Hendriks Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: CARP interfaces and mastership issue X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2011 16:18:35 -0000 On 1 October 2011 11:04, johan Hendriks wrote: > Op 29-09-11 16:49, Damien Fleuriot schreef: >>> >>> >>> Quick follow-up again. >>> >>> This is the code for sys/netinet/ip_carp.c on FreeBSD 8.2, OpenBSD >>> 3.8, OpenBSD 3.9 in function carp_setrun(struct carp_softc *sc, >>> sa_family_t af) >>> >>> >>> >>> FREEBSD 8.2-PRERELEASE with init + preempt =3D> =A0auto MASTER bug >>> Function starts at line 1371. >>> --- >>> =A0 =A0 =A0 =A0 switch (sc->sc_state) { >>> =A0 =A0 =A0 =A0 case INIT: >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (carp_opts[CARPCTL_PREEMPT]&& =A0!ca= rp_suppress_preempt) >>> { >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_ad_locked(sc)= ; >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_arp(sc); >>> #ifdef INET6 >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_na(sc); >>> #endif /* INET6 */ >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 CARP_LOG("%s: INIT -> = =A0MASTER (preempting)\n", >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 SC2IFP(sc)->if_= xname); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_set_state(sc, MAST= ER); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setroute(sc, RTM_A= DD); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } else { >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 CARP_LOG("%s: INIT -> = =A0BACKUP\n", >>> SC2IFP(sc)->if_xname); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_set_state(sc, BACK= UP); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setroute(sc, RTM_D= ELETE); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setrun(sc, 0); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 break; >>> --- >>> >>> OPENBSD 3.8 with init + preempt =3D> =A0auto MASTER bug >>> Function starts at line 1293. >>> --- >>> =A0 =A0 =A0 =A0 case INIT: >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (carp_opts[CARPCTL_PREEMPT]&& =A0!ca= rp_suppress_preempt) >>> { >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_set_state(sc, MAST= ER); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setroute(sc, RTM_A= DD); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_ad(sc); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_arp(sc); >>> #ifdef INET6 >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_send_na(sc); >>> #endif /* INET6 */ >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } else { >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_set_state(sc, BACK= UP); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setroute(sc, RTM_D= ELETE); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setrun(sc, 0); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 break; >>> --- >>> >>> >>> >>> OPENBSD 3.9 with bug fixed >>> Function starts at line 1348. >>> --- >>> =A0 =A0 =A0 =A0 switch (sc->sc_state) { >>> =A0 =A0 =A0 =A0 case INIT: >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_set_state(sc, BACKUP); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setroute(sc, RTM_DELETE); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 carp_setrun(sc, 0); >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 break; >>> --- >>> >>> >>> It looks like the root cause is there. >>> >>> I'll rebuild and test, keep you updated. >> >> >> >> Find below my test results with the OpenBSD39 implementation which >> forces an INIT -> =A0BACKUP transition regardless of preempt. >> >> >> >> >> >> >> # sysctl net.inet.carp.preempt >> net.inet.carp.preempt: 1 >> >> # sysctl net.inet.carp.suppress_preempt >> net.inet.carp.suppress_preempt: 0 >> >> # ifconfig carp17 >> carp17: flags=3D8 =A0metric 0 mtu 1500 >> =A0 =A0 =A0 =A0inet 46.182.[snip] netmask 0xffffffff >> =A0 =A0 =A0 =A0inet 46.182.[snip] netmask 0xffffffff >> =A0 =A0 =A0 =A0inet 46.182.[snip] netmask 0xffffffff >> =A0 =A0 =A0 =A0inet 46.182.[snip] netmask 0xffffffff >> =A0 =A0 =A0 =A0inet 46.182.[snip] netmask 0xffffffff >> =A0 =A0 =A0 =A0carp: INIT vhid 117 advbase 1 advskew 200 >> >> # ifconfig carp17 up; ./check_carp17_status.sh >> count: 0 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 1 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 2 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 3 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 4 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 5 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 6 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 7 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 8 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 9 =A0 =A0 =A0 =A0carp: BACKUP vhid 117 advbase 1 advskew 200 >> count: 10 =A0 =A0 =A0 carp: BACKUP vhid 117 advbase 1 advskew 200 >> >> # dmesg >> carp17: INIT -> =A0BACKUP >> carp17: link state changed to DOWN >> >> >> >> Looks like it works. >> >> I'm afraid I cannot test actual preemption by shutting down a physical >> interface on the MASTER, because they're actually used in production and >> I've got users logged in to their VPN on pf1. >> >> I see no reason this should break anything however, it only forces the >> CARP interface to assume a BACKUP state right after INIT, as it normally >> should, regardless of preemption. >> >> I'm filling a PR + patch. >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org= " > > Do you have the PR number for me and the patch, i have the same problem a= nd > like it to be fixed. > > regards > Johan Hendriks > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D161123 Here you go. Please note I do not have extensive C programming experience. I have simply removed the part that makes the CARP interface immediately transition from INIT to MASTER when preemption is enabled, and replaced it with an INIT->BACKUP transition like in OpenBSD3.9 in which the bug is fixed. I cannot guarantee the absence of side effects, although I couldn't find any in my testing. Also note that if you update your sources the patch will obviously be gone, take care when upgrading. You'll want to rebuild the kernel at the very least, to be on the safe side I also did the world, you never know. Regards