From owner-freebsd-pf@FreeBSD.ORG Thu Sep 15 09:26:20 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FF28106564A for ; Thu, 15 Sep 2011 09:26:20 +0000 (UTC) (envelope-from dfleuriot@hi-media.com) Received: from smtp1.hi-media-techno.com (smtp1.hi-media-techno.com [195.158.241.85]) by mx1.freebsd.org (Postfix) with ESMTP id DD38D8FC12 for ; Thu, 15 Sep 2011 09:26:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.hi-media-techno.com (Postfix) with ESMTP id DC8D893BFC for ; Thu, 15 Sep 2011 11:07:38 +0200 (CEST) Received: from smtp1.hi-media-techno.com ([127.0.0.1]) by localhost (smtp1.backbone.hi-media-techno.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UZq6gtCwF2kK for ; Thu, 15 Sep 2011 11:07:38 +0200 (CEST) Received: from mail1.hi-media-techno.com (unknown [192.168.27.17]) by smtp1.hi-media-techno.com (Postfix) with ESMTP id 1600193C0F for ; Thu, 15 Sep 2011 11:07:38 +0200 (CEST) Received: from dfleuriot-at-hi-media.com (unknown [83.167.62.196]) by mail1.hi-media-techno.com (Postfix) with ESMTPSA id 90CBD805469; Thu, 15 Sep 2011 11:07:37 +0200 (CEST) Message-ID: <4E71C059.5060404@hi-media.com> Date: Thu, 15 Sep 2011 11:07:37 +0200 From: Damien FLEURIOT User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: "freebsd-pf@freebsd.org" X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------080808040305070702060507" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: CARP interfaces and mastership issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Sep 2011 09:26:20 -0000 This is a multi-part message in MIME format. --------------080808040305070702060507 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hello list, TLDR: carp interface becomes MASTER for a split second after being created, even if another MASTER exists on the network with faster advertisements. Breaks connections. HOWTO prevent ? We've been experiencing this double mastership problem with CARP interfaces. Allow me to put some context here: 2 firewalls, PF1, PF2, each with 2 VLANs (for example, some have more) on a lagg device (link aggregation). These firewalls then share virtual IPs through CARP interfaces, let us assume the following: PF1: - vlan13 - vlan410 - carp13 (advskew 50) - carp410 (advskew 50) PF2: - vlan13 - vlan410 - carp13 (advskew 100) - carp410 (advskew 100) CARP preemption is turned on, so that if vlan13 should fail on PF1, PF2 would assume mastership on both CARP interfaces. Syscontrols below: net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 1 net.inet.carp.arpbalance: 0 net.inet.carp.suppress_preempt: 0 The problem we have is, say for example we reboot PF2. When it comes back up, it will, even for a split second, assume CARP mastership for its interfaces, at the same time as PF1. This breaks existing sessions, openvpn tunnels and new client connections. While I acknowledge the home-made demons should be built to support tiny network outages, this doesn't solve our main problem. We have the same issue when destroying/creating said CARP interfaces. Recently we upgraded some switches' IOS version on our backup datacenter (which also has 2 PF boxes, sharing the CARP IPs with the 2 PFs on our production DC). To prevent anything nasty happening, we forbade production VLANs on the switches' uplink ports and only allowed management traffic to allow us to perform the upgrade. Things went smoothly but when we brought the production VLANs up again at layer 2 on the switches, when spanning-tree converged we had again a double MASTER problem. I understand I could have avoided it by destroying/recreating the CARP interfaces, but even in this case there is a split second during which both firewalls are CARP MASTER. Is there any way to force CARP to assume INIT state for some time when coming up, and only after X seconds either become MASTER or BACKUP ? Any other idea how to solve this, guys ? --------------080808040305070702060507--