From owner-freebsd-net@FreeBSD.ORG Wed Jan 11 13:16:54 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BF6316A41F for ; Wed, 11 Jan 2006 13:16:54 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (cell.sick.ru [217.72.144.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6D0643D46 for ; Wed, 11 Jan 2006 13:16:53 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (glebius@localhost [127.0.0.1]) by cell.sick.ru (8.13.3/8.13.3) with ESMTP id k0BDGoW4060819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 11 Jan 2006 16:16:51 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.sick.ru (8.13.3/8.13.1/Submit) id k0BDGora060814; Wed, 11 Jan 2006 16:16:50 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.sick.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 11 Jan 2006 16:16:50 +0300 From: Gleb Smirnoff To: Sebastian Schwerdhoefer Message-ID: <20060111131650.GC57606@FreeBSD.org> Mail-Followup-To: Gleb Smirnoff , Sebastian Schwerdhoefer , freebsd-net@freebsd.org References: <20060111125354.GE9151@localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20060111125354.GE9151@localdomain> User-Agent: Mutt/1.5.6i Cc: freebsd-net@FreeBSD.org Subject: Re: Router with CARP: reproducible deadlock X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2006 13:16:54 -0000 On Wed, Jan 11, 2006 at 01:53:54PM +0100, Sebastian Schwerdhoefer wrote: S> My dream is to build a redundant router using carp. I build S> test environments, one with FreeBSD 6.0, another one using S> pfsense and I also tried OpenBSD 3.8. However in every S> environment I'm getting into a deadlock situation with the S> following steps: S> S> 1st: Set up "router1" with 2 NICs connection the networks S> "net1" and "net2". This machine is cloned to "router2", S> where we adjust the IP addresses on both physical S> Interfaces. Set up carp on both machines (carp0 is S> 192.168.0.240 and connected to net1, carp1 is 172.16.16.240 S> and connected to net2). Of course, execute 'sysctl -w S> net.inet.carp.allow=1; sysctl -w net.inet.carp.preempt=1' S> and connect "router2" in parallel to "router1". S> S> 2nd: Start up carp on both machines, which leads to router1 S> beeing MASTER with both carp interfaces and Router2 beeing S> BACKUP with both. Perfect for now. S> S> 3rd: Unplug router1's patch cable to net1. As router1 does S> not see any more advertisments from router2 at net1, S> router1's carp0 stays in MASTER mode. S> S> 4th: router2 doesn't see any more advertisments from router1 S> at net1 as well ('cause the cable is disconnected!). So S> router2's carp0 becomes MASTER. Even though S> net.inet.carp.preempt is set, router2's carp1 stays in S> BACKUP mode. S> S> 5th: Now we have the catastrophic situation: S> S> router1 S> carp0 (net1) is in MASTER mode (cable unplugged!) S> carp1 (net2) is in MASTER mode S> S> router2 S> carp0 (net1) is in MASTER mode S> carp1 (net2) is in BACKUP mode S> S> As you can see, packets that should be routed will be lost. S> S> I'm asking myself, if anybody uses carp based routers in S> production environment? If this is the normal behaviour, you S> should not! But I hope, that I misunderstood the carp S> documentation and someone will enlighten me :-) S> S> Any suggestions? This means that your NIC doesn't detect loss of Ethernet link. What NICs do you use? Please demask "net1" and "net2". -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE