From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 20:10:00 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A53F1065675 for ; Wed, 5 Mar 2008 20:10:00 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 95D698FC1D for ; Wed, 5 Mar 2008 20:09:59 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-052-136.pools.arcor-ip.net [88.66.52.136]) by mrelayeu.kundenserver.de (node=mrelayeu6) with ESMTP (Nemesis) id 0ML29c-1JWzw62R9X-0001fG; Wed, 05 Mar 2008 21:09:58 +0100 Received: (qmail 99172 invoked by uid 80); 5 Mar 2008 20:09:27 -0000 Received: from 192.168.4.151 (SquirrelMail authenticated user mlaier) by router with HTTP; Wed, 5 Mar 2008 21:09:27 +0100 (CET) Message-ID: <41303.192.168.4.151.1204747767.squirrel@router> In-Reply-To: <200803051139.01547.fjwcash@gmail.com> References: <200803041351.46053.fjwcash@gmail.com> <36735.192.168.4.151.1204669226.squirrel@router> <200803041525.42330.fjwcash@gmail.com> <200803051139.01547.fjwcash@gmail.com> Date: Wed, 5 Mar 2008 21:09:27 +0100 (CET) From: "Max Laier" To: "Freddie Cash" User-Agent: SquirrelMail/1.4.13 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Provags-ID: V01U2FsdGVkX19koy7EkMn3qzkvBTfcs+NYQoaVt9sT8MZoAzS Fps6QpmE8OKulMw0hPPnmBp9d5w+iIkrYCUg62rObNiUCxhqvO OB69P3LuXqClPe23XZKLw== Cc: freebsd-net@freebsd.org Subject: Re: Understanding the interplay of ipfw, vlan, and carp X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2008 20:10:00 -0000 Am Mi, 5.03.2008, 20:39, schrieb Freddie Cash: > On March 4, 2008 03:25 pm Freddie Cash wrote: >> On March 4, 2008 02:20 pm Max Laier wrote: >> > Am Di, 4.03.2008, 22:51, schrieb Freddie Cash: >> > ... >> > >> > > The lack of a "carpdev" option to directly link a carp device to an >> > > interface (similar to "vlandev" for vlan(4)) is what's really >> > > tripping me up. It appears the carp(4) driver looks at all the >> > > interfaces in the box to find one with an IP in the same subnet as >> > > the carp IP and then uses that as the physical device. >> > >> > You could try the attached patch. It adds carpdev support. You'll >> > have to recompile ifconfig to make use of it. >> > >> > This patch has some shortcomings that I wanted to address for a long >> > time now, but never found the time to do so. Mostly that IPv6 over >> > CARP is broken with this patch. Everything else is supposed to work >> > and I'd like to hear if you experience otherwise (success stories >> > welcome, too). This is from back in early January, but should apply >> > to RELENG_7 and HEAD w/o too much trouble. > > Patch applied cleanly to RELENG_7.0. However, there are a few strange > things happening now. > > If there are IPs on the physical devices (em0|em1) things only seem to > work if my ipfw rules allow traffic over em0|em1. If there are no IPs on > em0|em1, then the ipfw rules work fine using carp0|carp1. But it's not > consistent. Sometimes the counters for the em rules increment and > sometimes the counters for the carp rules increment. I'll look into this ... it would help if you could qualify "it's not consistent" a bit, so that I can reproduce. > If there are no IPs on the physical devices, and I configure rc.conf to > put two IPs onto carp0 (one with /24, one with /32) it loses the route > for the /24, can't find the default router, and traffic doesn't go > through. Manually adding the route via "route add -net > 192.168.0.0/24 -iface carp0" allows traffic to flow again. I see where the error is and will try to fix it. > The rc.conf entries are: > cloned_interfaces="carp0 carp2" > ifconfig_em0="up" > ifconfig_em2="up" > ifconfig_carp0="carpdev em0 vhid 100 pass whatever 192.168.0.11/24" > ifconfig_carp0_alias0="192.168.0.10/32" > ifconfig_carp2="carpdev em2 vhid 102 pass whatever2 172.20.0/1/24" > > I only upgraded one of my test boxes to RELENG_7_0. The other is still > RELENG_6_3. They no longer stay in sync. Even though > net.inet.carp.preempt=1 is set on both boxes, only the interface that I > pull the plug on or manually down will fail-over to the other box. > > The ifconfig ouput on the 6.3 box will show (unplug em2 on the 6.3 box): > carp0: flags=49 mtu 1500 > inet 192.168.0.11 netmask 0xffffff00 > inet 192.168.0.10 netmask 0xffffffff > carp: MASTER vhid 100 advbase 1 advskew 150 > carp2: flags=49 mtu 1500 > inet 172.20.0.1 netmask 0xffffff00 > carp: BACKUP vhid 102 advbase 1 advskew 150 > > And the ifconfig output on the 7.0 box will show: > carp0: flags=8843 metric 0 mtu > 1500 > ether 00:00:5e:00:01:64 > inet 192.168.0.10 netmask 0xffffffff > inet 192.168.0.11 netmask 0xffffff00 > carp: MASTER carpdev em0 vhid 100 advbase 1 advskew 0 > carp2: flags=8843 metric 0 mtu > 1500 > ether 00:00:5e:00:01:66 > inet 172.20.0.1 netmask 0xffffff00 > carp: MASTER carpdev em2 vhid 102 advbase 1 advskew 0 What does "netstat -ssp carp" say? It seems that vhid 100 doesn't sync at all. Might be a problem with the order of the address list. > And, finally, if I try to create two carp devices using the same physical > device, with IPs in the same subnet, the box crashes. The first time, it > locked up with the kernel panic. Every other time it just locks the box. > > The commands to do this are reproducable: > ifconfig em0 up > ifconfig carp0 create > ifconfig carp0 carpdev em0 vhid 1 192.168.0.1/24 > ifconfig carp1 create > ifconfig carp1 carpdev em0 vhid 2 192.168.0.2/24 > > It will complain once that it can't assign the requested address. If you > try the ifconfig command again, the box locks up. Might take two or > three tries if you're lucky. :) This is bad - I'll look at it. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News