Date: Wed, 5 Mar 2008 21:09:27 +0100 (CET) From: "Max Laier" <max@love2party.net> To: "Freddie Cash" <fjwcash@gmail.com> Cc: freebsd-net@freebsd.org Subject: Re: Understanding the interplay of ipfw, vlan, and carp Message-ID: <41303.192.168.4.151.1204747767.squirrel@router> In-Reply-To: <200803051139.01547.fjwcash@gmail.com> References: <200803041351.46053.fjwcash@gmail.com> <36735.192.168.4.151.1204669226.squirrel@router> <200803041525.42330.fjwcash@gmail.com> <200803051139.01547.fjwcash@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Am Mi, 5.03.2008, 20:39, schrieb Freddie Cash: > On March 4, 2008 03:25 pm Freddie Cash wrote: >> On March 4, 2008 02:20 pm Max Laier wrote: >> > Am Di, 4.03.2008, 22:51, schrieb Freddie Cash: >> > ... >> > >> > > The lack of a "carpdev" option to directly link a carp device to an >> > > interface (similar to "vlandev" for vlan(4)) is what's really >> > > tripping me up. It appears the carp(4) driver looks at all the >> > > interfaces in the box to find one with an IP in the same subnet as >> > > the carp IP and then uses that as the physical device. >> > >> > You could try the attached patch. It adds carpdev support. You'll >> > have to recompile ifconfig to make use of it. >> > >> > This patch has some shortcomings that I wanted to address for a long >> > time now, but never found the time to do so. Mostly that IPv6 over >> > CARP is broken with this patch. Everything else is supposed to work >> > and I'd like to hear if you experience otherwise (success stories >> > welcome, too). This is from back in early January, but should apply >> > to RELENG_7 and HEAD w/o too much trouble. > > Patch applied cleanly to RELENG_7.0. However, there are a few strange > things happening now. > > If there are IPs on the physical devices (em0|em1) things only seem to > work if my ipfw rules allow traffic over em0|em1. If there are no IPs on > em0|em1, then the ipfw rules work fine using carp0|carp1. But it's not > consistent. Sometimes the counters for the em rules increment and > sometimes the counters for the carp rules increment. I'll look into this ... it would help if you could qualify "it's not consistent" a bit, so that I can reproduce. > If there are no IPs on the physical devices, and I configure rc.conf to > put two IPs onto carp0 (one with /24, one with /32) it loses the route > for the /24, can't find the default router, and traffic doesn't go > through. Manually adding the route via "route add -net > 192.168.0.0/24 -iface carp0" allows traffic to flow again. I see where the error is and will try to fix it. > The rc.conf entries are: > cloned_interfaces="carp0 carp2" > ifconfig_em0="up" > ifconfig_em2="up" > ifconfig_carp0="carpdev em0 vhid 100 pass whatever 192.168.0.11/24" > ifconfig_carp0_alias0="192.168.0.10/32" > ifconfig_carp2="carpdev em2 vhid 102 pass whatever2 172.20.0/1/24" > > I only upgraded one of my test boxes to RELENG_7_0. The other is still > RELENG_6_3. They no longer stay in sync. Even though > net.inet.carp.preempt=1 is set on both boxes, only the interface that I > pull the plug on or manually down will fail-over to the other box. > > The ifconfig ouput on the 6.3 box will show (unplug em2 on the 6.3 box): > carp0: flags=49<UP,LOOPBACK,RUNNING> mtu 1500 > inet 192.168.0.11 netmask 0xffffff00 > inet 192.168.0.10 netmask 0xffffffff > carp: MASTER vhid 100 advbase 1 advskew 150 > carp2: flags=49<UP,LOOPBACK,RUNNING> mtu 1500 > inet 172.20.0.1 netmask 0xffffff00 > carp: BACKUP vhid 102 advbase 1 advskew 150 > > And the ifconfig output on the 7.0 box will show: > carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > ether 00:00:5e:00:01:64 > inet 192.168.0.10 netmask 0xffffffff > inet 192.168.0.11 netmask 0xffffff00 > carp: MASTER carpdev em0 vhid 100 advbase 1 advskew 0 > carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > ether 00:00:5e:00:01:66 > inet 172.20.0.1 netmask 0xffffff00 > carp: MASTER carpdev em2 vhid 102 advbase 1 advskew 0 What does "netstat -ssp carp" say? It seems that vhid 100 doesn't sync at all. Might be a problem with the order of the address list. > And, finally, if I try to create two carp devices using the same physical > device, with IPs in the same subnet, the box crashes. The first time, it > locked up with the kernel panic. Every other time it just locks the box. > > The commands to do this are reproducable: > ifconfig em0 up > ifconfig carp0 create > ifconfig carp0 carpdev em0 vhid 1 192.168.0.1/24 > ifconfig carp1 create > ifconfig carp1 carpdev em0 vhid 2 192.168.0.2/24 > > It will complain once that it can't assign the requested address. If you > try the ifconfig command again, the box locks up. Might take two or > three tries if you're lucky. :) This is bad - I'll look at it. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?41303.192.168.4.151.1204747767.squirrel>