From owner-freebsd-net@FreeBSD.ORG Sun Jan 16 12:41:31 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA17E106566B; Sun, 16 Jan 2011 12:41:31 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 407128FC0A; Sun, 16 Jan 2011 12:41:30 +0000 (UTC) Received: from mx1.psconsult.nl (psc11.adsl.iaf.nl [80.89.238.138]) by mx1.psconsult.nl (8.14.4/8.14.4) with ESMTP id p0GCfN1H074068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 16 Jan 2011 13:41:28 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.4/8.14.4/Submit) id p0GCfM1f074067; Sun, 16 Jan 2011 13:41:22 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Sun, 16 Jan 2011 13:41:22 +0100 From: Paul Schenkeveld To: freebsd-stable@freebsd.org, freebsd-net@freebsd.org Message-ID: <20110116124122.GA63358@psconsult.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) Cc: Subject: Carp seems completely broken on 8.2-RC2 and 8.2-PRERELEASE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 12:41:31 -0000 Hi, Trying to upgrade two Soekris firewalls to 8-STABLE or 8.2-PRERELEASE it appears that carp doesn't work at all. I've set up carp like I've done on many firewall pairs before and they all work correctly. With google, nor in the mailing lists, I could find anything about changes in the way carp get configured but if I missed something I'd be happy to hear that it's my fault. Here's the setup: net5501 test3 10.4.0.4/24 | -------------+------------- | | net4801 net4801 test1 test2 sis4: 10.4.0.2/24 sis4: 10.4.0.3/24 carp4:10.4.0.1/24 carp4:10.4.0.1/24 | | | | | | | | | | | | | | | | sis[0-3] connected to other networks, see explanation below. When I ping from test3 to 10.4.0.1, I see the following traffic using tcpdump: test3 # tcpdump -e -n -i vr3 not vrrp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vr3, link-type EN10MB (Ethernet), capture size 96 bytes 12:09:35.121831 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.4.0.1 tell 10.4.0.4, length 46 12:09:35.122144 00:00:24:c3:49:91 > 00:00:24:c9:30:ff, ethertype ARP (0x0806), length 60: Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 46 12:09:35.122173 00:00:24:c9:30:ff > 00:00:5e:00:01:68, ethertype IPv4 (0x0800), length 98: 10.4.0.4 > 10.4.0.1: ICMP echo request, id 40482, seq 0, length 64 test1 # tcpdump -e -n -i sis4 not vrrp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes 12:09:34.977570 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.4.0.1 tell 10.4.0.4, length 46 12:09:34.977705 00:00:24:c3:49:91 > 00:00:24:c9:30:ff, ethertype ARP (0x0806), length 42: Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 28 test2 # dump -e -n -i sis4 not vrrp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes 12:09:35.090050 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.4.0.1 tell 10.4.0.4, length 46 There is an ARP request which is replied to by the carp master (test). the ping to the carp address does not even appear on the sis4 interface of test1. This is the kernel config for test1 and test2: include GENERIC device carp makeoptions MODULES_OVERRIDE="" The relevant rc.conf bits: on test1 hostname="test1" cloned_interfaces="carp1 carp2 carp3 carp4" ifconfig_sis0="xxx.xxx.xxx.41/26" ifconfig_sis1="10.1.0.2/24" ifconfig_sis2="10.2.0.2/24" ifconfig_sis3="10.3.0.2/24" ifconfig_sis4="10.4.0.2/24" ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew 0" ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew 0" ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew 0" ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew 0" on test2 hostname="test2" cloned_interfaces="carp1 carp2 carp3 carp4" ifconfig_sis0="xxx.xxx.xxx.42/26" ifconfig_sis1="10.1.0.3/24" ifconfig_sis2="10.2.0.3/24" ifconfig_sis3="10.3.0.3/24" ifconfig_sis4="10.4.0.3/24" ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew 100" ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew 100" ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew 100" ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew 100" In /etc/sysctl.conf: net.inet.carp.preempt=1 Ifconfig output: test1 # ifconfig sis4 sis4: flags=8943 metric 0 mtu 1500 options=83808 ether 00:00:24:c3:49:91 inet 10.4.0.2 netmask 0xffffff00 broadcast 10.4.0.255 media: Ethernet autoselect (100baseTX ) status: active test1 # ifconfig carp4 carp4: flags=49 metric 0 mtu 1500 inet 10.4.0.1 netmask 0xffffff00 carp: MASTER vhid 104 advbase 1 advskew 0 test2 # ifconfig sis4 sis4: flags=8943 metric 0 mtu 1500 options=83808 ether 00:00:24:c3:49:7d inet 10.4.0.3 netmask 0xffffff00 broadcast 10.4.0.255 media: Ethernet autoselect (100baseTX ) status: active test2 # ifconfig carp4 carp4: flags=49 metric 0 mtu 1500 inet 10.4.0.1 netmask 0xffffff00 carp: BACKUP vhid 104 advbase 1 advskew 100 There are no packet filters in place, sis1, sis2 and sis3 are wired through cross-cables from test1 to test2, so no traffic there except for carp. The sis4 interfaces and vr3 of test3 are on a dumb switch with no other stuff connected. Setting net.inet.carp.log=7 does not result in any console/dmesg/messages output. I see carp traffic on sis4 which appears normal except that I don't understand the addrs(7): part but that used to be there on 8.0/8.1 firewalls too: 12:26:52.387140 00:00:5e:00:01:68 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 61070, offset 0, flags [DF], proto VRRP (112), length 56) 10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 104, prio 0, authtype none, intvl 1s, length 36, addrs(7): 198.145.25.33,1.75.182.226,80.169.106.108, 170.107.157.42,147.165.174.125,42.254.15.27,182.184.82.166 12:26:53.387903 00:00:5e:00:01:68 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 61479, offset 0, flags [DF], proto VRRP (112), length 56) 10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 104, prio 0, authtype none, intvl 1s, length 36, addrs(7): 101.233.35.135,163.243.214.16,230.125.241.59, 123.57.190.52,104.246.131.251,255.69.201.65,61.158.20.122 Regards, Paul Schenkeveld