Date: Wed, 16 Jan 2019 15:09:18 +0000 From: Pete French <petefrench@ingresso.co.uk> To: freebsd-stable@freebsd.org, killing@multiplay.co.uk, petefrench@ingresso.co.uk, thomas@gibfest.dk Subject: Re: CARP stopped working after upgrade from 11 to 12 Message-ID: <E1gjmog-000DnJ-QD@dilbert.ingresso.co.uk> In-Reply-To: <a130ba8f-9c30-212d-8ca3-c46047cd3ecb@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
> I can't see how any of those would impact carp unless pf is now
> incorrectly blocking carp packets, which seems unlikely from that commit.
Just looking at the code it does seem unlikely, true - but my working
system does not run pf+pfsync and the non working one does, so it is
suspiciously in the right "place". If Thomas can bisect it and show it works
before but nto after then it has to be in there somewhere I guess.
The dmesg "(preempting a slower master)" also makes me think
that it is reciving carp packets - though I havent checked the code to
see if it produces that if it cant see any other masters at all.
> Questions:
>
> * Are you running a firewall?
Yes, pf. The boxes are basically our external firewall/router. I also
run a laod balancer on them - relayd before, but now haproxy after
yesterdays thread on here.
> * What does sysctl net.inet.carp report?
$ sysctl net.inet.carp
net.inet.carp.ifdown_demotion_factor: 240
net.inet.carp.senderr_demotion_factor: 240
net.inet.carp.demotion: -240
net.inet.carp.log: 1
net.inet.carp.preempt: 1
net.inet.carp.dscp: 56
net.inet.carp.allow: 1
> * What exactly does ifconfig report about your carp on both hosts?
I only have carp enabled on one host for now, to pervent the downtime,
but ifconfig on the master is below. I am currently running with a separate
vhid for each address. I normally run with a separate vhid for each network
and address family though - i.e. 4 - but theres no difference in the
behaviour
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether 00:25:90:31:bf:a2
inet 10.32.10.1 netmask 0xffff0000 broadcast 10.32.255.255
inet 10.32.10.6 netmask 0xffff0000 broadcast 10.32.255.255 vhid 1
inet6 fe80::225:90ff:fe31:bfa2%em0 prefixlen 64 scopeid 0x1
inet6 2a02:1658:1:2:e550::1 prefixlen 64
inet6 2a02:1658:1:2:e550::6 prefixlen 64 vhid 2
carp: MASTER vhid 1 advbase 1 advskew 10
carp: MASTER vhid 2 advbase 1 advskew 10
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether 00:25:90:31:bf:a3
inet 178.250.73.196 netmask 0xffffffc0 broadcast 178.250.73.255
inet 178.250.73.198 netmask 0xffffffc0 broadcast 178.250.73.255 vhid 3
inet 178.250.73.199 netmask 0xffffffc0 broadcast 178.250.73.255 vhid 5
inet 178.250.73.200 netmask 0xffffffc0 broadcast 178.250.73.255 vhid 6
inet 178.250.73.221 netmask 0xffffffc0 broadcast 178.250.73.255 vhid 7
inet6 fe80::225:90ff:fe31:bfa3%em1 prefixlen 64 scopeid 0x2
inet6 2a02:1658:1:1::1:2 prefixlen 64
inet6 2a02:1658:1:1::1:1 prefixlen 64 vhid 4
carp: MASTER vhid 3 advbase 1 advskew 10
carp: MASTER vhid 5 advbase 1 advskew 10
carp: MASTER vhid 6 advbase 1 advskew 10
carp: MASTER vhid 7 advbase 1 advskew 10
carp: MASTER vhid 4 advbase 1 advskew 10
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1500
pfsync: syncdev: em0 syncpeer: 10.32.10.2 maxupd: 128 defer: off
groups: pfsync
> * Have you tried enabling more detailed carp logging using sysctl
> net.inet.carp.log?
I didnt have tme unfortuntely - at the point where all the alerts went off
and all of the systems were offline then I just did what I needed to in
order to get it working again (i.e. shut down the passive side). This
is our main production firewall pair, so any downtime cause lots of problems
and we cant make any sales.
Is there anythng in the above which looks fishy to you though ?
-pete.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1gjmog-000DnJ-QD>