Date: Thu, 29 Aug 2019 12:54:41 +0200 From: Patrick Lamaiziere <patfbsd@davenulle.org> To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: problem with carp on 11.3-RELEASE Message-ID: <20190829125441.12197ea8@mr185033.univ-rennes1.fr>
next in thread | raw e-mail | index | archive | help
Hello, I've upgraded our two firewalls from 11.2-RELEASE-p11 to 11.3 release p3 and I'm seeing a problem with carp, the carp slave becomes briefly MASTER and returns to the slave state. This occurs often. the firewalls use PF / PFSYNC / CARP and the configuration is the same as on 11.2. On 11.2 that works very well. there are two firewalls, the first firewall (fucop1) is normally the carp master but at boot time it starts demoted (via sysctl net.inet.carp.demotion=1000) to prevent it to become master (we promote it manually). The second fw (fucop2) is normally the carp slave. rc.conf --- fucop1--- #-------------------------------- # Packet Filter + pfsync #- pf_enable="YES" pf_rules="/etc/pf.conf" pflog_enable="YES" pfsync_enable="YES" pfsync_syncdev="igb3" pfsync_ifconfig="maxupd 254" pfsync_syncpeer="192.168.255.254" # ix2 : ifconfig_ix2="inet XX.XX.XX.251/24 -tso -lro -vlanhwtso description RI_PF_CKP group CARPDEV group IFCKP" ifconfig_ix2_alias0="inet vhid 3 advskew 0 pass abcdef alias XX.XX.XX.254/32" # igb0 : ifconfig_igb0="inet YY.YY.YY.251/24 -tso -lro -vlanhwtso description PUBLIC_UR1 group CARPDEV group IFPUB1UR" ifconfig_igb0_alias0="inet vhid 2 advskew 0 pass abcdef alias YY.YY.YY.254/32" # igb1 : ifconfig_igb1="inet 192.168.20.251/24 -tso -lro -vlanhwtso description RI_UR1_CHU group CARPDEV group IFCHU" ifconfig_igb1_alias0="inet vhid 6 advskew 0 pass abcdef alias 192.168.20.254/32" # igb3 : pfsync ifconfig_igb3="inet 192.168.255.253/30 -tso -lro -vlanhwtso description PF_SYNC" --- fucop2 --- #-------------------------------- # Packet Filter + pfsync #- pf_enable="YES" pf_rules="/etc/pf.conf" pflog_enable="YES" pfsync_enable="YES" pfsync_syncdev="igb3" pfsync_ifconfig="maxupd 254" pfsync_syncpeer="192.168.255.253" # ix2 : ifconfig_ix2="inet XX.XX.XX.252/24 -tso -lro -vlanhwtso description RI_PF_CKP group CARPDEV group IFCKP" ifconfig_ix2_alias0="inet vhid 3 advskew 100 pass abcdef alias XX.XX.XX.254/32" # igb0 : ifconfig_igb0="inet YY.YY.YY.YY/24 -tso -lro -vlanhwtso description PUBLIC_UR1 group CARPDEV group IFPUB1UR" ifconfig_igb0_alias0="inet vhid 2 advskew 100 pass abcdef alias YY.YY.YY.254/32" # igb1 : ifconfig_igb1="inet 192.168.20.252/24 -tso -lro -vlanhwtso description RI_UR1_CHU group CARPDEV group IFCHU" ifconfig_igb1_alias0="inet vhid 6 advskew 100 pass abcdef alias 192.168.20.254/32" # igb3 : pfsync ifconfig_igb3="inet 192.168.255.254/30 -tso -lro -vlanhwtso description PF_SYNC" ------- Log: here fucop2 is the carp MASTER, fucop1 just booted but not promoted. fucop1 becomes briefly master from time to time. ex: Aug 28 15:08:28 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:05:39 fucop1 kernel: carp: demoted by 1000 to 1000 (sysctl) Aug 28 15:05:39 fucop1 kernel: carp: demoted by 240 to 1240 (interface down) Aug 28 15:05:39 fucop1 kernel: carp: demoted by 240 to 1480 (interface down) Aug 28 15:05:39 fucop1 kernel: carp: demoted by 240 to 1720 (interface down) Aug 28 15:05:39 fucop1 kernel: carp: 3@ix2: INIT -> BACKUP (initialization complete) Aug 28 15:05:39 fucop1 kernel: carp: demoted by -240 to 1480 (interface up) Aug 28 15:05:39 fucop1 kernel: carp: 6@igb1: INIT -> BACKUP (initialization complete) Aug 28 15:05:39 fucop1 kernel: carp: demoted by -240 to 1240 (interface up) Aug 28 15:05:39 fucop1 kernel: carp: 2@igb0: INIT -> BACKUP (initialization complete) Aug 28 15:05:39 fucop1 kernel: carp: demoted by -240 to 1000 (interface up) Aug 28 15:05:39 fucop1 kernel: carp: demoted by 0 to 1000 (pfsync bulk start) Aug 28 15:05:42 fucop1 kernel: carp: demoted by 1000 to 2000 (sysctl) Aug 28 15:05:44 fucop1 kernel: carp: demoted by 1000 to 3000 (sysctl) Aug 28 15:08:28 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:08:28 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:08:28 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:08:28 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:08:28 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:08:28 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:12:31 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:12:31 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:12:31 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:12:31 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:12:31 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:12:31 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:14:10 fucop1 kernel: carp: demoted by 0 to 3000 (pfsync bulk done) Aug 28 15:18:40 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:18:40 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:18:40 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:18:42 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:18:42 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:18:42 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:23:46 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:23:46 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:23:46 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:23:47 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:23:47 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:23:47 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:24:51 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:24:51 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:24:51 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:24:51 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:24:51 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:24:51 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:29:58 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) Aug 28 15:29:58 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:29:58 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:29:58 fucop1 kernel: carp: 3@ix2: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:29:58 fucop1 kernel: carp: 6@igb1: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:29:58 fucop1 kernel: carp: 2@igb0: MASTER -> BACKUP (more frequent advertisement received) Aug 28 15:34:01 fucop1 kernel: carp: 6@igb1: BACKUP -> MASTER (master timed out) Aug 28 15:34:01 fucop1 kernel: carp: 2@igb0: BACKUP -> MASTER (master timed out) Aug 28 15:34:01 fucop1 kernel: carp: 3@ix2: BACKUP -> MASTER (master timed out) The log on the other firewall (fucop2) contains nothing related to carp. Any clue will be welcome. A question anyway, is pfsync compatible between 11.2 and 11.3 ? Because when we upgrade we start to upgrade fucop2, keeping fucop1 in production. Then we put fucop2 in production (doing a pfsync bulk) and upgrade fucop1. I'm not sure if it could be a concern, looks like there are a lot of changes in pfsync since 11.2. I've put the firewalls back to 11.2 and they work fine. Thanks, best regards.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190829125441.12197ea8>