From owner-freebsd-pf@FreeBSD.ORG Fri Jul 7 18:32:49 2006 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CB5616A4DF for ; Fri, 7 Jul 2006 18:32:49 +0000 (UTC) (envelope-from rand@meridian-enviro.com) Received: from newman.meridian-enviro.com (newman.meridian-enviro.com [207.109.235.166]) by mx1.FreeBSD.org (Postfix) with ESMTP id 03E7943D72 for ; Fri, 7 Jul 2006 18:32:29 +0000 (GMT) (envelope-from rand@meridian-enviro.com) X-Envelope-To: Received: from delta.meridian-enviro.com (delta.meridian-enviro.com [10.10.10.43]) by newman.meridian-enviro.com (8.13.1/8.13.1) with ESMTP id k67IWSFc030120 for ; Fri, 7 Jul 2006 13:32:28 -0500 (CDT) (envelope-from rand@meridian-enviro.com) Received: (from rand@localhost) by delta.meridian-enviro.com (8.13.6/8.13.6/Submit) id k67IWSSc004648; Fri, 7 Jul 2006 13:32:28 -0500 (CDT) (envelope-from rand@delta.meridian-enviro.com) To: freebsd-pf@freebsd.org References: <87ejwx1edf.wl%rand@meridian-enviro.com> From: rand@meridian-enviro.com (Douglas K. Rand) Date: 07 Jul 2006 13:32:26 -0500 In-Reply-To: <87ejwx1edf.wl%rand@meridian-enviro.com> Message-ID: <87zmfl466d.fsf@delta.meridian-enviro.com> Lines: 45 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: ClamAV 0.88/1589/Fri Jul 7 09:37:51 2006 on newman.meridian-enviro.com X-Virus-Status: Clean Subject: Re: pfsync & carp problems X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jul 2006 18:32:49 -0000 Doug> I'm testing a new set of firewalls using pfsync and carp to replace an Doug> existing IP Filter firewall and I'm having occasional problems with Doug> TCP sessions failing over. Some more information after I discovered the -x loud option to pfctl. When the master firewall goes down and the already established TCP session hangs, I get these messages on the slave: pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev pf: State failure on: 1 | And after the master comes up, I see these on the master: pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=0:0 dir=in,rev pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=0:0 dir=in,rev pf: State failure on: 1 | The state table on the master includes: self tcp 67.134.74.224:52173 -> 204.152.184.134:80 TIME_WAIT:TIME_WAIT [2943781408 + 65535] wscale 1 [3255565389 + 63712] wscale 0 age 00:08:29, expires in 00:00:48, 0:1 pkts, 0:40 bytes self tcp 204.152.184.134:80 <- 67.134.74.224:52173 TIME_WAIT:TIME_WAIT [3255565389 + 65160] wscale 0 [2943781408 + 65535] wscale 1 age 00:08:30, expires in 00:00:48, 0:1 pkts, 0:40 bytes And the slave has: self tcp 67.134.74.224:52173 -> 204.152.184.134:80 ESTABLISHED:ESTABLISHED [2943781408 + 65535] wscale 1 [3255565389 + 63712] wscale 0 age 00:07:10, expires in 23:56:40, 21109:24835 pkts, 1100808:37201523 bytes self tcp 204.152.184.134:80 <- 67.134.74.224:52173 ESTABLISHED:ESTABLISHED [3255565389 + 65160] wscale 0 [2943781408 + 65535] wscale 1 age 00:07:10, expires in 23:56:40, 21109:24835 pkts, 1100808:37201523 bytes