Date: Thu, 03 Mar 2005 11:48:56 -0600 From: Matthew Grooms <mgrooms@seton.org> To: Daniel Hartmeier <daniel@benzedrine.cx> Cc: freebsd-pf@freebsd.org Subject: Re: Fwd: pf + pfsync + carp testing ... Message-ID: <42274E08.7050404@seton.org> In-Reply-To: <20050303013807.GH25140@insomnia.benzedrine.cx> References: <200502282232.17646.max@love2party.net> <4223931C.9000607@seton.org> <200502282326.41760.max@love2party.net> <4224B078.9020301@seton.org> <20050301185431.GA81982@cell.sick.ru> <4225174C.801@seton.org> <20050302081051.GB87159@cell.sick.ru> <422600A2.2080907@seton.org> <20050302191656.GA93112@cell.sick.ru> <42264A0A.1090301@seton.org> <20050303013807.GH25140@insomnia.benzedrine.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
Daniel,
Please let me know if this is not what you want. I will try to do
what it takes to get you any data that you may require. The stalled
connection is coming from 192.168.254.51 to 192.168.251.100:80. Sorry
for not paring it down but I did not want to cut out something you may
want to see due to ignorance on my part. I will prepare the other output
you requested unless I hear back from you first.
example 1 - fw1 - pfctl -vvss
self tcp 192.168.254.2:22 <- 192.168.254.51:4461
ESTABLISHED:ESTABLISHED
[895578000 + 63960] [3194607704 + 65483]
age 00:02:33, expires in 24:00:00, 511:579 pkts, 49580:61016 bytes,
rule 4
id: 4226ef910000001f creatorid: 5357f190
self tcp 192.168.254.3:22 -> 192.168.254.51:4462
ESTABLISHED:ESTABLISHED
[1673568462 + 63104] [3196457500 + 65535]
age 00:02:42, expires in 23:59:11, 0:0 pkts, 0:0 bytes
id: 4226ef8800000018 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4469 TIME_WAIT:TIME_WAIT
[32533272 + 272] [3248810405 + 65535]
age 00:02:10, expires in 00:01:12, 85852:161987 pkts,
3434080:242936092 bytes
id: 4226ef8800000019 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4470
ESTABLISHED:ESTABLISHED
[2020265516 + 64512] [3277486902 + 65535]
age 00:00:16, expires in 23:59:57, 22968:43193 pkts, 919669:64635019
bytes, rule 4
id: 4226ef9100000022 creatorid: 5357f190
self tcp 192.168.254.51:4469 -> 192.168.251.100:80 TIME_WAIT:TIME_WAIT
[3248810405 + 65535] [32533272 + 272]
age 00:02:10, expires in 00:01:12, 85852:161987 pkts,
3434080:242936092 bytes
id: 4226ef880000001a creatorid: 5357f190
self tcp 192.168.254.51:4470 -> 192.168.251.100:80
ESTABLISHED:ESTABLISHED
[3277486902 + 65535] [2020265516 + 64512]
age 00:00:16, expires in 23:59:57, 22968:43193 pkts, 919669:64635019
bytes, rule 4
id: 4226ef9100000023 creatorid: 5357f190
self tcp 192.168.253.1:62481 <- 64.233.187.104:80 TIME_WAIT:TIME_WAIT
[3223153423 + 8190] [2943726748 + 2]
age 00:00:41, expires in 00:00:49, 1:1 pkts, 40:40 bytes, rule 4
id: 4226ef9100000021 creatorid: 5357f190
example 1 - fw2 - pfctl -vvss
self tcp 192.168.254.2:22 <- 192.168.254.51:4461
ESTABLISHED:ESTABLISHED
[895580236 + 63532] [3194608276 + 65535]
age 00:02:35, expires in 23:59:58, 0:0 pkts, 0:0 bytes
id: 4226ef910000001f creatorid: 5357f190
self tcp 192.168.254.3:22 -> 192.168.254.51:4462
ESTABLISHED:ESTABLISHED
[1673568634 + 64408] [3196457656 + 65535]
age 00:02:44, expires in 24:00:00, 227:206 pkts, 37788:12244 bytes,
rule 4
id: 4226ef8800000018 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4469 TIME_WAIT:TIME_WAIT
[32533272 + 272] [3248810405 + 65535]
age 00:02:13, expires in 00:01:11, 155592:293304 pkts,
6224629:439872827 bytes, rule 4
id: 4226ef8800000019 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4470
ESTABLISHED:ESTABLISHED
[2016193576 + 51372] [3277486902 + 65535]
age 00:00:18, expires in 23:59:54, 1479:2789 pkts, 59160:4183500 bytes
id: 4226ef9100000022 creatorid: 5357f190
self tcp 192.168.254.51:4469 -> 192.168.251.100:80 TIME_WAIT:TIME_WAIT
[3248810405 + 65535] [32533272 + 272]
age 00:02:13, expires in 00:01:11, 155592:293304 pkts,
6224629:439872827 bytes, rule 4
id: 4226ef880000001a creatorid: 5357f190
self tcp 192.168.254.51:4470 -> 192.168.251.100:80
ESTABLISHED:ESTABLISHED
[3277486902 + 65535] [2016193576 + 51372]
age 00:00:18, expires in 23:59:54, 1479:2789 pkts, 59160:4183500 bytes
id: 4226ef9100000023 creatorid: 5357f190
self tcp 192.168.253.1:62481 <- 64.233.187.104:80 TIME_WAIT:TIME_WAIT
[3223153423 + 8190] [2943726748 + 2]
age 00:00:43, expires in 00:00:47, 0:0 pkts, 0:0 bytes
id: 4226ef9100000021 creatorid: 5357f190
example 2 - fw1 - pfctl -vvss
self tcp 192.168.254.2:22 <- 192.168.254.51:4461
ESTABLISHED:ESTABLISHED
[895581492 + 63756] [3194610408 + 65483]
age 00:05:55, expires in 24:00:00, 560:633 pkts, 54244:66668 bytes,
rule 4
id: 4226ef910000001f creatorid: 5357f190
self tcp 192.168.254.3:22 -> 192.168.254.51:4462
ESTABLISHED:ESTABLISHED
[1673570802 + 63856] [3196458072 + 65535]
age 00:06:04, expires in 23:56:41, 0:0 pkts, 0:0 bytes
id: 4226ef8800000018 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4470
FIN_WAIT_2:FIN_WAIT_2
[2629520121 + 64512] [3277486903 + 65535]
age 00:03:38, expires in 00:01:04, 244235:460539 pkts,
9770349:690583463 bytes, rule 4
id: 4226ef9100000022 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4472
ESTABLISHED:ESTABLISHED
[679995100 + 272] [3327911464 + 65535]
age 00:00:16, expires in 23:59:57, 25897:49715 pkts,
1036349:74514782 bytes, rule 4
id: 4226ef9100000027 creatorid: 5357f190
self tcp 192.168.254.51:4470 -> 192.168.251.100:80
FIN_WAIT_2:FIN_WAIT_2
[3277486903 + 65535] [2629520121 + 64512]
age 00:03:38, expires in 00:01:04, 244235:460539 pkts,
9770349:690583463 bytes, rule 4
id: 4226ef9100000023 creatorid: 5357f190
self tcp 192.168.254.51:4472 -> 192.168.251.100:80
ESTABLISHED:ESTABLISHED
[3327911464 + 65535] [679995100 + 272]
age 00:00:16, expires in 23:59:57, 25897:49715 pkts,
1036349:74514782 bytes, rule 4
id: 4226ef9100000028 creatorid: 5357f190
example 2 - fw2 - pfctl -vvss
self tcp 192.168.254.2:22 <- 192.168.254.51:4461
ESTABLISHED:ESTABLISHED
[895583468 + 63448] [3194610928 + 65535]
age 00:05:57, expires in 23:59:59, 0:0 pkts, 0:0 bytes
id: 4226ef910000001f creatorid: 5357f190
self tcp 192.168.254.3:22 -> 192.168.254.51:4462
ESTABLISHED:ESTABLISHED
[1673570974 + 63684] [3196458384 + 65483]
age 00:06:06, expires in 24:00:00, 244:219 pkts, 40808:13492 bytes,
rule 4
id: 4226ef8800000018 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4470
FIN_WAIT_2:FIN_WAIT_2
[2629520121 + 64512] [3277486903 + 65535]
age 00:03:40, expires in 00:01:03, 1479:2789 pkts, 59160:4183500 bytes
id: 4226ef9100000022 creatorid: 5357f190
self tcp 192.168.251.100:80 <- 192.168.254.51:4472
ESTABLISHED:ESTABLISHED
[673471820 + 35312] [3327911464 + 65535]
age 00:00:18, expires in 23:59:54, 2370:4468 pkts, 94800:6702000 bytes
id: 4226ef9100000027 creatorid: 5357f190
self tcp 192.168.254.51:4470 -> 192.168.251.100:80
FIN_WAIT_2:FIN_WAIT_2
[3277486903 + 65535] [2629520121 + 64512]
age 00:03:40, expires in 00:01:03, 1479:2789 pkts, 59160:4183500 bytes
id: 4226ef9100000023 creatorid: 5357f190
self tcp 192.168.254.51:4472 -> 192.168.251.100:80
ESTABLISHED:ESTABLISHED
[3327911464 + 65535] [673471820 + 35312]
age 00:00:18, expires in 23:59:54, 2370:4468 pkts, 94800:6702000 bytes
id: 4226ef9100000028 creatorid: 5357f190
Matthew Grooms
Network Engineer
Seton Healthcare Network
mgrooms@seton.org
(512) 324 9913
Daniel Hartmeier wrote:
> On Wed, Mar 02, 2005 at 05:19:38PM -0600, Matthew Grooms wrote:
>
>
>> On a slightly more depressing note, I don't think that state via
>>pfsync seems to be working right between the two firewalls. Sometimes (
>>maybe every 1 out of 4 tries ) when the interfaces fail over, the
>>traffic flow stops. The reason why I believe it is a state sync issue is
>>that new connections can always be opened even while the previously
>>opened connections are stalled. This doesn't always happen when an
>>interface is going down either. It happens just as often when an
>>interface is coming back up and reclaims a MASTER state. Any ideas?
>
>
> It would help isolate the problem if you can provide the output of pfctl
> -vvss for one such stalling connection on both boxes, for comparison.
>
> The obvious requirement is that the state is actually present on the
> secondary box. If it is present, maybe we spot an inconsistency between
> the two state entries. If they look the same, maybe you can get a
> tcpdump -vvvS for the stalled connection (which matches the state
> entry).
>
> If the state is not present on the secondary, a tcpdump -nvvvei pfsync0
> over the time between when the state was created on the primary and when
> it should have arrived at the secondary would help.
>
> Daniel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42274E08.7050404>