From owner-freebsd-pf@FreeBSD.ORG Fri Dec 16 18:34:55 2005 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6409A16A43D for ; Fri, 16 Dec 2005 18:34:55 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (insomnia.benzedrine.cx [62.65.145.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6AF6F43D64 for ; Fri, 16 Dec 2005 18:34:52 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (dhartmei@localhost [127.0.0.1]) by insomnia.benzedrine.cx (8.13.4/8.12.11) with ESMTP id jBGIYmIY006327 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Fri, 16 Dec 2005 19:34:48 +0100 (MET) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.13.4/8.12.10/Submit) id jBGIYm8E015224; Fri, 16 Dec 2005 19:34:48 +0100 (MET) Date: Fri, 16 Dec 2005 19:34:47 +0100 From: Daniel Hartmeier To: Paul Dokas Message-ID: <20051216183447.GA14269@insomnia.benzedrine.cx> References: <20051216100915.73fef758.dokas@oitsec.umn.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20051216100915.73fef758.dokas@oitsec.umn.edu> User-Agent: Mutt/1.5.10i Cc: frantzen@openbsd.org, freebsd-pf@freebsd.org Subject: Re: very odd PF + FreeBSD6.0 problems X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2005 18:34:55 -0000 On Fri, Dec 16, 2005 at 10:09:15AM -0600, Paul Dokas wrote: > I recently upgrade to FreeBSD 6.0 via a full reinstall and I've run into a very > strange problem with PF. First of all, I'm using the same PF ruleset that I > used on 5.4. I know for a fact that it works correctly there. This is related to TCP timestamp checks which were introduced in OpenBSD on May 5 2004 [1]. They were not part of FreeBSD 5.4 [2], because that branched from OpenBSD 3.6 (before the change), but got merged later, making it into FreeBSD 6.0. The additional checks are automatically enabled when using "reassemble tcp", which explains why the same ruleset didn't block the packets on 5.4 but now does on 6.0. You can disable "reassemble tcp" and the new (and old) TCP checks won't run. See the updated pf.conf(5) man page for a full list of checks that this feature enables/disables. The question now is whether your traffic is valid (and the timestamp checks are wrong), or whether the traffic is invalid and the checks rightfully block the packets. Let's see. 09:12:00.516180 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh: S 2843439405:2843439405(0) win 65535 09:12:00.516597 IP that.host.umn.edu.ssh > this.host.umn.edu.54746: S 1786857104:1786857104(0) ack 2843439406 win 65535 09:12:00.516664 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh: . ack 1 win 33304 09:12:00.518506 IP that.host.umn.edu.ssh > this.host.umn.edu.54746: P 1:42(41) ack 1 win 33304 09:12:00.618331 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh: . ack 42 win 33304 09:14:00.601413 IP that.host.umn.edu.ssh > this.host.umn.edu.54746: F 42:42(0) ack 1 win 33304 The last packet above is blocked by pf, which gives the following reason: pf_normalize_tcp_stateful: Timestamp failed 1 pf_normalize_tcp_stateful: tsval: 1424952994 tsecr: 336877 +ticks: 165091 idle: 120s 82ms pf_normalize_tcp_stateful: src->tsval: 1424712993 tsecr: 336673 pf_normalize_tcp_stateful: dst->tsval: 336877 tsecr: 1424712993 tsval0: 336672 TCP A.B.C.D:54746 A.B.C.D:54746 W.X.Y.Z:22 [lo=2843439405 high=2843506014 win=33304 modulator=0 wscale=1] [lo=1786857146 high=1786923754 win=33304 modulator=0 wscale=1] 4:4 FA Which means the following condition was violated: SEQ_GT(tsval, src->scrub->pfss_tsval + tsval_from_last) because 1424952994 > 1424712993 + 165091 (== 1424878084) >From the logged values and the source code we can deduce that the last two packets from the SSH server (that.host) to the client (this.host) were seen (by pf, in the kernel) exactly delta_ts.tv_sec == 120 delta_ts.tv_usec == 82719 apart. This approximately matches the difference in the bpf log, too. So, between those two subsequent packets, the server incremented its timestamp by delta_tsval == 1424952994 - 1424712993 == 240001 within the timespan of delta_usec == 120 * 1000000 + 82719 == 2082719 which means it incremented its timestamp with a frequency of about ts_freq == 240001 / 2082719 usec ~= 115 kHz This is much higher than what pf allows, because RFC1323 [3] (section 4.2.2) says the maximum clock rate is 10 kHz. Even with some fudge applied (pf allows 10% clock skew plus 30s for routing delays), the frequency seen is too high. If you have this problem with only specific hosts, try to find out what OS they are using. If you feel that their timestamp implementation is really RFC1323 conformant and pf is at fault, please argue about it with Mike Frantzen , who wrote these checks :) If you don't care who's right, and just want connections to work to these hosts (even if they're at fault), exclude them from the 'scrub reassemble tcp' rule. If you can't pin down the set of peers that might fall in this category, disable 'reassemble tcp' completely. If you would have liked to enable just the check that 'reassemble tcp' enabled in 5.4, but not the new timestamp checks in 6.0, you're out of luck. HTH, Daniel [1] http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf_norm.c (revision 1.85) [2] http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/contrib/pf/net/pf_norm.c (revision 1.10) [3] http://www.faqs.org/rfcs/rfc1323.html