Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 16 Dec 2005 19:34:47 +0100
From:      Daniel Hartmeier <daniel@benzedrine.cx>
To:        Paul Dokas <dokas@oitsec.umn.edu>
Cc:        frantzen@openbsd.org, freebsd-pf@freebsd.org
Subject:   Re: very odd PF + FreeBSD6.0 problems
Message-ID:  <20051216183447.GA14269@insomnia.benzedrine.cx>
In-Reply-To: <20051216100915.73fef758.dokas@oitsec.umn.edu>
References:  <20051216100915.73fef758.dokas@oitsec.umn.edu>

next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Dec 16, 2005 at 10:09:15AM -0600, Paul Dokas wrote:

> I recently upgrade to FreeBSD 6.0 via a full reinstall and I've run into a very
> strange problem with PF.  First of all, I'm using the same PF ruleset that I
> used on 5.4.  I know for a fact that it works correctly there.

This is related to TCP timestamp checks which were introduced in OpenBSD
on May 5 2004 [1]. They were not part of FreeBSD 5.4 [2], because that
branched from OpenBSD 3.6 (before the change), but got merged later,
making it into FreeBSD 6.0.

The additional checks are automatically enabled when using "reassemble
tcp", which explains why the same ruleset didn't block the packets on
5.4 but now does on 6.0. You can disable "reassemble tcp" and the new
(and old) TCP checks won't run. See the updated pf.conf(5) man page for
a full list of checks that this feature enables/disables.

The question now is whether your traffic is valid (and the timestamp
checks are wrong), or whether the traffic is invalid and the checks
rightfully block the packets. Let's see.

  09:12:00.516180 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh:
    S 2843439405:2843439405(0) win 65535
    <mss 1460,nop,wscale 1,nop,nop,timestamp 2382230823 0,sackOK,eol>
  09:12:00.516597 IP that.host.umn.edu.ssh > this.host.umn.edu.54746:
    S 1786857104:1786857104(0) ack 2843439406 win 65535
    <mss 1460,nop,wscale 1,nop,nop,timestamp 1424712989 2382230823,nop,nop,sackOK>
  09:12:00.516664 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh: 
    .  ack 1 win 33304
    <nop,nop,timestamp 2382230824 1424712989>
  09:12:00.518506 IP that.host.umn.edu.ssh > this.host.umn.edu.54746:
    P 1:42(41) ack 1 win 33304
    <nop,nop,timestamp 1424712993 2382230824>
  09:12:00.618331 IP this.host.umn.edu.54746 > that.host.umn.edu.ssh:
    . ack 42 win 33304
    <nop,nop,timestamp 2382231028 1424712993>
  09:14:00.601413 IP that.host.umn.edu.ssh > this.host.umn.edu.54746:
    F 42:42(0) ack 1 win 33304
    <nop,nop,timestamp 1424952994 2382231028>

The last packet above is blocked by pf, which gives the following reason:

  pf_normalize_tcp_stateful: Timestamp failed 1
  pf_normalize_tcp_stateful:  tsval: 1424952994  tsecr: 336877  +ticks: 165091  idle: 120s 82ms
  pf_normalize_tcp_stateful:  src->tsval: 1424712993  tsecr: 336673
  pf_normalize_tcp_stateful:  dst->tsval: 336877  tsecr: 1424712993  tsval0: 336672
  TCP A.B.C.D:54746 A.B.C.D:54746 W.X.Y.Z:22
  [lo=2843439405 high=2843506014 win=33304 modulator=0 wscale=1]
  [lo=1786857146 high=1786923754 win=33304 modulator=0 wscale=1] 4:4 FA

Which means the following condition was violated:

  SEQ_GT(tsval, src->scrub->pfss_tsval + tsval_from_last)

because

  1424952994 > 1424712993 + 165091 (== 1424878084)

>From the logged values and the source code we can deduce that the last
two packets from the SSH server (that.host) to the client (this.host)
were seen (by pf, in the kernel) exactly

  delta_ts.tv_sec == 120
  delta_ts.tv_usec == 82719

apart. This approximately matches the difference in the bpf log, too.

So, between those two subsequent packets, the server incremented its
timestamp by

  delta_tsval == 1424952994 - 1424712993 == 240001

within the timespan of

  delta_usec == 120 * 1000000 + 82719 == 2082719

which means it incremented its timestamp with a frequency of about

  ts_freq == 240001 / 2082719 usec ~= 115 kHz

This is much higher than what pf allows, because RFC1323 [3] (section
4.2.2) says the maximum clock rate is 10 kHz. Even with some fudge
applied (pf allows 10% clock skew plus 30s for routing delays),
the frequency seen is too high.

If you have this problem with only specific hosts, try to find out what
OS they are using. If you feel that their timestamp implementation is
really RFC1323 conformant and pf is at fault, please argue about it with
Mike Frantzen <frantzen@openbsd.org>, who wrote these checks :)

If you don't care who's right, and just want connections to work to
these hosts (even if they're at fault), exclude them from the 'scrub
reassemble tcp' rule. If you can't pin down the set of peers that might
fall in this category, disable 'reassemble tcp' completely. If you would
have liked to enable just the check that 'reassemble tcp' enabled in
5.4, but not the new timestamp checks in 6.0, you're out of luck.

HTH,
Daniel

[1] http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf_norm.c
    (revision 1.85)
[2] http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/contrib/pf/net/pf_norm.c
    (revision 1.10)
[3] http://www.faqs.org/rfcs/rfc1323.html




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20051216183447.GA14269>