Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 28 Oct 2008 09:19:15 -0700
From:      Jeremy Chadwick <koitsu@FreeBSD.org>
To:        Niek Dekker <niekdekker@gmail.com>
Cc:        freebsd-pf@freebsd.org
Subject:   Re: Pf: packets on lo0 blocked in spite of pass rule
Message-ID:  <20081028161915.GA53560@icarus.home.lan>
In-Reply-To: <49072B6A.7010305@gmail.com>
References:  <49072B6A.7010305@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Oct 28, 2008 at 04:10:34PM +0100, Niek Dekker wrote:
> Hi,
> 
> I upgraded recently from 6.2 to 7.0 release p5 (i386) and I'm using pf.
> After the upgrade connection problems arised on lo0, for java > mysql
> and apache > tomcat.
> The network interfaces are all in default setup.
> 
> Here is the output of pfctl -sr, cleaned from network numbers.
> 
> scrub in all fragment reassemble
> block drop in log all
> block drop in log quick on fxp0 from <priv_nets> to any
> block drop out log quick on fxp0 from any to <priv_nets>
> block drop in log quick on fxp0 from <banned> to any
> pass in on fxp0 inet proto tcp from any to ext_if port = smtp flags S/SA
> keep state
> pass in on fxp0 inet proto tcp from any to ext_if port = http flags S/SA
> keep state
> pass in on fxp0 inet proto tcp from any to ext_if port = ssh flags S/SA
> keep state
> pass out on fxp0 proto tcp all flags S/SA keep state
> pass out on fxp0 proto udp all keep state
> pass on lo0 proto tcp all flags S/SA keep state
> pass on lo0 proto udp all keep state
> block drop in on ! fxp0 inet from ext_network/25 to any
> block drop in inet from ext_if to any
> 
> Since the upgrade to 7.0, some packets on lo0 are  being blocked
> nevertheless. Apache httpd is connecting to Tomcat ajp on port 8009.
> Some, but not all of these packets are blocked. For example (pflog):
> 
> 627926 rule 0/0(match): block in on lo0: 127.0.0.1.57243 >
> 127.0.0.1.8009: P 0:719(719) ack 1 win 8960 <nop,nop,timestamp 300647202
> 132868137>

I'm betting money this is a rule order problem.  I *highly* recommend
you stop with the "lo0" rules and use "set skip lo0" like you mention
later on.  This is a good idea for performance reasons as well; don't
waste cycles having pf(4) parse packets for lo0, as nothing can talk
to that interface except local stuff anyway.

Also, because you're using FreeBSD 7.x, you do not need "keep state" or
"flags S/SA" on any of your rules.  Only 6.x and below need this, or
explicit situations where you're using a mix of "no state" and other
things.

> In some of these lines, there is mention of "[bad hdr length 0 - too
> short, < 20]" BUT NOT IN ALL.

That's because you're using tcpdump against a pflog interface.  You need
to increase the snaplen from 68 bytes to something larger; try -s 256
and that message will go away.  It's harmless.

> The state table isn't full by far (78).
> There is some 123 'state mismatch' in the output of pfctl -s all.

Probably normal.  Consider upgrading to 7.1-PRERELEASE, which contains a
fix for re-use of sockets in some situations (I can point you to a PR if
you want to read it).  "state mismatch" is also normal depending upon
the circumstances; I wouldn't worry too much about it.  For example, our
production webserver running RELENG_6 with the aforementioned fix:

Status: Enabled for 25 days 04:49:53          Debug: Urgent

Counters
  state-mismatch                     53454            0.0/s

This number was significantly higher prior to the fix being committed.

> I have "set skip on lo0" to prevent the problem, but it seems to me
> there is an issue to address here. I am likely to submit a PR, unless
> someone comes up with a solution.

You *should* be using "set skip on lo0".  You're gaining nothing (in
your setup) by applying firewall rules to loopback.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20081028161915.GA53560>