Date: Mon, 8 Aug 2005 15:52:55 +0400 From: Sergey Lapin <slapinid@gmail.com> To: freebsd-pf@freebsd.org Subject: Re: Fwd: pf problems Message-ID: <48239d390508080452270c8d10@mail.gmail.com> In-Reply-To: <200508060411.05482.max@love2party.net> References: <48239d390508040958265ce62@mail.gmail.com> <48239d3905080504297b3ebc89@mail.gmail.com> <200508060411.05482.max@love2party.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> What version of FreeBSD are you running?
5.4-RELEASE
> Do you have a SMP/PREEMPTION kernel?
No
> Does setting debug.mpsafenet=3D0 in loader.conf change the situation? Do=
you
> have a chance to attach a remote debugger or can you try to break into th=
e
> debugger from the console?
Will try it, thanks for idea.
Here's more proper description of the problem:
Hi.
While trying to configure pf on our freebsd 5.4 router we have
encountered three bugs, one of which is minor, second is more serious.
And the worst of all, because of the second bug we have to use
workaround and that workaround triggers third bug which is actually
critical (because it hangs device and floods the network). I'll try to
explain these bugs briefly. Please advice where to report each fo
them. We haev no idea if they are PF global or FreeBSD specific. We
are going to repeat all the scenarios on OpenBSD.
=20
Background: we have two ISP connections and it is the source of all
our problems - looks like work in such a mode is not debugged very
well in pf.
=20
Setup:
Let's say ISP1 gives us 1.0.0.0/24 block with their gateway at
1.0.0.1. Our router is at 1.0.0.254.
Similarly, ISP2 gives us 2.0.0.0/24, their gateway is at 2.0.0.1
and our router is at 2.0.0.254.
ISP2 gateway (2.0.0.1) is our default gateway.
We have DMZ where hosts of both ISP network live
(dmz_net1=3D1.0.0.128/25 and dmz_net2=3D2.0.0.128/25).
+---------------+
1.0.0.254 | |
-------------------+ ext_if1 |
| | dmz_net1
| dmz_if +--------
2.0.0.254 | | dmz_net2
-------------------+ ext_if2 |
| |
+---------------+
=20
Note that firewall rules below are oversimplified to make rule listing
compact. In our "real" firewall we differentiate between TCP and
UDP/ICMP traffic to make sure all outgoing TCP connections use
"modulate state" insted of "keep state" etc. Also ruleset below
loosened in terms it uses "any" where our real ruleset explicitly
defines networks. Again, this is only to demostrate problem with
minimum amount of rules.
=20
****************************************************
****************** BUG#1 (minor) *******************
****************************************************
=20
When pf blocks incoming packet with "block return" rule, it does not
return RST or ICMP packet to the interface from which original packet
came from but always use default gateway instead. This way if we have
default gateway set to ISP2's 2.0.0.1 and packet destined to 1.0.0.254
comes from ISP1 interface (ext_if1) and this packet gets blocked with
"block return", the TCP RST packet with source address 1.0.0.254 will
be sent through 2.0.0.1 gateway. Obviously, ISP2 drops packets which
source does not belong to their network so basically "block return"
does not work at all.
=20
****************************************************
****************** Bug#2 (severe) ******************
****************************************************
=20
ruleset:
#########################################################33
=20
# nat outgoing connections on each internet interface
nat on $ext_if1 from { $dmz_net2 } to any -> ($ext_if1)
nat on $ext_if2 from { $dmz_net1 } to any -> ($ext_if2)
=20
# default deny silently
block drop all
=20
# pass in quick any packets destined for the gateway itself
pass in quick on $dmz_if from any to $dmz_if keep state
=20
pass quick on lo0
=20
# Classify traffic from DMZ
# Allow all outgoing connections from DMZ
=20
pass in on $dmz_if inet from $dmz_net1 to any keep state tag DMZ_TO_EXT1
pass in on $dmz_if inet from $dmz_net2 to any keep state tag DMZ_TO_EXT2
=20
# Allow gateway to route between different networks on the DMZ
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to { $dmz_net1,
$dmz_net2 } keep state tag DMZ_TO_DMZ
=20
# Reroute OUT traffic appropriately
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) tagged
DMZ_TO_EXT2 keep state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) tagged
DMZ_TO_EXT1 keep state
=20
# general "pass out" rules for external interfaces
pass out on { $ext_if1, $ext_if2, $dmz_if } from any to any keep state
=20
#########################################################33
=20
Note that route-to rules should re-route all the traffic to proper
interface regardless of default gateway specified.
=20
What happens with such a rules:=20
* When a host from dmz_net1 sends packets to the internet and default
gateway is ISP1 one, everything works fine and no translation is
performed.
* When we change default gateway to the ISP2 very interesting thing
happens: packet from dmz_net1 host leaves ext_if1 BUT it gets
translated like it should leave ext_if2!
=20
I do not understand this one completely.
=20
****************************************************
***************** Bug#3 (critical) *****************
****************************************************
=20
Because route-to for "out" rules did not work well, we decided to add
the redirection for "in" rules so we added
=20
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) tagged
DMZ_TO_EXT1 keep state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) tagged
DMZ_TO_EXT2 keep state
=20
right before before "pass out quick" rules. Everything started working
right and it worked fine for some time but then firewall died (machine
do not respond to keyboard and the only bring it back to live was to
bounce power). After rebooting situation repeated - router was working
well for 2 to 10 minutes and then unexpectedly freeze. We discovered
that problem was caused by broadcast UDP packet coming on dmz_if and
destined to 255.255.255.255 Later we discovered that this situation is
triggered by any packet for which both conditions are true:
1. destination MAC is broadcast
2. destination IP is none of router's directly connected networks
=20
Any such a packet kills the router. Actually, router is not completely
dead - it sends that damn packet over and over at huge speed to the
outer interface.
=20
Even if I'm doing something completely wrong, router should NOT hang that w=
ay.
=20
=20
No idea if these bugs are PF's or they are FreeBSD specific.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48239d390508080452270c8d10>