Date: Thu, 11 Aug 2005 18:32:01 +0400 From: Sergey Lapin <slapinid@gmail.com> To: FreeBSD-gnats-submit@FreeBSD.org Subject: kern/84801: kernel hangs with pf and route-to Message-ID: <48239d39050811073220124afa@mail.gmail.com> Resent-Message-ID: <200508111440.j7BEeFdd028433@freefall.freebsd.org> In-Reply-To: <200508111429.j7BETf9v001115@int-s60-3.dart.spb> References: <200508111429.j7BETf9v001115@int-s60-3.dart.spb>
next in thread | previous in thread | raw e-mail | index | archive | help
>Number: 84801
>Category: kern
>Synopsis: kernel hangs with pf and route-to
>Confidential: no
>Severity: critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Aug 11 14:40:15 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator: Sergey Lapin
>Release: FreeBSD 6.0-BETA2 i386
>Organization:
>Environment:
System: FreeBSD int-s60-3.dart.spb 6.0-BETA2 FreeBSD 6.0-BETA2 #0: Wed
Aug 3 08:22:24 UTC 2005
root@harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
>Description:
We are have 2 upstream providers. They give us 2 networks.
First, 1.1.1.0/24 is split by us into 2 parts by /25 mask - first part
is outer, and second (129-254) is behind our router (DMZ1)
second is 2.2.2.0/25 given by second ISP, is routed to our router,
and provider gives /30 p-to-p network 2.2.3.172/30 so 174 is our side and
173 is their's. So 2.2.2.0/25 is routed to 2.2.3.174 from their side.
Configuration:
(all addresses fake, 1.1.1.x - from ISP1, 2.2.2 - from ISP2)
# grep ifconfig /etc/rc.conf
ifconfig_xl0=3D"inet 1.1.1.254 netmask 255.255.255.128"
ifconfig_xl0_alias0=3D"inet 2.2.2.2 netmask 255.255.255.128"
ifconfig_xl1=3D"inet 192.168.255.1 netmask 255.255.255.255"
ifconfig_vlan0=3D"inet 1.1.1.3 netmask 255.255.255.0 vlan 1001 vlandev
xl1 mtu 1496"
ifconfig_vlan1=3D"inet 2.2.3.174 netmask 255.255.255.252 vlan 1004
vlandev xl1 mtu 1496"
# grep defaultrouter /etc/rc.conf
defaultrouter=3D"1.1.1.1"
# cat /etc/pf.conf
# $FreeBSD: src/etc/pf.conf,v 1.1.2.1 2004/09/17 18:27:14 mlaier Exp =
$
# $OpenBSD: pf.conf,v 1.21 2003/09/02 20:38:44 david Exp $
#
# See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
# Required order: options, normalization, queueing, translation, filtering.
# Macros and tables may be defined and used anywhere.
# Note that translation rules are first match while filter rules are last m=
atch.
ext_if1 =3D "vlan0"
ext_if2 =3D "vlan1"
dmz_if =3D "xl0"
ext_gw1 =3D "1.1.1.1"
ext_gw2 =3D "2.2.3.173"
lan_net =3D "192.168.0.0/16"
dmz_net1 =3D "1.1.1.128/25"
dmz_net2 =3D "2.2.2.0/25"
table <our_nets> const { $dmz_net1, $dmz_net2, $lan_net }
set block-policy drop
set state-policy floating
# Normalize all incoming streams
scrub in on $ext_if1
scrub in on $ext_if2
###########################################################################=
######
# NAT
#
###########################################################################=
######
# nat outgoing connections on each internet interface
nat on $ext_if1 from { $lan_net $dmz_net2 } to any -> ($ext_if1)
nat on $ext_if2 from { $lan_net $dmz_net1 } to any -> ($ext_if2)
###########################################################################=
######
# Block everything by default
#
###########################################################################=
######
# default deny silently
block drop all
# block IDENT notifying sender to prevent sendmail and the like from
# wasting time waiting for timeout
block return in on { $ext_if1 $ext_if2 } proto { tcp, udp } to port =3D aut=
h
block drop log on xl0 all
###########################################################################=
######
# Traffic to gateway itself
#
###########################################################################=
######
# pass in quick any packets destined for the gateway itself
pass in quick on $dmz_if proto tcp from any to $dmz_if flags S/SA keep stat=
e
pass in quick on $dmz_if inet proto { udp, icmp } from any to $dmz_if keep =
state
# pass multicast and IGMP traffic
pass quick on $dmz_if inet from any to 224.0.0.0/4 allow-opts keep state
pass quick on lo0
###########################################################################=
######
# Classify traffic from DMZ
#
###########################################################################=
######
# pass traffic from DMZ to Internet
pass in on $dmz_if proto udp from $dmz_net1 to any port =3D 53 keep
state tag DMZ_TO_EXT1
pass in on $dmz_if proto udp from $dmz_net2 to any port =3D 53 keep
state tag DMZ_TO_EXT2
# Allow all outgoing connections from DMZ
pass in on $dmz_if inet proto tcp from $dmz_net1 to any flags S/SA
keep state tag DMZ_TO_EXT1
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net1 to any keep
state tag DMZ_TO_EXT1
pass in on $dmz_if inet proto tcp from $dmz_net2 to any flags S/SA
keep state tag DMZ_TO_EXT2
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net2 to any keep
state tag DMZ_TO_EXT2
# Allow gateway to route between different networks on the DMZ
# DMZ nets -> DMZ nets
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 } to {
$dmz_net1, $dmz_net2 } flags S/SA keep state tag DMZ_TO_DMZ
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to { $dmz_net1,
$dmz_net2 } keep state tag DMZ_TO_DMZ
# DMZ nets -> LAN net
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 } to
$lan_net flags S/SA keep state tag DMZ_TO_LAN
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to $lan_net keep
state tag DMZ_TO_LAN
# LAN net -> DMZ nets
pass in on $dmz_if inet proto tcp from $lan_net to { $dmz_net1,
$dmz_net2 } flags S/SA keep state tag LAN_TO_DMZ
pass in on $dmz_if inet from $lan_net to { $dmz_net1, $dmz_net2 } keep
state tag LAN_TO_DMZ
###########################################################################=
######
# Allow classified traffic from DMZ
#
###########################################################################=
######
# Allow incoming packets from DMZ one more time and route them appropriate=
ly
# This must be done to IN packets because if we only do it for OUT
packets, it happens to late -
# packet is routed appropriately, but NAT rule for wrong interface gets fi=
red
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto tcp tagged
DMZ_TO_EXT1 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto { udp,
icmp } tagged DMZ_TO_EXT1 keep state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto tcp tagged
DMZ_TO_EXT2 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto { udp,
icmp } tagged DMZ_TO_EXT2 keep state
# Allow OUT traffic
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto tcp
tagged DMZ_TO_EXT2 flags S/SA modulate state
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto { udp,
icmp } tagged DMZ_TO_EXT2 keep state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto tcp
tagged DMZ_TO_EXT1 flags S/SA modulate state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto { udp,
icmp } tagged DMZ_TO_EXT1 keep state
###########################################################################=
######
# Clasify traffic from Internet to DMZ
#
###########################################################################=
######
# WHISKEY
pass in on vlan0 proto tcp from any to 1.1.1.144/32 port =3D 22 flags
S/SA keep state tag EXT1_TO_DMZ
pass in on vlan1 proto tcp from any to 2.2.2.2/32 port =3D 22 flags S/SA
keep state tag EXT2_TO_DMZ
###########################################################################=
######
# Allow classified traffic from Internet to DMZ
#
###########################################################################=
######
# Pass to DMZ traffic already approved by earlier rules
# and route replies to corresponding interface
# EXT1
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) proto tcp
tagged EXT1_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) tagged
EXT1_TO_DMZ keep state
# EXT2
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) proto tcp
tagged EXT2_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) tagged
EXT2_TO_DMZ keep state
###########################################################################=
######
# Other traffic
#
###########################################################################=
######
# general "pass out" rules for external interfaces
pass out on { $ext_if1, $ext_if2, $dmz_if } proto tcp from any to any
flags S/SA modulate state
pass out on { $ext_if1, $ext_if2, $dmz_if } proto { udp, icmp } from
any to any keep state
# Zebra uses IGMP so let it work on DMZ interface
pass out on $dmz_if proto igmp from any to any allow-opts
Test case:
(done from Linix machine from 1.1.1.128/25)
tcpreplay -e 1.1.1.133:255.255.255.255 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet using tcpdump -peni=
)
or
tcpreplay -e 1.1.1.133:10.2.2.2 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet)
kills machine.
Machine hangs and doesn't react on keyboard, whatever.
Only reset helps.
More than that - machine sends that packet throught interface (that
the same packet) continuously, like in loop, even while we're not
sending it.
I don't need to say that the same ruleset works well on OpenBSD 3.7
(with s/modulate/keep/g)
Any ideas?
Thanks a lot!
>How-To-Repeat:
Just read description, find UDP packet with broadcast mac and
try to send it.
It doesn't reproduce inside VmWare.
>Fix:
Temp. workaround idea is disable broadcast UDP packets, but I don't
think it's too good.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48239d39050811073220124afa>