Date: Sat, 2 Sep 2017 11:44:51 +1000 From: Graham Menhennitt <graham@menhennitt.com.au> To: freebsd-ipfw@freebsd.org Subject: IPFW NAT behaviour different on 10-Stable versus 11-Stable Message-ID: <e0f5f6bb-490e-ba36-25dc-c510bcae8c53@menhennitt.com.au>
next in thread | raw e-mail | index | archive | help
I have a problem that seems to be a difference between ipfw/NAT=20
behaviour in 10-Stable versus 11-Stable. I have two servers: one running=20
10-Stable and one running 11-Stable. I'm using the same rule set on both=20
(see below). It works correctly on 10-Stable but not on 11.
The problem is seen on two places: an outgoing SMTP connection on port=20
465, and an incoming to an IMAP server on port 993. In both cases, there=20
are lost packets and retransmissions. See below for a tshark capture of=20
one attempted SMTP session.
Setting sysctl net.inet.ip.fw.one_pass to one or zero makes no=20
difference. Deleting the sshguard rule (table 22) makes no difference.=20
Deleting the nat rule makes everything work for this SMTP session (but=20
breaks the other machines on my network obviously).
I have no doubt that I have misconfigured the firewall, but I don't see=20
what. And why is 11 different to 10? Any help would be much appreciated.
Thanks in advance,
Graham
Tshark:
(XXX is the SMTP server, YYY is my public IP address)
root# tshark -Y tcp.port=3D=3D465 -i igb1
Capturing on 'igb1'
4 0.722919 YYY =E2=86=92 XXX TLSv1.2 180 Client Key Exchange, Chan=
ge=20
Cipher Spec, Encrypted Handshake Message
527 17.822843 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 63024 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D1 Ack=3D1 Win=3D65535 Len=3D126
1335 51.814540 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 63024 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D1 Ack=3D1 Win=3D65535 Len=3D126
1393 85.806537 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 63024 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D1 Ack=3D1 Win=3D65535 Len=3D126
2142 107.799346 XXX =E2=86=92 YYY TCP 60 465 =E2=86=92 63024 [FIN, ACK]=
Seq=3D1 Ack=3D1=20
Win=3D15544 Len=3D0
2143 107.799393 YYY =E2=86=92 XXX TCP 54 63024 =E2=86=92 465 [ACK] Seq=3D=
127 Ack=3D2=20
Win=3D65535 Len=3D0
2144 107.800135 YYY =E2=86=92 XXX TCP 54 63024 =E2=86=92 465 [FIN, ACK]=
Seq=3D127 Ack=3D2=20
Win=3D65535 Len=3D0
2145 107.822047 YYY =E2=86=92 XXX TCP 74 53762 =E2=86=92 465 [SYN] Seq=3D=
0 Win=3D65535=20
Len=3D0 MSS=3D1460 WS=3D64 SACK_PERM=3D1 TSval=3D2591962 TSecr=3D0
2146 107.977234 XXX =E2=86=92 YYY TCP 60 465 =E2=86=92 63024 [RST] Seq=3D=
2 Win=3D0 Len=3D0
2149 108.001214 XXX =E2=86=92 YYY TCP 62 465 =E2=86=92 53762 [SYN, ACK]=
Seq=3D0 Ack=3D1=20
Win=3D14600 Len=3D0 MSS=3D1460 SACK_PERM=3D1
2150 108.001270 YYY =E2=86=92 XXX TCP 54 53762 =E2=86=92 465 [ACK] Seq=3D=
1 Ack=3D1=20
Win=3D65535 Len=3D0
2151 108.009014 YYY =E2=86=92 XXX TLSv1 323 Client Hello
2160 108.187708 XXX =E2=86=92 YYY TCP 60 465 =E2=86=92 53762 [ACK] Seq=3D=
1 Ack=3D270=20
Win=3D15544 Len=3D0
2176 108.687644 XXX =E2=86=92 YYY TLSv1.2 1514 Server Hello
2177 108.687884 XXX =E2=86=92 YYY TCP 1514 465 =E2=86=92 53762 [PSH, AC=
K] Seq=3D1461=20
Ack=3D270 Win=3D15544 Len=3D1460 [TCP segment of a reassembled PDU]
2178 108.687949 YYY =E2=86=92 XXX TCP 54 53762 =E2=86=92 465 [ACK] Seq=3D=
270 Ack=3D2921=20
Win=3D62874 Len=3D0
2179 108.688175 XXX =E2=86=92 YYY TCP 1230 465 =E2=86=92 53762 [PSH, AC=
K] Seq=3D2921=20
Ack=3D270 Win=3D15544 Len=3D1176 [TCP segment of a reassembled PDU]
2180 108.704012 XXX =E2=86=92 YYY TCP 1514 465 =E2=86=92 53762 [ACK] Se=
q=3D4097 Ack=3D270=20
Win=3D15544 Len=3D1460 [TCP segment of a reassembled PDU]
2181 108.704052 YYY =E2=86=92 XXX TCP 54 53762 =E2=86=92 465 [ACK] Seq=3D=
270 Ack=3D5557=20
Win=3D64240 Len=3D0
2182 108.704625 XXX =E2=86=92 YYY TLSv1.2 969 Certificate, Server Key=20
Exchange, Server Hello Done
2183 108.715222 YYY =E2=86=92 XXX TLSv1.2 180 Client Key Exchange, Chan=
ge=20
Cipher Spec, Encrypted Handshake Message
2211 109.133829 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2238 109.443030 XXX =E2=86=92 YYY TCP 969 [TCP Spurious Retransmission]=
465 =E2=86=92=20
53762 [PSH, ACK] Seq=3D5557 Ack=3D270 Win=3D15544 Len=3D915[Reassembly er=
ror,=20
protocol TCP: New fragment overlaps old data (retransmission?)]
2239 109.443099 YYY =E2=86=92 XXX TCP 54 [TCP Dup ACK 2183#1] 53762 =E2=
=86=92 465=20
[ACK] Seq=3D396 Ack=3D6472 Win=3D65535 Len=3D0
2244 109.772021 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2301 110.827331 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2402 112.770796 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2612 116.391551 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2711 119.018591 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2737 123.957850 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2789 133.632511 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
2859 152.776509 YYY =E2=86=92 XXX TCP 180 [TCP Retransmission] 53762 =E2=
=86=92 465=20
[PSH, ACK] Seq=3D270 Ack=3D6472 Win=3D65535 Len=3D126
^C32 packets captured
root#
Rules:
# stop spoofing
add deny all from LAN_NET to any in via OUTSIDE_IF
add deny all from WIFI_NET to any in via OUTSIDE_IF
# allow anything on the LAN
add allow all from any to any via LAN_IF
# and from the VPN
add allow all from any to any via VPN_IF
# allow anything from the wireless network to the outside world (but not=20
to the LAN)
add allow ip from any to not LAN_NET via WIFI_IF
# create a table of addresses to block
#table 1 destroy
#table 1 create type addr
table 1 flush
# add RFC1918 nets
table 1 add 10.0.0.0/8
table 1 add 172.16.0.0/12
table 1 add 192.168.0.0/16
# and draft-manning-dsua-03.txt nets
table 1 add 0.0.0.0/8
table 1 add 169.254.0.0/16
table 1 add 192.0.2.0/24
table 1 add 224.0.0.0/4
table 1 add 240.0.0.0/4
# stop entries in the table coming in on the outside interface
add deny all from table(1) to any in recv OUTSIDE_IF
# similarly for IPv6
#table 2 destroy
#table 2 create type addr
table 2 flush
# Stop unique local unicast address on the outside interface
table 2 add fc00::/7
# Stop site-local on the outside interface
table 2 add fec0::/10
# Disallow "internal" addresses to appear on the wire.
table 2 add ::ffff:0.0.0.0/96
# Disallow packets to malicious IPv4 compatible prefix.
#table 2 add ::224.0.0.0/100 gives error "Use IPv4 instead of v4-compatib=
le"
#table 2 add ::127.0.0.0/104 ditto
table 2 add ::0.0.0.0/104
#table 2 add ::255.0.0.0/104 ditto
#
table 2 add ::0.0.0.0/96
# Disallow packets to malicious 6to4 prefix.
table 2 add 2002:e000::/20
table 2 add 2002:7f00::/24
table 2 add 2002:0000::/24
table 2 add 2002:ff00::/24
#
table 2 add 2002:0a00::/24
table 2 add 2002:ac10::/28
table 2 add 2002:c0a8::/32
#
table 2 add ff05::/16
# block these addresses both incoming and outgoing
add deny all from table(2) to any via IPV6_IF
add deny all from any to table(2) via IPV6_IF
# block sshguard entries
add reset ip from table(22) to me
# allow setup of incoming SSH, IMAPS, and OpenVPN
add allow tcp from any to me ssh setup
add allow tcp from any to me6 ssh setup
add allow tcp from any to me imaps setup
add allow tcp from any to me6 imaps setup
add allow tcp from any to me openvpn setup
add allow tcp from any to me6 openvpn setup
add allow udp from any to me openvpn
# allow IPP, IMAPS, and SMTP from wireless
add allow ip from any to LAN_NET dst-port printer setup via WIFI_IF
add allow ip from any to me dst-port ipp setup via WIFI_IF
add allow ip from any to me dst-port smtp setup via WIFI_IF
add allow ip from any to me dst-port imaps setup via WIFI_IF
# allow some ICMP types but nothing else
add allow icmp from any to any icmptypes 0,3,8,11
add deny icmp from any to any
#add allow ipv6 from any to any
# NAT
# redirect ports to PS4
nat 1 config if OUTSIDE_IF same_ports deny_in redirect_port tcp=20
PS4_ADDR:1935 1935 redirect_port tcp PS4_ADDR:3478 3478 redirect_port=20
tcp PS4_ADDR:3479 3479 redirect_port tcp PS4_ADDR:3480 3480=20
redirect_port udp PS4_ADDR:3478 3478 redirect_port udp PS4_ADDR:3479 3479
add nat 1 ip4 from any to any via OUTSIDE_IF
# and block the above table again outbound
add deny all from table(1) to any out xmit OUTSIDE_IF
# allow TCP through if setup succeeded
add pass tcp from any to any established
# allow IP fragments to pass through
add pass all from any to any frag
# allow TCP ports needed for PS4
add allow tcp from any to PS4_ADDR 1935 in via OUTSIDE_IF setup
add allow tcp from any to PS4_ADDR 3478 in via OUTSIDE_IF setup
add allow tcp from any to PS4_ADDR 3479 in via OUTSIDE_IF setup
add allow tcp from any to PS4_ADDR 3480 in via OUTSIDE_IF setup
add allow udp from any to PS4_ADDR 3478 in via OUTSIDE_IF
add allow udp from any to PS4_ADDR 3479 in via OUTSIDE_IF
# allow DNS & NTP queries out to the world (and their replies back in)
add allow udp from me to any 53 keep-state
add allow udp from me to any 123 keep-state
# but no other UDP in from outside
add deny udp from any to any in via OUTSIDE_IF
# and allow any other UDP
add allow udp from any to any
# reject all setup of incoming connections from the outside
add deny tcp from any to any in via OUTSIDE_IF setup
# reject all setup of incoming connections from the IPV6 tunnel
add deny tcp from any to any in via gif0 setup
# reject all setup of incoming connections from the wireless
add deny tcp from any to any in via WIFI_IF setup
# allow setup of any other TCP connection
add pass tcp from any to any setup
# Everything else is denied by default, unless the=20
IPFIREWALL_DEFAULT_TO_ACCEPT option is set in your kernel config file.=20
But we add this rule anyway to allow logging.
add deny all from any to any
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e0f5f6bb-490e-ba36-25dc-c510bcae8c53>