From owner-freebsd-ipfw@FreeBSD.ORG Sat Dec 2 05:43:54 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0EA2A16A403 for ; Sat, 2 Dec 2006 05:43:54 +0000 (UTC) (envelope-from jhalstead@fsisys.com) Received: from hub.fsisys.com (hub.fsisys.com [65.73.42.190]) by mx1.FreeBSD.org (Postfix) with ESMTP id 517C743CA2 for ; Sat, 2 Dec 2006 05:43:35 +0000 (GMT) (envelope-from jhalstead@fsisys.com) Received: from 127.0.0.1 (localhost [127.0.0.1]) by dummy.domain.name (Postfix) with SMTP id EB08236781; Sat, 2 Dec 2006 00:43:52 -0500 (EST) Received: from [127.0.0.1] (gateway.fsisys.com [192.168.1.1]) by hub.fsisys.com (Postfix) with ESMTP id 20FE0366B9; Sat, 2 Dec 2006 00:43:51 -0500 (EST) Message-ID: <45711296.8010709@fsisys.com> Date: Fri, 01 Dec 2006 23:43:50 -0600 From: James Halstead User-Agent: Thunderbird 1.5.0.8 (X11/20061115) MIME-Version: 1.0 To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Mysterious packets with stateful ipfw+nat X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Dec 2006 05:43:55 -0000 Ok, this has been driving me nuts for a while. I recently noticed that my 5.4-RELEASE firewall was having a problem with packet "leakage". I am seeing the occasional packet on the outside interface with an internal src ip. I put a hub between my firewall and cable modem and verified that the packets are indeed on the wire. Now I am in the process of setting up a new 6.1-RELEASE box and the same issue was happening on my test network. So far I don't get it. I must be missing something obvious. At least everything still works in general. The test setup is a clean install of 6.1-RELEASE, using GENERIC with the ipfw.ko and ipdivert.ko modules loaded. After searching around I was basing the configuration off of: http://lists.freebsd.org/mailman/htdig/freebsd-ipfw/2004-June/001182.html The test box has two Ethernet interfaces, renamed to be isp0 and net0. isp0 is using DHCP, and receives the address 10.42.0.220/24. net0 is running a DHCP server, and sits on 192.168.1.1/24. There is one single piece of hardware on net0 which is always assigned 192.168.1.230. The gateway to the actual Internet sits on 10.42.0.254. A pretty simple setup. The internal machine is just constantly connecting to an external web server to generate traffic. I see the same basic type of thing happen for other usage as well on my main network (ssh sessions, https/http sessions, etc). When looking at tcpdump I am occasionally seeing (on isp0): 19:35:27.591761 aa:aa:aa:5b:db:99 > bb:bb:bb:1f:33:da, 192.168.1.230.2542 > xx.xx.53.84.80: ., cksum 0xfade (correct), 2295591733:2295591733(0) ack 167570634 win 0 If this packet was truly supposed to be going out on the external interface, it should have gone through NAT and show a src ip of 10.42.0.220. To make it more frustrating, even if I enable ifpfw at layer 2, I am unable to capture these rogue packets. If I watch tcpdump on net0 at the same time, I see the following: 19:35:27.591767 aa:aa:aa:5b:db:98 > cc:cc:cc:10:04:ce, xx.xx.53.84.80 > 192.168.1.230.2542: ., cksum 0xfade (correct), 913:913(0) ack 1256 win 0 The only other thing that I have noticed, is that the packets seem to show up on the external interface at about the same time as the dynamic rules expire. The dynamic rule would look like: 192.168.1.230 2542 <-> xx.xx.53.84 80 Which is pretty much what I would expect. The same setup with a non-stateful ipfw ruleset (using established keyword) doesn't seem to have this problem. Any ideas? configuration follows. **** natd.conf **** unregistered_only yes dynamic yes #deny_incoming yes log_denied yes log_ipfw_denied yes (deny_incoming was set, turned it off to see if it helped but it works the same). ***** ipfw.rules **** # Test stateful firewall + natd script cmd="/sbin/ipfw add" natout="skipto 1000" oif="isp0" iif="net0" inet="192.168.1.0/24" NOROUTE="( 172.16.0.0/12 or 192.168.0.0/16 or \ 0.0.0.0/8 or 169.254.0.0/16 or 192.0.2.0/24 or 224.0.0.0/4 or 240.0.0.0/4 )" #### # Start with a clean ruleset /sbin/ipfw -q -f flush #### # Allow all traffic on the loopback and internal network, to keep this simple. $cmd 2 allow all from any to any via lo0 $cmd 5 allow all from any to any in via $iif $cmd 6 allow all from any to any out xmit $iif # Translate incoming traffic here $cmd 200 divert natd ip from any to any in via $oif $cmd 205 check-state # Outbound # Use stateful inspection to allow any connection from the internal network. $cmd 300 $natout tcp from any to any out via $oif setup keep-state $cmd 305 $natout udp from any to any out via $oif keep-state $cmd 310 $natout icmp from any to any out via $oif keep-state # Inbound # Prevent non-routable networks on the external interface. $cmd 400 deny all from $NOROUTE to any in via $oif # Allow incoming DHCP for external network address assignment. $cmd 450 allow udp from any to any 68 in via $oif keep-state # Allow incoming SSH to this machine $cmd 455 allow tcp from any to me 22 in via $oif setup keep-state # Allow incoming ICMP $cmd 460 allow icmp from any to any icmptypes 0,3,11,12 in via $oif $cmd 999 deny log ip from any to any # NAT rule for outgoing traffic. $cmd 1000 divert natd ip from any to any out via $oif $cmd 1005 allow ip from any to any Thanks for any insight, -James