Date: Mon, 21 Jun 2004 23:43:08 -0500 (CDT) From: Gordon Burditt <gordon@hammy.burditt.org> To: freebsd-ipfw@freebsd.org Cc: gordon@burditt.org Subject: stateful firewall vs. closing TCP connections Message-ID: <E1Bcd7k-0003KE-UM@hammy.burditt.org>
next in thread | raw e-mail | index | archive | help
I've been attempting to update my firewall rules. The old one worked OK with ipfw 1, but it didn't use a stateful firewall, and ipfw 2 allows some shortcuts and has more features that I might like to use. I'm using FreeBSD-4.9 with IPFW2. Yes, I've rebuilt everything with IPFW2=TRUE However, I've been running into a problem that the stateful firewall seems to be closing itself too quickly. It may not be causing observable errors, but my logs are full of junk for denied packets from LEGITIMATE connections. The objective here was to look at packets rejected under the new rules but not rejected under the old rules before actually denying them. Then I deny these and see only the port scans, probes, and other bad traffic in the logs. This set of rules actually is intended to run on both my gateway machine and my server machine on the local LAN (with the gateway having some extra rules to allow passthru), but I'm testing this on the server machine only. The server machine accepts a lot of http, smtp, and mysql connections from outside ("outside" here is any other machine, including my LAN), and initiates a number of outgoing smtp (outgoing mail and Exim callout verify) and mysql connections. There is no NAT in this setup (at least not when I first started working on this). All of the machines have public IPs. The LAN is 100baseTX and the outside connection is DSL. The old rules worked like this (in a section applicable to TCP only): 20100 allow tcp from any to any established ... 21000 skipto 60000 tcp from { MYSQL_CLIENT_1, MYSQL_CLIENT_2, MYSQL_CLIENT_3} to me 3306 setup 21010 skipto 60000 tcp from me to { MYSQL_SERVER_1, MYSQL_SERVER_2} 3306 setup ... 21700 skipto 60000 tcp from me to any http setup 21710 skipto 60000 tcp from any to me http setup ... 29999 deny log all from any to any 60000 allow all from any to any Now, this worked OK, but it's not stateful. So, for testing purposes, I changed it to this: 20099 check-state 20100 allow log tcp from any to any established 20110 allow log tcp from any to any not setup ... 21000 skipto 60000 tcp from { MYSQL_CLIENT_1, MYSQL_CLIENT_2, MYSQL_CLIENT_3} to me 3306 setup keep-state 21010 skipto 60000 tcp from me to { MYSQL_SERVER_1, MYSQL_SERVER_2} 3306 setup keep-state ... 21700 skipto 60000 tcp from me to any http setup keep-state 21710 skipto 60000 tcp from any to me http setup keep-state ... 29999 deny log all from any to any 60000 allow all from any to any The rules go through a preprocessor where things like MYSQL_SERVER_* and MYSQL_CLIENT_* get filled in with a real (and outside public) ip. There are a lot more rules to allow tcp setups in the full set of rules, ending in "... setup keep-state". Rule 20100 is not supposed to catch anything, as the established packets should be accepted by rule 20099. Rule 20110 only catches packets that are neither setup nor established (and probably malicious). I intend to change these rules to "deny log" (or just delete them and let rule 29999 handle it) after making sure doing this doesn't disrupt anything. So, what happens? I get the usual bunch of port scans in the log. BUT, I also get a lot of packets logged from LEGITIMATE connections by rule 20100, which I believe are coming at the tail end of the connection. This happens too often for me to believe it's due to retransmitted packets. Also, I can match up stuff in the ipfw logs against mail or Apache logs to see that a real connection came in. I cannot, however, match tcpdump output against log output: there is not enough detail in the logs to determine which packet from that particular server was logged. But it seems like it's the tail end of the connection. I tried setting net.inet.ip.fw.dyn_rst_lifetime and net.inet.ip.fw.dyn_fin_lifetime to 3 from 1. No change. Since 'log' doesn't log much of the packet details, I split rule 20100 into several rules with various combinations of tcpflags so I could use the rule number to figure out what the flags were. What seems to be happening is that the SERVER side (my host acts as both client and server at various times, and the symptoms seem to follow the role it's playing) sends a FIN-ACK or FIN packet and the CLIENT side sends back a RST. If I change rules 20100-20109 to "deny log", the server side sends two FIN-ACK packets and I don't get back the RST, so it seems the server is retrying the packet. I have not observed any actual problems with the transactions from denying the packet. But I'm not sure how many programs actually check for errors on close. Question: if one side of the connection is closed, does the stateful firewall close off both directions? Actually, I don't see why this should be a problem, especially with the lifetime set to 3 seconds, since I don't think a whole http transaction even takes 3 seconds. Is this normal behavior of a TCP connection? Is this a firewall bug? Is there a way I can avoid logging these, but log the real malicious stuff? What's the deal with rule 60000? At one point I was trying to deal with two outside connections, and packets FROM public netblock A had to go out interface de2 (using fwd) and packets FROM public netblock B had to go out tun0. That applies only to the gateway anyway. Gordon L. Burditt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1Bcd7k-0003KE-UM>