From owner-freebsd-ipfw@FreeBSD.ORG Wed May 16 12:35:27 2012 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6FC2B1065691 for ; Wed, 16 May 2012 12:35:27 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 036808FC14 for ; Wed, 16 May 2012 12:35:26 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id q4GC711s047625 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 16 May 2012 15:07:05 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4FB39865.50806@digsys.bg> Date: Wed, 16 May 2012 15:07:01 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.4) Gecko/20120501 Thunderbird/10.0.4 MIME-Version: 1.0 To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Subject: IPFW tables trouble X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2012 12:35:27 -0000 Hello, I am having an persistent problem when using tables with ipfw. On a number of routers, built with various FreeBSD versions, with ipfw as loadable module or statically compiled, the problem remains the same. First, the versions: (compiled in) FreeBSD router8 7.1-STABLE FreeBSD 7.1-STABLE #0: Tue Feb 3 11:36:55 EET 2009 root@localhost:/usr/obj/usr/src/sys/ROUTER3 amd64 FreeBSD router6 7.2-STABLE FreeBSD 7.2-STABLE #0: Sat Aug 29 20:39:46 EEST 2009 root@localhost:/usr/obj/usr/src/sys/ROUTER amd64 (module) FreeBSD router7 8.2-STABLE FreeBSD 8.2-STABLE #0: Fri Sep 30 16:17:47 EEST 2011 root@localhost:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD router6x 9.0-STABLE FreeBSD 9.0-STABLE #1: Wed Apr 18 20:19:12 EEST 2012 root@localhost:/usr/obj/usr/src/sys/GENERIC amd64 There were more versions in-between but only those remain at the moment. My usage: I have an script, that runs say hourly to create a list of IP addresses, that should be filtered. The script generates output in the form 193.68.223.206/31 193.68.223.208/30 193.68.223.213/32 193.68.223.214/31 a list of prefix/mask. There should be no overlapping prefix/mask in here. Then, a script loads this into an ipfw table, like this: # flush old table ipfw table 1 flush # load new table cat /tmp/iptable | while read line;do ipfw table 1 add $line 1 done The actual filtering rule is like this: # filter unknown addresses ipfw add deny ip from "table(1)" to any ipfw add deny ip from any to "table(1)" Now, the problem. From time to time, ipfw spews errors like this: Non-unique normal route, mask not entered Non-unique normal route, mask not entered or rn_delete: couldn't find our annotation rn_delete: couldn't find our annotation rn_delete: couldn't find our annotation Sometimes, after such output, if one does: ipfw table 1 flush ipfw table 1 list the output is non-empty. It should be empty, right? On the routers with loadable ipfw module, I have resorted to running periodically an script like this kldunload ipfw kldload ipfw /root/rc.firewall /root/loadiptable Sometimes, when that script runs, the output is IP firewall unloaded Warning: memory type ipfw_tbl leaked memory on destroy (20 allocations, 5120 bytes leaked). ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to accept, logging disabled Apparently, there is a memory leak somewhere, which is clearly detected by the module unload code... but it seems this memory leak hasn't been fixed for a number of years... When a glitch like this happens, on the routers where ipfw is compiled within the kernel, IP addresses that remain "unremovable" from the table, like in the output from ipfw table 1 flush ipfw table 1 list are permanently filtered. Sometimes IP addresses that are not shown this way get filtered silently as well requiring reboot of those routers. Very painful and unfortunately, always manual task. So my question is, has someone seen anything like this? Is there a solution? Should I just abandon ipfw altogether and seek other method to filter these addresses? (suppose, nobody is going to fix it). This problem has troubled me for a number of years already. Thanks in advance, Daniel