From owner-freebsd-ipfw@freebsd.org Wed Mar 9 17:33:07 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2539FAC9B0B for ; Wed, 9 Mar 2016 17:33:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B7A1967 for ; Wed, 9 Mar 2016 17:33:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u29HX05o011028 for ; Wed, 9 Mar 2016 09:33:04 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201603091733.u29HX05o011028@gw.catspoiler.org> Date: Wed, 9 Mar 2016 09:32:59 -0800 (PST) From: Don Lewis Subject: ipwf dummynet vs. kernel NAT and firewall rules To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2016 17:33:07 -0000 I'm trying to add FQ-CoDEL AQM to my FreeBSD 10 firewall box using this patch: , but I'm running into a problem that I think is caused by an interaction between in-kernel NAT and dummynet. I've set up two dummynet pipe/sched/queue instances using example 3.3a from this document with the appropriate bandwidths, but otherwise default tunings to shape both inbound and outbound traffic. My inside network is a /24 and I have an external /29 (ext/29) network that I don't want to rate limit. My outside network interface is re0. I'm using the /etc/rc.firewall "simple" firewall configuration. The problem that I'm having crops up when I actually try to add the firewall rules to select the traffic that I want to rate limit. The first rule in the list is: 100 allow ip from any to any via lo0 The second rule is numbered 200 and is first anti-spoofing rule. If I add *either* of these two rules, then I'm no longer able to communicate between hosts on my internal network and the rest of the world: ipfw 110 add queue 1 ip from not ext/29 to any in recv re0 ipfw 120 add queue 2 ip from any to not ext/29 out xmit re0 It seems like the inbound rule should be early in the rule list so that any inbound traffic that gets dropped by the firewall rules gets counted even if it is dropped by later rules. It also seems like the outbound rule needs to be before any allow rules since an allow rule would skip the remaining rules and would not count that traffic. Unfortunately the ipfw documentation doesn't really describe the interaction between dummynet, NAT, and other firewall rules. Unfortunately this is a live system, so it is difficult to do controlled experiments and look at the ipfw counters to see where things might be going into the weeds ...