From owner-freebsd-ipfw@FreeBSD.ORG Sun Apr 14 17:26:59 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B7401E58 for ; Sun, 14 Apr 2013 17:26:59 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id C855CA1B for ; Sun, 14 Apr 2013 17:26:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r3EHQoqd071149; Mon, 15 Apr 2013 03:26:50 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 15 Apr 2013 03:26:50 +1000 (EST) From: Ian Smith To: Spil Oss Subject: Re: Problems with ipfw/natd and axe(4) In-Reply-To: Message-ID: <20130415015850.Y56386@sola.nimnet.asn.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-ipfw@freebsd.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 17:26:59 -0000 On Sat, 13 Apr 2013 15:34:39 +0200, Spil Oss wrote: > Hi All, > > I can't use ipfw with natd with my ASIX AX88772B USB NIC > > ipfw ruleset (slightly modified /etc/rc.firewall simple ruleset) I see you omitted the 2 anti-spoofing rules for 172.16.0.0/12 either side of the divert rule, as you had to, seeing your outside interface (so, gateway) is on 172.17.2.111(/24?). I didn't see any other changes to 'simple' apart from rules 10 and the equally weird rule 4200, plus your extra tcp services below. Is that right? > 00010 allow ip from any to me dst-port 22 recv ue0 > 00010 allow tcp from me 22 to any xmit ue0 That these rules make the particular client connection from the gateway work seemed pretty confusing, as you properly allow ssh connections in below, and allow all established tcp, however .. 'allow ip' aka 'allow all' doesn't usually take a port number, which applies only to tcp and udp. I'm surprised this rule isn't rejected, unless perhaps the proto is now also tested? but in any case, this rule should allow at least any non-tcp and non-udp IP packet through. And that means all ICMP in from the outside will be accepted, whereas a long-standing bug in rc.firewall 'simple' means that NO icmp, neither in nor out - not even on the inside network - is passed with this ruleset. Try replacing rules 10 with this section from the 'workstation' ruleset (though further down, probably after NAT, would be a better place): # Allow "mandatory" ICMP in. ${fwcmd} add pass icmp from any to any icmptype 3,4,11 if [ $ipv6_available -eq 0 ]; then ${fwcmd} add pass ipv6-icmp from any to any icmp6type 3 fi Of course Luigi is likely right about the offloaded checksumming too, but there's no reason not to have ICMP functioning on a router/NAT box, especially when it's "mandatory" for path MTU discovery. It's possible that there may be an MTU issue with these inbound connections here? Neither do I see any reason not to allow at least yourself and your LAN clients to ping. Block inbound pings from the outside if you prefer but allowing them outbound and ping responses in doesn't seem risky to me. > 00100 allow ip from any to any via lo0 > 00200 deny ip from any to 127.0.0.0/8 > 00300 deny ip from 127.0.0.0/8 to any > 00400 deny ip from any to ::1 > 00500 deny ip from ::1 to any > 00600 allow ipv6-icmp from :: to ff02::/16 > 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 > 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 > 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 > 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 > 01100 deny ip from 10.16.2.1 to any in via ue0 > 01200 deny ip from 172.17.2.111 to any in via re0 So following rc.firewall, one learns that ue0 is your outside interface and re0 the inside. Looking in /etc/defaults/rc.conf you'll see that ${inet} and ${onet} are intended to be network addresses (ip/mask) to protect from spoofed networks. Yours only protect against those single addresses, not the rest of the networks they're in - but it works now. > 01300 deny ip from any to 10.0.0.0/8 via ue0 > 01500 deny ip from any to 192.168.0.0/16 via ue0 > 01600 deny ip from any to 0.0.0.0/8 via ue0 > 01700 deny ip from any to 169.254.0.0/16 via ue0 > 01800 deny ip from any to 192.0.2.0/24 via ue0 > 01900 deny ip from any to 224.0.0.0/4 via ue0 > 02000 deny ip from any to 240.0.0.0/4 via ue0 > 02100 divert 8668 ip4 from any to any via ue0 > 02200 deny ip from 10.0.0.0/8 to any via ue0 > 02400 deny ip from 192.168.0.0/16 to any via ue0 > 02500 deny ip from 0.0.0.0/8 to any via ue0 > 02600 deny ip from 169.254.0.0/16 to any via ue0 > 02700 deny ip from 192.0.2.0/24 to any via ue0 > 02800 deny ip from 224.0.0.0/4 to any via ue0 > 02900 deny ip from 240.0.0.0/4 to any via ue0 Longwinded these may be, and although it's a pity some couldn't convince some others that it was a good idea to put these into a table years ago, they are correct and have been done like that for ~15 years. Another more recent omission in 'simple' is allowing use of kernel ipfw_nat as well as or instead of natd, as you can for 'open' and 'client' rulesets. Yes I did submit patches :( I'm unsure whether ipfw_nat might or might not work better in the case Luigi outlined? > 03000 allow tcp from any to any established > 03100 allow ip from any to any frag > 03200 allow tcp from any to me dst-port 22 setup > 03300 allow tcp from any to me dst-port 25 setup > 03400 allow tcp from any to me dst-port 465 setup > 03500 allow tcp from any to me dst-port 587 setup > 03600 allow tcp from any to me dst-port 80 setup > 03700 allow tcp from any to me dst-port 443 setup Standard, with additional services. > 03800 deny log logamount 5 ip4 from any to any in via ue0 setup proto tcp > 03900 allow tcp from any to any setup > 04000 allow udp from me to any dst-port 53 keep-state > 04100 allow udp from me to any dst-port 123 keep-state Standard, and correctly using keep-state, for connections from this box and from NAT'd clients, who are after NAT also from 'me'. > 04200 allow ip from any to any dst-port 22 recv ue0 As the first rule 10, that's perilously close to 'allow ip from any to any recv ue0' .. at least for non-tcp|udp protocols, if not for all. > 65535 deny ip from any to any > > If I remove rule 10 it will NOT work with ue0, the same ruleset without > rule 10 DOES work with re0 on the same machine (re0 as external and ue0 as > internal NIC). This only made sense to me as an ICMP issue; I couldn't see how else adding those rules could resolve any rx/txcsum issues? Oh wait, yes they do bypass the divert .. well I stand by whinging about the lack of icmp in this ruleset, but when in doubt, listen to Luigi! > If I connect from the gateway on 172.17.2.1 to the ssh server on this > machine, I can see the ACK and SYN+ACK but there's no ACK from the client > to the server to establish the tcp session. Only difference I could find > was that the checksum was incorrect. You might need to post a tcpdump segment of the exchange, not forgetting to show the icmp too .. > Found an older PR kern/170081 about fxp having trouble with nat when > rxcsum/txcsum was enabled, that is why I started fiddling with > rxcsum/txcsum and found that the NIC is unusable/dead without rxcsum/txcsum > enabled so this was not an option. It would seem to be a bug in ue? or axe? if the NIC won't work without. Meanwhile is using re0 as the external interface an ongoing option? Of course, that might just move the problem to the inside interface .. > # ifconfig ue0 > ue0: flags=8843 metric 0 mtu 1500 > options=8000b > ether 00:60:6e:42:5b:53 > inet6 fe80::260:6eff:fe42:5b53%ue0 prefixlen 64 scopeid 0x7 > inet 172.17.2.111 netmask 0xffffff00 broadcast 172.17.2.255 > nd6 options=21 > media: Ethernet autoselect (100baseTX ) > status: active > > Any suggestions or pointers? > > Kind regards, > > Spil. cheers, Ian From owner-freebsd-ipfw@FreeBSD.ORG Sun Apr 14 17:34:12 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9F09EFDC for ; Sun, 14 Apr 2013 17:34:12 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-oa0-f48.google.com (mail-oa0-f48.google.com [209.85.219.48]) by mx1.freebsd.org (Postfix) with ESMTP id 714E0A5E for ; Sun, 14 Apr 2013 17:34:12 +0000 (UTC) Received: by mail-oa0-f48.google.com with SMTP id j1so3809960oag.21 for ; Sun, 14 Apr 2013 10:34:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=W8NIUOJfg1Vd8yByUQ4oQzVN+tp3rJ2oj4+ls+4wDsU=; b=Qd2MJRuMZG/dlM8PpKcxElc8Je0dyfiyC6Urn+oMOE9681GQA1N7OiIMwgBM0vEZat Q9j5pavc9L/Dxi4F/BZTdj/uHpjws9NV+xmF2Gd+0apJFengun5vp3YMPGeH3XvMWb/B TPBPd2pJnkPblrl+ugpah1lOomJMRPqVvtdIQsJTV8C+GosG3w2YMRXGgjyYqbbXDUVl 9uQrxuY0Qqr4Lqf/2nkihR5TnV61TaBpPyFyBWkHUndH0Wl7vkOvftxwsTEZxMQ3yoUy 3Qs414/ZPqJZNoJi8rVs01v/9J7LrAK8iLt4bbpg5jo5v+6ul+pm6Ip8gBOfcZM5f8aw XFxw== MIME-Version: 1.0 X-Received: by 10.182.125.200 with SMTP id ms8mr2278829obb.67.1365960846244; Sun, 14 Apr 2013 10:34:06 -0700 (PDT) Received: by 10.60.140.229 with HTTP; Sun, 14 Apr 2013 10:34:06 -0700 (PDT) In-Reply-To: <20130415015850.Y56386@sola.nimnet.asn.au> References: <20130415015850.Y56386@sola.nimnet.asn.au> Date: Sun, 14 Apr 2013 10:34:06 -0700 Message-ID: Subject: Re: Problems with ipfw/natd and axe(4) From: Michael Sierchio To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQl9+xagT1vml+DLT39q29iJ9GKCQbMJQdiaQmcjXXWC4lI/l2WHs+NKDV7IQUekC1qZOrKM X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 17:34:12 -0000 On Sun, Apr 14, 2013 at 10:26 AM, Ian Smith wrote: > 'allow ip' aka 'allow all' doesn't usually take a port number, which > applies only to tcp and udp. It does in ipfw - in which case it means ( udp | tcp ) From owner-freebsd-ipfw@FreeBSD.ORG Sun Apr 14 18:49:21 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B96AFDF9 for ; Sun, 14 Apr 2013 18:49:21 +0000 (UTC) (envelope-from spil.oss@gmail.com) Received: from mail-ia0-x22c.google.com (mail-ia0-x22c.google.com [IPv6:2607:f8b0:4001:c02::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 8BA95D5A for ; Sun, 14 Apr 2013 18:49:21 +0000 (UTC) Received: by mail-ia0-f172.google.com with SMTP id k38so3782051iah.3 for ; Sun, 14 Apr 2013 11:49:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=KDtZMRCzmAR6ydKNQeAy6PY2oFFK2NLTr/lt359QLhU=; b=xZLSfGpOolh4GGBxVlOaP47rnpR3ukJTQsAzc/CrkK2Zolb/BfneZVlMVXxjAOgqHz Su/3D+v9bJHkdXL9RTFRlO/znRY05XIU4FTHhTBRzsN9zztzEUma7iJ/uE2mmOPR4EiN +c+/nX+Sqj/FqilQydZgZNlb7S0Q0mMUmXMk4RhjSgRmuzsX4/aRfNs85+uIxP0BAfxw Od9Gfa+e02GN/dAogq6p/UJYNjg6vpu5SLOB1J0emriyfa8OajGtzt+KPiGBwKhOzMUb LJCGZCauBUg0i7SR6jqUoZHADnoITKkyqbiREqZN/AsxV0KMN2dkv3uhN9UeXSzUrKPY cxYQ== MIME-Version: 1.0 X-Received: by 10.50.50.40 with SMTP id z8mr3641396ign.59.1365965361169; Sun, 14 Apr 2013 11:49:21 -0700 (PDT) Received: by 10.42.189.4 with HTTP; Sun, 14 Apr 2013 11:49:20 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 20:49:20 +0200 Message-ID: Subject: Re: Problems with ipfw/natd and axe(4) From: Spil Oss To: Michael Sierchio Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-ipfw@freebsd.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: spil.oss@gmail.com List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 18:49:21 -0000 Hi Michael, I was aware that these deny-rules could be created more efficiently, but I wanted to stay as close as possible to the 'simple' ruleset from the base /etc/rc.firewall rules. settings for /etc/rc.firewall natd_interface=ue0 firewall_simple_iif=re0 # Inside IPv4 network interface. firewall_simple_inet=10.16.1.2.1 # Inside IPv4 network address. firewall_simple_oif=ue0 # Outside IPv4 network interface. firewall_simple_onet=172.17.2.111 # Outside IPv4 network address. #firewall_simple_iif_ipv6= Inside IPv6 network interface. #firewall_simple_inet_ipv6= Inside IPv6 network prefix. #firewall_simple_oif_ipv6= Outside IPv6 network interface. #firewall_simple_onet_ipv6= Outside IPv6 network prefix. 1. Requirements and intentions This machine is my next router/firewall and web/mail/etc-server. ue0 connected to my cable modem, re0 connected to my LAN, ath0 my access point. re0 and ath0 will later be part of bridge0. Webserver and mail-services running in separate jails using natd redirect_port, cannot initiate outbound connections. Inbound ssh, http(s), smtp, imaps, submission allowed in from internet. I'm aware that with checksum offloading the tcpdump checksum will appear incorrect. Earlier I worked with the examples from the handbook which I was told on this mailing-list are not a good starting point for a ruleset. This is why I have taken the ruleset delivered in base as my starting point. The fact is that if I reverse re0 and ue0 (configuration as well as physical connections) the ruleset behaves as expected. 2. Interaction of NAT + stateful I'm trying to understand the interaction, starting with the most default. I had tcpdump, /var/log/security, natd -v running in separate tmux windows to try and figure out what happens. 3. Gradually add complexity I had taken /etc/rc.firewall as the lowest level of complexity and intend to add the extra functions. Will bring it down to the bare minimum and try again. natd config from /etc/rc.conf natd_interface=ue0 natd_flags=" -config /etc/natd.conf" or natd -n ue0 -f /etc/natd.conf -v /etc/natd.conf same_ports yes dynamic yes # redirect http, https to web-jail # Disabled Thanks for the pointers! Kind regards, Spil. On Sat, Apr 13, 2013 at 8:01 PM, Michael Sierchio wrote: > There are some things about this ruleset that are confused. Multiple > deny rules where one will do, et. > > > 01100 deny ip from 10.16.2.1 to any in via ue0 > > 01200 deny ip from 172.17.2.111 to any in via re0 > > 01300 deny ip from any to 10.0.0.0/8 via ue0 > > 01500 deny ip from any to 192.168.0.0/16 via ue0 > > 01600 deny ip from any to 0.0.0.0/8 via ue0 > > 01700 deny ip from any to 169.254.0.0/16 via ue0 > > 01800 deny ip from any to 192.0.2.0/24 via ue0 > > 01900 deny ip from any to 224.0.0.0/4 via ue0 > > 02000 deny ip from any to 240.0.0.0/4 via ue0 > > and you need to think about inbound and outbound traffic, and a few > other things. You have keep-state rules way down the ruleset and a > divert natd in the middle. This won't do. > > 1. State what the requirements and intent are. I'm reluctant to dive > into the solution space for an ill-defined problem. You conclude that > the problem is with the NIC, and I think it's with your ruleset. For > example, which interfaces are external, what's the topology, do > external interfaces have public or private addresses, etc? Is this > a firewall or a standalone box? > > Note that if you do a tcpdump, the checksums will look wrong because > they're offloaded onto the NIC. That's normal. > > 2. Until you understand the interaction of NAT + stateful rules, > don't use them. > > 3. Start with a small ruleset and nat config (show us your natd > config) that is permissive, then gradually add protection. natd by > itself is stateful, and will probably provide all you need. > > - M > > On Sat, Apr 13, 2013 at 6:34 AM, Spil Oss wrote: > > Hi All, > > > > I can't use ipfw with natd with my ASIX AX88772B USB NIC > > > > ipfw ruleset (slightly modified /etc/rc.firewall simple ruleset) > > 00010 allow ip from any to me dst-port 22 recv ue0 > > 00010 allow tcp from me 22 to any xmit ue0 > > 00100 allow ip from any to any via lo0 > > 00200 deny ip from any to 127.0.0.0/8 > > 00300 deny ip from 127.0.0.0/8 to any > > 00400 deny ip from any to ::1 > > 00500 deny ip from ::1 to any > > 00600 allow ipv6-icmp from :: to ff02::/16 > > 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 > > 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 > > 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 > > 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 > > 01100 deny ip from 10.16.2.1 to any in via ue0 > > 01200 deny ip from 172.17.2.111 to any in via re0 > > 01300 deny ip from any to 10.0.0.0/8 via ue0 > > 01500 deny ip from any to 192.168.0.0/16 via ue0 > > 01600 deny ip from any to 0.0.0.0/8 via ue0 > > 01700 deny ip from any to 169.254.0.0/16 via ue0 > > 01800 deny ip from any to 192.0.2.0/24 via ue0 > > 01900 deny ip from any to 224.0.0.0/4 via ue0 > > 02000 deny ip from any to 240.0.0.0/4 via ue0 > > 02100 divert 8668 ip4 from any to any via ue0 > > 02200 deny ip from 10.0.0.0/8 to any via ue0 > > 02400 deny ip from 192.168.0.0/16 to any via ue0 > > 02500 deny ip from 0.0.0.0/8 to any via ue0 > > 02600 deny ip from 169.254.0.0/16 to any via ue0 > > 02700 deny ip from 192.0.2.0/24 to any via ue0 > > 02800 deny ip from 224.0.0.0/4 to any via ue0 > > 02900 deny ip from 240.0.0.0/4 to any via ue0 > > 03000 allow tcp from any to any established > > 03100 allow ip from any to any frag > > 03200 allow tcp from any to me dst-port 22 setup > > 03300 allow tcp from any to me dst-port 25 setup > > 03400 allow tcp from any to me dst-port 465 setup > > 03500 allow tcp from any to me dst-port 587 setup > > 03600 allow tcp from any to me dst-port 80 setup > > 03700 allow tcp from any to me dst-port 443 setup > > 03800 deny log logamount 5 ip4 from any to any in via ue0 setup proto tcp > > 03900 allow tcp from any to any setup > > 04000 allow udp from me to any dst-port 53 keep-state > > 04100 allow udp from me to any dst-port 123 keep-state > > 04200 allow ip from any to any dst-port 22 recv ue0 > > 65535 deny ip from any to any > > > > If I remove rule 10 it will NOT work with ue0, the same ruleset without > > rule 10 DOES work with re0 on the same machine (re0 as external and ue0 > as > > internal NIC). > > > > If I connect from the gateway on 172.17.2.1 to the ssh server on this > > machine, I can see the ACK and SYN+ACK but there's no ACK from the client > > to the server to establish the tcp session. Only difference I could find > > was that the checksum was incorrect. > > > > Found an older PR kern/170081 about fxp having trouble with nat when > > rxcsum/txcsum was enabled, that is why I started fiddling with > > rxcsum/txcsum and found that the NIC is unusable/dead without > rxcsum/txcsum > > enabled so this was not an option. > > > > # ifconfig ue0 > > ue0: flags=8843 metric 0 mtu 1500 > > options=8000b > > ether 00:60:6e:42:5b:53 > > inet6 fe80::260:6eff:fe42:5b53%ue0 prefixlen 64 scopeid 0x7 > > inet 172.17.2.111 netmask 0xffffff00 broadcast 172.17.2.255 > > nd6 options=21 > > media: Ethernet autoselect (100baseTX ) > > status: active > > > > Any suggestions or pointers? > > > > Kind regards, > > > > Spil. > > _______________________________________________ > > freebsd-ipfw@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > From owner-freebsd-ipfw@FreeBSD.ORG Sun Apr 14 19:02:06 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1ECB621E for ; Sun, 14 Apr 2013 19:02:06 +0000 (UTC) (envelope-from spil.oss@gmail.com) Received: from mail-ia0-x22d.google.com (mail-ia0-x22d.google.com [IPv6:2607:f8b0:4001:c02::22d]) by mx1.freebsd.org (Postfix) with ESMTP id E7962DD4 for ; Sun, 14 Apr 2013 19:02:05 +0000 (UTC) Received: by mail-ia0-f173.google.com with SMTP id j5so3239698iaf.4 for ; Sun, 14 Apr 2013 12:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=SrJeHA7S4wY+uMJt3ZFd37KVjlS5+i/WkzZKfU+hu7c=; b=RxfL0GYU4v6g29cVWf924yaX2vzkyAGuvH3M8GBBU9vZ38tBlG0+CwKiyPeYCMkWhl USowQK9tbFUeopW1zKK7wD6MQmTehqBG/wlac4fVFCxUX+/UeOcZe4oSjZRbmPHthLwU QFQ6jl8QtfzcPIYE8fQNkIdrLUB9BXYmz4DwM/TN2ppa5aiAVG3IiTrn2ILDsiKEdO99 IDm90axMCvi/co64IN0ItagBZv6+NIE56AN35VTsxrWyYReoyzJWdsqSCHzIDgYbIvLT TP7hSLKdJPjuvq/x0VXcER29bHzh83VXJmkiYl/4F8Oce6wvECtD2eqHpd+BE/iRW7/Y EgnQ== MIME-Version: 1.0 X-Received: by 10.43.8.133 with SMTP id os5mr1698736icb.38.1365966125532; Sun, 14 Apr 2013 12:02:05 -0700 (PDT) Received: by 10.42.189.4 with HTTP; Sun, 14 Apr 2013 12:02:05 -0700 (PDT) In-Reply-To: <20130415015850.Y56386@sola.nimnet.asn.au> References: <20130415015850.Y56386@sola.nimnet.asn.au> Date: Sun, 14 Apr 2013 21:02:05 +0200 Message-ID: Subject: Re: Problems with ipfw/natd and axe(4) From: Spil Oss To: Ian Smith Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-ipfw@freebsd.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: spil.oss@gmail.com List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 19:02:06 -0000 Hi Ian, Testing the machine I connect it to my LAN which runs on 172.17.2.1/24 so I had to delete these 2 anti-spoofing rules whilst setting up 4200 is me forgetting to set the rulenumber 10 to at least allow me to get to the machine... And indeed not very safe :D Not an issue as long as the machine is in testing on my LAN Added some services indeed to the default set but haven't gotten further than testing ssh. I'd love the patches to test in-kernel nat :D I had plans of switching to ipfw_nat but haven't yet wanting to stay close to the default ruleset, learn and then improve. Your comments in my ruleset are a nice narrative of why and how these rules work! Thanks! I'll create some tcpdumps including icmp, in various test-cases. ue0 <-> re0 reversed as well to show differences. I have a thread about the axe NIC as well not wanting to work with checksumming disabled, will follow up there as well! Switching re0 and ue0 is an option for testing, but running my LAN on Fast Ethernet wasn't in my plans :D I'm still looking for some other USB NIC as this box but find ASIX parts only. This machine has no other interfaces I can connect an additional NIC to. Thanks! Spil. On Sun, Apr 14, 2013 at 7:26 PM, Ian Smith wrote: > On Sat, 13 Apr 2013 15:34:39 +0200, Spil Oss wrote: > > Hi All, > > > > I can't use ipfw with natd with my ASIX AX88772B USB NIC > > > > ipfw ruleset (slightly modified /etc/rc.firewall simple ruleset) > > I see you omitted the 2 anti-spoofing rules for 172.16.0.0/12 either > side of the divert rule, as you had to, seeing your outside interface > (so, gateway) is on 172.17.2.111(/24?). I didn't see any other changes > to 'simple' apart from rules 10 and the equally weird rule 4200, plus > your extra tcp services below. Is that right? > > > 00010 allow ip from any to me dst-port 22 recv ue0 > > 00010 allow tcp from me 22 to any xmit ue0 > > That these rules make the particular client connection from the gateway > work seemed pretty confusing, as you properly allow ssh connections in > below, and allow all established tcp, however .. > > 'allow ip' aka 'allow all' doesn't usually take a port number, which > applies only to tcp and udp. I'm surprised this rule isn't rejected, > unless perhaps the proto is now also tested? but in any case, this rule > should allow at least any non-tcp and non-udp IP packet through. > > And that means all ICMP in from the outside will be accepted, whereas a > long-standing bug in rc.firewall 'simple' means that NO icmp, neither in > nor out - not even on the inside network - is passed with this ruleset. > > Try replacing rules 10 with this section from the 'workstation' ruleset > (though further down, probably after NAT, would be a better place): > > # Allow "mandatory" ICMP in. > ${fwcmd} add pass icmp from any to any icmptype 3,4,11 > if [ $ipv6_available -eq 0 ]; then > ${fwcmd} add pass ipv6-icmp from any to any icmp6type 3 > fi > > Of course Luigi is likely right about the offloaded checksumming too, > but there's no reason not to have ICMP functioning on a router/NAT box, > especially when it's "mandatory" for path MTU discovery. It's possible > that there may be an MTU issue with these inbound connections here? > > Neither do I see any reason not to allow at least yourself and your LAN > clients to ping. Block inbound pings from the outside if you prefer but > allowing them outbound and ping responses in doesn't seem risky to me. > > > 00100 allow ip from any to any via lo0 > > 00200 deny ip from any to 127.0.0.0/8 > > 00300 deny ip from 127.0.0.0/8 to any > > 00400 deny ip from any to ::1 > > 00500 deny ip from ::1 to any > > 00600 allow ipv6-icmp from :: to ff02::/16 > > 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 > > 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 > > 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 > > 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 > > > 01100 deny ip from 10.16.2.1 to any in via ue0 > > 01200 deny ip from 172.17.2.111 to any in via re0 > > So following rc.firewall, one learns that ue0 is your outside interface > and re0 the inside. Looking in /etc/defaults/rc.conf you'll see that > ${inet} and ${onet} are intended to be network addresses (ip/mask) to > protect from spoofed networks. Yours only protect against those single > addresses, not the rest of the networks they're in - but it works now. > > > 01300 deny ip from any to 10.0.0.0/8 via ue0 > > 01500 deny ip from any to 192.168.0.0/16 via ue0 > > 01600 deny ip from any to 0.0.0.0/8 via ue0 > > 01700 deny ip from any to 169.254.0.0/16 via ue0 > > 01800 deny ip from any to 192.0.2.0/24 via ue0 > > 01900 deny ip from any to 224.0.0.0/4 via ue0 > > 02000 deny ip from any to 240.0.0.0/4 via ue0 > > 02100 divert 8668 ip4 from any to any via ue0 > > 02200 deny ip from 10.0.0.0/8 to any via ue0 > > 02400 deny ip from 192.168.0.0/16 to any via ue0 > > 02500 deny ip from 0.0.0.0/8 to any via ue0 > > 02600 deny ip from 169.254.0.0/16 to any via ue0 > > 02700 deny ip from 192.0.2.0/24 to any via ue0 > > 02800 deny ip from 224.0.0.0/4 to any via ue0 > > 02900 deny ip from 240.0.0.0/4 to any via ue0 > > Longwinded these may be, and although it's a pity some couldn't convince > some others that it was a good idea to put these into a table years ago, > they are correct and have been done like that for ~15 years. > > Another more recent omission in 'simple' is allowing use of kernel > ipfw_nat as well as or instead of natd, as you can for 'open' and > 'client' rulesets. Yes I did submit patches :( I'm unsure whether > ipfw_nat might or might not work better in the case Luigi outlined? > > > 03000 allow tcp from any to any established > > 03100 allow ip from any to any frag > > 03200 allow tcp from any to me dst-port 22 setup > > 03300 allow tcp from any to me dst-port 25 setup > > 03400 allow tcp from any to me dst-port 465 setup > > 03500 allow tcp from any to me dst-port 587 setup > > 03600 allow tcp from any to me dst-port 80 setup > > 03700 allow tcp from any to me dst-port 443 setup > > Standard, with additional services. > > > 03800 deny log logamount 5 ip4 from any to any in via ue0 setup proto > tcp > > 03900 allow tcp from any to any setup > > 04000 allow udp from me to any dst-port 53 keep-state > > 04100 allow udp from me to any dst-port 123 keep-state > > Standard, and correctly using keep-state, for connections from this box > and from NAT'd clients, who are after NAT also from 'me'. > > > 04200 allow ip from any to any dst-port 22 recv ue0 > > As the first rule 10, that's perilously close to 'allow ip from any to > any recv ue0' .. at least for non-tcp|udp protocols, if not for all. > > > 65535 deny ip from any to any > > > > If I remove rule 10 it will NOT work with ue0, the same ruleset without > > rule 10 DOES work with re0 on the same machine (re0 as external and ue0 > as > > internal NIC). > > This only made sense to me as an ICMP issue; I couldn't see how else > adding those rules could resolve any rx/txcsum issues? > > Oh wait, yes they do bypass the divert .. well I stand by whinging about > the lack of icmp in this ruleset, but when in doubt, listen to Luigi! > > > If I connect from the gateway on 172.17.2.1 to the ssh server on this > > machine, I can see the ACK and SYN+ACK but there's no ACK from the > client > > to the server to establish the tcp session. Only difference I could find > > was that the checksum was incorrect. > > You might need to post a tcpdump segment of the exchange, not forgetting > to show the icmp too .. > > > Found an older PR kern/170081 about fxp having trouble with nat when > > rxcsum/txcsum was enabled, that is why I started fiddling with > > rxcsum/txcsum and found that the NIC is unusable/dead without > rxcsum/txcsum > > enabled so this was not an option. > > It would seem to be a bug in ue? or axe? if the NIC won't work without. > Meanwhile is using re0 as the external interface an ongoing option? Of > course, that might just move the problem to the inside interface .. > > > # ifconfig ue0 > > ue0: flags=8843 metric 0 mtu > 1500 > > options=8000b > > ether 00:60:6e:42:5b:53 > > inet6 fe80::260:6eff:fe42:5b53%ue0 prefixlen 64 scopeid 0x7 > > inet 172.17.2.111 netmask 0xffffff00 broadcast 172.17.2.255 > > nd6 options=21 > > media: Ethernet autoselect (100baseTX ) > > status: active > > > > Any suggestions or pointers? > > > > Kind regards, > > > > Spil. > > cheers, Ian > From owner-freebsd-ipfw@FreeBSD.ORG Mon Apr 15 06:25:12 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id ECCB4912 for ; Mon, 15 Apr 2013 06:25:12 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 661B86C0 for ; Mon, 15 Apr 2013 06:25:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r3F6P9Uw097111; Mon, 15 Apr 2013 16:25:09 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 15 Apr 2013 16:25:09 +1000 (EST) From: Ian Smith To: Michael Sierchio Subject: Re: Problems with ipfw/natd and axe(4) In-Reply-To: Message-ID: <20130415160625.K56386@sola.nimnet.asn.au> References: <20130415015850.Y56386@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Spil Oss , freebsd-ipfw@freebsd.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 06:25:13 -0000 On Sun, 14 Apr 2013 10:34:06 -0700, Michael Sierchio wrote: > On Sun, Apr 14, 2013 at 10:26 AM, Ian Smith wrote: > > > 'allow ip' aka 'allow all' doesn't usually take a port number, which > > applies only to tcp and udp. > > It does in ipfw - in which case it means ( udp | tcp ) You're quite right, and my assumption that it would also permit icmp was quite wrong, after a quick test. Which appears to leave the bypassed divert not working with rx/txcsum the only viable suspect. The ruleset is otherwise 'out of the box'. Does anyone know whether this is an issue with libalias(3) generally - in which case using nat instead of divert shouldn't help - or just with natd in particular? cheers, Ian From owner-freebsd-ipfw@FreeBSD.ORG Mon Apr 15 11:06:45 2013 Return-Path: Delivered-To: freebsd-ipfw@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D7227965 for ; Mon, 15 Apr 2013 11:06:45 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id C7D1C7A0 for ; Mon, 15 Apr 2013 11:06:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3FB6jxI015134 for ; Mon, 15 Apr 2013 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3FB6jIW015132 for freebsd-ipfw@FreeBSD.org; Mon, 15 Apr 2013 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 15 Apr 2013 11:06:45 GMT Message-Id: <201304151106.r3FB6jIW015132@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-ipfw@FreeBSD.org Subject: Current problem reports assigned to freebsd-ipfw@FreeBSD.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 11:06:45 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/176503 ipfw [ipfw] ipfw layer2 problem o kern/174749 ipfw Unexpected change of default route o kern/169206 ipfw [ipfw] ipfw does not flush entries in table o conf/167822 ipfw [ipfw] [patch] start script doesn't load firewall_type o kern/166406 ipfw [ipfw] ipfw does not set ALTQ identifier for ipv6 traf o kern/165939 ipfw [ipfw] bug: incomplete firewall rules loaded if tables o kern/165190 ipfw [ipfw] [lo] [patch] loopback interface is not marking o kern/158066 ipfw [ipfw] ipfw + netgraph + multicast = multicast packets o kern/157796 ipfw [ipfw] IPFW in-kernel NAT nat loopback / Default Route o kern/157689 ipfw [ipfw] ipfw nat config does not accept nonexistent int f kern/155927 ipfw [ipfw] ipfw stops to check packets for compliance with o bin/153252 ipfw [ipfw][patch] ipfw lockdown system in subsequent call o kern/153161 ipfw [ipfw] does not support specifying rules with ICMP cod o kern/152113 ipfw [ipfw] page fault on 8.1-RELEASE caused by certain amo o kern/148827 ipfw [ipfw] divert broken with in-kernel ipfw o kern/148430 ipfw [ipfw] IPFW schedule delete broken. o kern/148091 ipfw [ipfw] ipfw ipv6 handling broken. f kern/143973 ipfw [ipfw] [panic] ipfw forward option causes kernel reboo o kern/143621 ipfw [ipfw] [dummynet] [patch] dummynet and vnet use result o kern/137346 ipfw [ipfw] ipfw nat redirect_proto is broken o kern/137232 ipfw [ipfw] parser troubles o kern/135476 ipfw [ipfw] IPFW table breaks after adding a large number o o kern/129036 ipfw [ipfw] 'ipfw fwd' does not change outgoing interface n o kern/127230 ipfw [ipfw] [patch] Feature request to add UID and/or GID l f kern/122963 ipfw [ipfw] tcpdump does not show packets redirected by 'ip s kern/121807 ipfw [request] TCP and UDP port_table in ipfw o kern/116009 ipfw [ipfw] [patch] Ignore errors when loading ruleset from o bin/104921 ipfw [patch] ipfw(8) sometimes treats ipv6 input as ipv4 (a o kern/104682 ipfw [ipfw] [patch] Some minor language consistency fixes a o kern/103454 ipfw [ipfw] [patch] [request] add a facility to modify DF b o kern/103328 ipfw [ipfw] [request] sugestions about ipfw table o kern/97951 ipfw [ipfw] [patch] ipfw does not tie interface details to o kern/95084 ipfw [ipfw] [regression] [patch] IPFW2 ignores "recv/xmit/v o kern/86957 ipfw [ipfw] [patch] ipfw mac logging o bin/83046 ipfw [ipfw] ipfw2 error: "setup" is allowed for icmp, but s o kern/82724 ipfw [ipfw] [patch] [request] Add setnexthop and defaultrou o bin/78785 ipfw [patch] ipfw(8) verbosity locks machine if /etc/rc.fir o kern/60719 ipfw [ipfw] Headerless fragments generate cryptic error mes s kern/55984 ipfw [ipfw] [patch] time based firewalling support for ipfw o kern/48172 ipfw [ipfw] [patch] ipfw does not log size and flags o kern/46159 ipfw [ipfw] [patch] [request] ipfw dynamic rules lifetime f a kern/26534 ipfw [ipfw] Add an option to ipfw to log gid/uid of who cau 42 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Mon Apr 15 19:04:33 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 01610439 for ; Mon, 15 Apr 2013 19:04:33 +0000 (UTC) (envelope-from spil.oss@gmail.com) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) by mx1.freebsd.org (Postfix) with ESMTP id C49AF1436 for ; Mon, 15 Apr 2013 19:04:32 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id bn7so6156499ieb.37 for ; Mon, 15 Apr 2013 12:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=8At6pn7/Vw9B628R8DnlMDIb15vaMUpu3QUylTflrNg=; b=qC7CDxzXQQ5vBx+DKTZAJAT0N9FXr9ZCjLx/YxTzaN73u3mJMOG3iA7qMNxMD8sIwB 86/p6BdVE1lYV8XnQJVjUlp1mnRRvKIcDf6GbUpMFmqhjyEjVkosxu+Tbg13MCE6cd2I sblqmy2pX5OIEUbhD/pTuJQxIggXCWqJdoWD2wJZ/7r33DNgBOCit5m2aM0mKl0JfTmB YZI/CaOA/CHSs4+YUY1wMzci2Yso3DANZaC1/Htf9HjfXacdPOZCKiwWPY+ZQX4ySvJX CqiZYGInzUTvrw8WF9ooI5wf4i5ZPrTFb/GHXMYyj0pl4T7Iip+NAmIfKxDmEv/Jldph M/5Q== MIME-Version: 1.0 X-Received: by 10.50.50.40 with SMTP id z8mr6134733ign.59.1366052672464; Mon, 15 Apr 2013 12:04:32 -0700 (PDT) Received: by 10.42.189.4 with HTTP; Mon, 15 Apr 2013 12:04:32 -0700 (PDT) In-Reply-To: <20130415160625.K56386@sola.nimnet.asn.au> References: <20130415015850.Y56386@sola.nimnet.asn.au> <20130415160625.K56386@sola.nimnet.asn.au> Date: Mon, 15 Apr 2013 21:04:32 +0200 Message-ID: Subject: Re: Problems with ipfw/natd and axe(4) From: Spil Oss To: Ian Smith Content-Type: multipart/mixed; boundary=089e0118320c496d1604da6aec84 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-ipfw@freebsd.org, Michael Sierchio X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: spil.oss@gmail.com List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:04:33 -0000 --089e0118320c496d1604da6aec84 Content-Type: text/plain; charset=ISO-8859-1 Hi all, Network dumps as promised On 172.17.2.1: tcpdump -p -i bridge0 -s 0 -w ssh-fail.pcap host not 172.17.2.167 >From 172.17.2.1 I ran telnet 172.17.2.111/157 22 In Wireshark I trimmed the capture a bit further with expression 'not stp and not http' Initial setup (ue0 ext, re0 int, rule 10 to allow ssh) -> ue0-ssh-success.pcap Removed rule 10 -> ue0-ssh-fail.pcap Switched re0 and ue0, default ruleset (without 10) -> re0-ssh-success.pcap According to YungHyeong the sample ASIX NIC he has works normally when checksumming is disabled. Kind regards, Spil. On Mon, Apr 15, 2013 at 8:25 AM, Ian Smith wrote: > On Sun, 14 Apr 2013 10:34:06 -0700, Michael Sierchio wrote: > > On Sun, Apr 14, 2013 at 10:26 AM, Ian Smith > wrote: > > > > > 'allow ip' aka 'allow all' doesn't usually take a port number, which > > > applies only to tcp and udp. > > > > It does in ipfw - in which case it means ( udp | tcp ) > > You're quite right, and my assumption that it would also permit icmp > was quite wrong, after a quick test. > > Which appears to leave the bypassed divert not working with rx/txcsum > the only viable suspect. The ruleset is otherwise 'out of the box'. > > Does anyone know whether this is an issue with libalias(3) generally - > in which case using nat instead of divert shouldn't help - or just with > natd in particular? > > cheers, Ian > --089e0118320c496d1604da6aec84 Content-Type: application/octet-stream; name="ue0-ssh-success.pcap" Content-Disposition: attachment; filename="ue0-ssh-success.pcap" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hfk07tn90 1MOyoQIABAAAAAAAAAAAAP//AAABAAAAAEhsUVpTBQBKAAAASgAAAABgbkJbU2a7ILCfuwgARRAA PNEqQABABgzvrBECAawRAm81kwAWf3SVWQAAAACgAv//8WAAAAIEBbQBAwMGBAIICqd3CB8AAAAA AEhsUUBaBQBKAAAASgAAAGa7ILCfuwBgbkJbUwgARQAAPAByQABABt23rBECb6wRAgEAFjWTkzJU Y390lVqgEv//d74AAAIEBbQBAwMGBAIICqz+5PyndwgfAEhsUapaBQBCAAAAQgAAAABgbkJbU2a7 ILCfuwgARRAANNEsQABABgz1rBECAawRAm81kwAWf3SVWpMyVGSAEAQQoncAAAEBCAqndwghrP7k /ABIbFF4oQUAcQAAAHEAAABmuyCwn7sAYG5CW1MIAEUAAGMAc0AAQAbdj6wRAm+sEQIBABY1k5My VGR/dJVagBgEEGHoAAABAQgKrP7lDad3CCFTU0gtMi4wLU9wZW5TU0hfNi4xX2hwbjEzdjExIEZy ZWVCU0QtMjAxMjA5MDENCgBIbFEhKAcAQgAAAEIAAAAAYG5CW1NmuyCwn7sIAEUQADTRQ0AAQAYM 3qwRAgGsEQJvNZMAFn90lVqTMlSTgBAEEKHBAAABAQgKp3cIl6z+5Q0GSGxRHdUJAEIAAABCAAAA AGBuQltTZrsgsJ+7CABFEAA01StAAEAGCPasEQIBrBECbzWTABZ/dJVakzJUk4ARBBCJoQAAAQEI Cqd3ILas/uUNBkhsUQvcCQBCAAAAQgAAAGa7ILCfuwBgbkJbUwgARQAANAB0QABABt29rBECb6wR AgEAFjWTkzJUk390lVuAEAQQcRgAAAEBCAqs/v2Wp3cgtgZIbFEJ3wkAQgAAAEIAAABmuyCwn7sA YG5CW1MIAEUAADQAdUAAQAbdvKwRAm+sEQIBABY1k5MyVJN/dJVbgBEEEHEXAAABAQgKrP79lqd3 ILYGSGxRTN8JAEIAAABCAAAAAGBuQltTZrsgsJ+7CABFEAA01S1AAEAGCPSsEQIBrBECbzWTABZ/ dJVbkzJUlIAQBBBxFAAAAQEICqd3ILms/v2W --089e0118320c496d1604da6aec84 Content-Type: application/octet-stream; name="ue0-ssh-fail.pcap" Content-Disposition: attachment; filename="ue0-ssh-fail.pcap" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hfk0dsas1 1MOyoQIABAAAAAAAAAAAAP//AAABAAAAqkhsUfTjBQBKAAAASgAAAABgbkJbU2a7ILCfuwgARRAA PFF+QABABoybrBECAawRAm/zXAAWr5htewAAAACgAv//kxkAAAIEBbQBAwMGBAIICqd5oFQAAAAA qkhsUdnqBQBKAAAASgAAAGa7ILCfuwBgbkJbUwgARQAAPACAQABABt2prBECb6wRAgEAFvNc6BsY a6+YbXygEv//XMEAAAIEBbQBAwMGBAIICkWr/nKneaBUrUhsUb/lBQBKAAAASgAAAABgbkJbU2a7 ILCfuwgARRAAPFNmQABABoqzrBECAawRAm/zXAAWr5htewAAAACgAv//h2EAAAIEBbQBAwMGBAII Cqd5rAwAAAAArUhsUajsBQBKAAAASgAAAGa7ILCfuwBgbkJbUwgARQAAPACBQABABt2orBECb6wR AgEAFvNc6BsYa6+YbXygEv//XMEAAAIEBbQBAwMGBAIICkWr/nKneaBUrUhsUT7tBQBKAAAASgAA AGa7ILCfuwBgbkJbUwgARQAAPACCQABABt2nrBECb6wRAgEAFvNc6BsYa6+YbXygEv//XMEAAAIE BbQBAwMGBAIICkWr/nKneawMsEhsUQkABgBKAAAASgAAAGa7ILCfuwBgbkJbUwgARQAAPACDQABA Bt2mrBECb6wRAgEAFvNc6BsYa6+YbXygEv//XMEAAAIEBbQBAwMGBAIICkWr/nKneawMsEhsUbn1 CABKAAAASgAAAABgbkJbU2a7ILCfuwgARRAAPFVZQABABojArBECAawRAm/zXAAWr5htewAAAACg Av//euAAAAIEBbQBAwMGBAIICqd5uI0AAAAAsEhsUab8CABKAAAASgAAAGa7ILCfuwBgbkJbUwgA RQAAPACEQABABt2lrBECb6wRAgEAFvNc6BsYa6+YbXygEv//XMEAAAIEBbQBAwMGBAIICkWr/nKn ebiNs0hsUQ0NCQBKAAAASgAAAGa7ILCfuwBgbkJbUwgARQAAPACFQABABt2krBECb6wRAgEAFvNc 6BsYa6+YbXygEv//XMEAAAIEBbQBAwMGBAIICkWr/nKnebiNs0hsUZ8FDAA+AAAAPgAAAABgbkJb U2a7ILCfuwgARRAAMFeGQABABoafrBECAawRAm/zXAAWr5htewAAAABwAv//FwcAAAIEBbQEAgAA s0hsUXwMDABCAAAAQgAAAGa7ILCfuwBgbkJbUwgARQAANACGQABABt2rrBECb6wRAgEAFvNc6BsY a6+YbXyAEv//XLkAAAIEBbQBAwMGBAIAAA== --089e0118320c496d1604da6aec84 Content-Type: application/octet-stream; name="re0-ssh-success.pcap" Content-Disposition: attachment; filename="re0-ssh-success.pcap" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hfk0dxij2 1MOyoQIABAAAAAAAAAAAAP//AAABAAAAmUpsUYUgAgBKAAAASgAAAEBhhhCGwWa7ILCfuwgARRAA PKa0QABABjc3rBECAawRAp2KXgAWnq5AegAAAACgAv//rS0AAAIEBbQBAwMGBAIICqeBLPQAAAAA mUpsUWohAgBKAAAASgAAAGa7ILCfu0BhhhCGwQgARQAAPAC1QABABt1GrBECnawRAgEAFopetvA6 556uQHugEv//3FEAAAIEBbQBAwMGBAIICmWfeVOngSz0mUpsUdQhAgBCAAAAQgAAAEBhhhCGwWa7 ILCfuwgARRAANKa1QABABjc+rBECAawRAp2KXgAWnq5Ae7bwOuiAEAQQBwwAAAEBCAqngSz1ZZ95 U5lKbFGjYgIAcQAAAHEAAABmuyCwn7tAYYYQhsEIAEUAAGMAtkAAQAbdHqwRAp2sEQIBABaKXrbw OuierkB7gBgEEMZ5AAABAQgKZZ95Z6eBLPVTU0gtMi4wLU9wZW5TU0hfNi4xX2hwbjEzdjExIEZy ZWVCU0QtMjAxMjA5MDENCplKbFFl5gMAQgAAAEIAAABAYYYQhsFmuyCwn7sIAEUQADSm1UAAQAY3 HqwRAgGsEQKdil4AFp6uQHu28DsXgBAEEAZUAAABAQgKp4EtamWfeWeeSmxRA5QMAEIAAABCAAAA QGGGEIbBZrsgsJ+7CABFEAA0qnFAAEAGM4KsEQIBrBECnYpeABaerkB7tvA7F4ARBBDwkgAAAQEI CqeBQypln3lnnkpsUc6UDABCAAAAQgAAAGa7ILCfu0BhhhCGwQgARQAANAC3QABABt1MrBECnawR AgEAFopetvA7F56uQHyAEAQQ2mwAAAEBCApln4+Np4FDKp5KbFF3mAwAQgAAAEIAAABmuyCwn7tA YYYQhsEIAEUAADQAuEAAQAbdS6wRAp2sEQIBABaKXrbwOxeerkB8gBEEENprAAABAQgKZZ+PjaeB QyqeSmxR0pgMAEIAAABCAAAAQGGGEIbBZrsgsJ+7CABFEAA0qnJAAEAGM4GsEQIBrBECnYpeABae rkB8tvA7GIAQBBDaagAAAQEICqeBQytln4+N --089e0118320c496d1604da6aec84-- From owner-freebsd-ipfw@FreeBSD.ORG Tue Apr 16 18:52:06 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 29DFD42A for ; Tue, 16 Apr 2013 18:52:06 +0000 (UTC) (envelope-from spil.oss@gmail.com) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) by mx1.freebsd.org (Postfix) with ESMTP id F25BEF3A for ; Tue, 16 Apr 2013 18:52:05 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id bn7so935335ieb.37 for ; Tue, 16 Apr 2013 11:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=kvfuQkZYAKDUnKypveb+JTkWwQAYzNUNxT4YvR5gOmc=; b=R0DDPRdg5ok/lT8hDtQzLFte02ZOxJCCyEG/i13+1hs/RWUOthXQ0WHL2+ZxjdvhdV FrmzTCIivq7HAJ9WuDews0lgebhXYpnn/znkioB6G5fc/7UdIA0+x4YYGg65x14X6SMw yYr9pRBeeKRCuRCnBu5+iVcwGgE6kSyIJuTh/KEm0tbzNwyt4pC+wDR0JlgLS4mCm+iX yL9VOiQ4buk9WNmgzdFgZsU0GgiFUa7SFQHkS88DMY+0O8frZjBIJMTg3Y98J9kEJfPa vwp2ewx0jq3z33vuIezN2DkWw1f82KDfKQuQ0bLXbnTzSsbWiFyCu6qmotQtMbm9Krwk 3AyA== MIME-Version: 1.0 X-Received: by 10.42.80.196 with SMTP id w4mr1544008ick.43.1366138325737; Tue, 16 Apr 2013 11:52:05 -0700 (PDT) Received: by 10.42.189.4 with HTTP; Tue, 16 Apr 2013 11:52:05 -0700 (PDT) In-Reply-To: References: <20130415015850.Y56386@sola.nimnet.asn.au> <20130415160625.K56386@sola.nimnet.asn.au> Date: Tue, 16 Apr 2013 20:52:05 +0200 Message-ID: Subject: Re: Problems with ipfw/natd and axe(4) From: Spil Oss To: Ian Smith Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-ipfw@freebsd.org, Michael Sierchio X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: spil.oss@gmail.com List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 18:52:06 -0000 Hi all, If I disable checksum offloading on the NIC I do the tcpdump on, then I assume that the checksum-check will provide accurate results? With checksum disabled, I see that the checksum is incorrect when the client does not respond to the SYN,ACK, and correct when it does. Out of curiousity I tried with pf as well and it behaves the same. Kind regards, Spil. On Mon, Apr 15, 2013 at 9:04 PM, Spil Oss wrote: > Hi all, > > Network dumps as promised > On 172.17.2.1: > tcpdump -p -i bridge0 -s 0 -w ssh-fail.pcap host not 172.17.2.167 > From 172.17.2.1 I ran > telnet 172.17.2.111/157 22 > In Wireshark I trimmed the capture a bit further with expression > 'not stp and not http' > > Initial setup (ue0 ext, re0 int, rule 10 to allow ssh) > -> ue0-ssh-success.pcap > Removed rule 10 > -> ue0-ssh-fail.pcap > Switched re0 and ue0, default ruleset (without 10) > -> re0-ssh-success.pcap > > According to YungHyeong the sample ASIX NIC he has works normally when > checksumming is disabled. > > Kind regards, > > Spil. > > > > > On Mon, Apr 15, 2013 at 8:25 AM, Ian Smith wrote: > >> On Sun, 14 Apr 2013 10:34:06 -0700, Michael Sierchio wrote: >> > On Sun, Apr 14, 2013 at 10:26 AM, Ian Smith >> wrote: >> > >> > > 'allow ip' aka 'allow all' doesn't usually take a port number, which >> > > applies only to tcp and udp. >> > >> > It does in ipfw - in which case it means ( udp | tcp ) >> >> You're quite right, and my assumption that it would also permit icmp >> was quite wrong, after a quick test. >> >> Which appears to leave the bypassed divert not working with rx/txcsum >> the only viable suspect. The ruleset is otherwise 'out of the box'. >> >> Does anyone know whether this is an issue with libalias(3) generally - >> in which case using nat instead of divert shouldn't help - or just with >> natd in particular? >> >> cheers, Ian >> > > From owner-freebsd-ipfw@FreeBSD.ORG Wed Apr 17 05:03:27 2013 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9E795FFE for ; Wed, 17 Apr 2013 05:03:27 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id EAE25A82 for ; Wed, 17 Apr 2013 05:03:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r3H53HMc094293; Wed, 17 Apr 2013 15:03:17 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Wed, 17 Apr 2013 15:03:17 +1000 (EST) From: Ian Smith To: Spil Oss Subject: Re: Problems with ipfw/natd and axe(4) In-Reply-To: Message-ID: <20130417133637.W56386@sola.nimnet.asn.au> References: <20130415015850.Y56386@sola.nimnet.asn.au> <20130415160625.K56386@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-ipfw@freebsd.org, Michael Sierchio X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Apr 2013 05:03:27 -0000 On Tue, 16 Apr 2013 20:52:05 +0200, Spil Oss wrote: > Hi all, > > If I disable checksum offloading on the NIC I do the tcpdump on, then I > assume that the checksum-check will provide accurate results? It certainly should. > With checksum disabled, I see that the checksum is incorrect when the > client does not respond to the SYN,ACK, and correct when it does. I'm having trouble fully parsing that. Using 'tcpdump -vr ue0-ssh-fail.pcap | less -S' shows these incorrect checksums alright; before adding -v I'd only noticed 172.17.2.1 sending SYNs and clearly not responding to 172.17.2.111's SYN/ACKs. Since it works ok with the divert rule bypassed - presumably still with tx/rxcsum enabled - then it seems that (surprise!) Luigi picked the issue being in natd / divert socket handling. > Out of curiousity I tried with pf as well and it behaves the same. Can't comment on that. What's not clear is why the NIC "doesn't work" (symptoms?) with -txcsum -rxcsum. With the 'fail' pcap it seems the received checksum from the client SYN is ok on capture, and the server is responding with SYN/ACK (with mangled cksum), but the rxcsum must be ok after natd, so maybe it's only -txcsum needed? Might that work? Sorry, I'm just bouncing around on what I can see from here and could be missing something someone else might find obvious, I'm just an amateur.. > On Mon, Apr 15, 2013 at 9:04 PM, Spil Oss wrote: > > Network dumps as promised > > On 172.17.2.1: > > tcpdump -p -i bridge0 -s 0 -w ssh-fail.pcap host not 172.17.2.167 You didn't post that one; I assume it showed the bad cksums back from 172.17.2.111? ie that the SYN/ACK packet make it to the client's interface, but was dropped for its bad cksum on the client side? > > From 172.17.2.1 I ran > > telnet 172.17.2.111/157 22 > > In Wireshark I trimmed the capture a bit further with expression > > 'not stp and not http' > > > > Initial setup (ue0 ext, re0 int, rule 10 to allow ssh) > > -> ue0-ssh-success.pcap > > Removed rule 10 > > -> ue0-ssh-fail.pcap > > Switched re0 and ue0, default ruleset (without 10) > > -> re0-ssh-success.pcap > > > > According to YungHyeong the sample ASIX NIC he has works normally when > > checksumming is disabled. I guess trying another of the same NIC is the only way to rule out a faulty unit? I'm having similarly frustrating issues with a cardbus USB2 card, unrelated but proving just as indeterminate .. [..] > >> Does anyone know whether this is an issue with libalias(3) generally - > >> in which case using nat instead of divert shouldn't help - or just with > >> natd in particular? Question still stands .. I could redo that rc.firewall patch for nat in 'simple' but if the problem is with libalias(3) it won't help with this. cheers, Ian From owner-freebsd-ipfw@FreeBSD.ORG Sat Apr 20 17:04:59 2013 Return-Path: Delivered-To: freebsd-ipfw@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1B787D2; Sat, 20 Apr 2013 17:04:59 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E98735E5; Sat, 20 Apr 2013 17:04:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3KH4wc4060676; Sat, 20 Apr 2013 17:04:58 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3KH4wID060675; Sat, 20 Apr 2013 17:04:58 GMT (envelope-from linimon) Date: Sat, 20 Apr 2013 17:04:58 GMT Message-Id: <201304201704.r3KH4wID060675@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-ipfw@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/177948: [ipfw] ipfw fails to parse port ranges (p1-p2) for udp X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 17:04:59 -0000 Old Synopsis: ipfw fails to parse port ranges (p1-p2) for udp New Synopsis: [ipfw] ipfw fails to parse port ranges (p1-p2) for udp Responsible-Changed-From-To: freebsd-bugs->freebsd-ipfw Responsible-Changed-By: linimon Responsible-Changed-When: Sat Apr 20 17:04:46 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=177948