From owner-freebsd-questions@FreeBSD.ORG Fri Nov 7 01:03:42 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4E98401 for ; Fri, 7 Nov 2014 01:03:42 +0000 (UTC) Received: from mx2.blackfoot.net (mx2.blackfoot.net [216.14.232.11]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "spam.blackfoot.net", Issuer "GeoTrust DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8EFFCB6 for ; Fri, 7 Nov 2014 01:03:42 +0000 (UTC) Received: from blackfoot.vision.net ([216.220.3.42]) by mx2.blackfoot.net ({f463150a-8fc3-47f8-9d9f-72f34f8bb0de}) via TCP (outbound) with ESMTP id 20141107010325132; Fri, 07 Nov 2014 01:03:25 +0000 X-RC-FROM: Received: from webmail.blackfoot.net (unknown [10.40.25.30]) (Authenticated sender: vagabond) by blackfoot.vision.net (Postfix) with ESMTPA id 7320071A6; Thu, 6 Nov 2014 18:03:23 -0700 (MST) Received: from 66.109.141.62 (SquirrelMail authenticated user vagabond) by webmail.blackfoot.net with HTTP; Thu, 6 Nov 2014 18:03:23 -0700 Message-ID: <7fe88aca6228abad2e4ce66abaf42893.squirrel@webmail.blackfoot.net> Date: Thu, 6 Nov 2014 18:03:23 -0700 Subject: Re: natd not translating? From: "Gary Aitken" To: smithi@nimnet.asn.au, freebsd-questions@freebsd.org User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-MAG-OUTBOUND: blackfoot.redcondor.net@216.220.3.42/32 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2014 01:03:43 -0000 > > On 11/03/14 22:37, Ian Smith wrote: > > > In freebsd-questions Digest, Vol 544, Issue 1, Message: 9 > > > On Sun, 2 Nov 2014 17:36:36 -0700 "Gary Aitken" wrote: > > world -> ep0 (66.109.141.*) fbsdbox (192.168.1.1) xl0 -> internal > > 66.109.141.60 is one of my assigned ip addrs. > You have a /24? I can hardly afford my /29 these days. Sloppy. 66.109.141.56/29, so .60 is in it. If I could afford a /24, that's not what I'd spend it on! :-) > Is fbsdbox where all your addresses are routed to? > If so, to paraphrase julian@, you don't want to waste natd's time > handling packets it doesn't care about, meaning packets that will never > be eligible to be mapped to/from your internal network .. but that's > just a refinement, for later. Yes, fbsd box is the only way out. Hmmm, I was wondering about bypassing natd for internal only traffic. Then realized it shouldn't be diverted because it stays on the internal interface. However, I hadn't considered traffic that just went in and back out the gateway. I have a non-gateway ip addr reserved for use by natd, and currently have divert 8668 ip from any to any via ep0 Since I have a non-gateway addr reserved for the natd xlations, it seems like divert 8668 ip4 from not me to not me via ep0 should have identical behavior; but it doesn't. It seems like nothing came through to clients. > Are you running any services accessible from outside on any of your IPs? yes. All served by the fbsd gateway box at the moment, on outward-facing (ep0) interface ip. Although sometimes also served by a different internal fbsd box, and for those I have a dedicated ip passed straight through by natd. > > I *think* I got the above problem even with ipfw wide open: > > 00005 allow ip from any to any > > 00010 divert 8668 ip from any to any via ep0 > Rule 5 allows everything, so no packets will get as far as rule 10. Oops, that was typed in rather than copied at the time of failure. I was debugging a dns problem and had temporarily bypassed natd. I probably had rule 5 at rule 11, after the divert when working on natd. > Swap those and you do indeed have an open firewall, doing only NAT, > though it's important to specify 'ip4' rather than 'ip' or 'all' in the > divert rule .. natd gets quite upset (TSTL) when passed IPv6 traffic. Thanks, didn't realize that. > > I say *think* because I am further along but did not go back and verify the cause. > > My head is a bit damaged and the wall is bloody. > > I believe the problem was a missing entry in /boot/loader.conf > > (ipdivert_load="YES") > > which I found as a result of this note and the references to others in it: > > http://freebsd.1045724.n5.nabble.com/Kernel-Update-IPFW-not-working-td4208637.html > Ah yes. This was fixed sometime before 9.3 on stable/9 in /etc/rc.d/ipfw: > ipfw_prestart() > so I guess you're running 8.x or an earlier 9.x? uname -a? heh. Due to various problems along the way, I'm pleading the 5th. Waiting for a disk delivery to upgrade to 9.3 > > Anyway, I'm past that problem and most things are working. > > However, still having some trouble working out my ipfw rules > > but if I can see what's happening I think I can figure it out. > Please show your ruleset; the output of 'ipfw show' will do nicely. > Personally, for a setup like yours, I would (and did) start with the > /etc/rc.firewall 'simple' ruleset. Apart from needing rules added to > pass ICMP traffic, still not fixed after many years - it's a good basic > firewall for a small network, unlike those still suggested in the IPFW > handbook page .. though there's been some work done there recently too. Thanks. My config has grown over the years and it's probably good to revisit the "simple" template and redo / modify from there. > > I can't seem to get logging to work. I have the following in natd.conf: > > log_denied > > log_ipfw_denied > > log_facility local0 > > and the following in syslog.conf > > !local0 > > *.* /var/log/natd.log > > If I run natd with verbose, I occasionally see > > "natd: failed to write packet back: Permission denied" > > errors on the controlling terminal. > > If I run without verbose (detached), I see no entries in > > /var/log/natd.log. > > That failure may relate to use of log_ipfw_denied (default when using > 'verbose' anyway) or it could be to do with IPv6 traffic, as above. > > You see no log entries at all? I'd try using the default log. I never > found much value in /var/log/alias.log (natd's default log), compared to > adding a few temporary 'count log' rules before and after the divert > rule/s, and/or running tcpdump in two consoles, one inside and one > outside, while verifying various test traffic as working. The natd alias.log file contains counts, but nothing else, which as you say seems not of much value, at least for debug purposes. After some experimentation introducing rules to force rejection of packets after natd coversion: Absent "log_facility" a message goes to "messages", but unfortunately that message is only the "failed to write packet back" message, and not the detail you get when running -verbose where you get the ip addrs; so not particularly useful. With the "log_facility" I didn't see anything different, which is probably a result of a poorly configured syslog.conf file. I'll worry about that later. I've been using tcpdump on the outward facing net and running natd -verbose to see the translations and the complaints, which works pretty well. I guess I was expecting logging to show the rejected packets, not just tell me something was rejected. I'll lower my expectations :-( > And have 'log yes' in natd.conf as well as those above. That doesn't seem to affect logging of rejected packets; only provides the stats. > If I were starting again I'd be using ipfw_nat (in-kernel NAT) instead > of natd anyway; natd(8) is still a useful reference, the descriptions in > ipfw(8) are rather terse if you don't already know natd terminology, but > it maps pretty well one-to-one with natd / divert usage, and is faster. Thanks, will look into that after upgrade. > Well let's see your ruleset (offlist if considered sensitive) and full > natd.conf, and related rules from rc.conf (gateway_enable and such); > also ifconfig, less anything sensitive, could provide a clue or two. I think I'm ok at this point. Turns out much of the problem was due to a perfect storm sort of thing -- reduction of my subnet size which forced use of natd; forced change of ips which forced reconfig of dns, and isp didn't forward the reverse dns. DSL modem has some strange behavior just discovered in the process as well. So there were some additional things going on I wasn't exactly looking at/for at the time. Thanks for your insights and suggestions; only open question at the moment is the divert 8668 ip4 from not me to not me via ep0 one. Gary