Date: Tue, 24 Nov 2009 15:51:11 +1100 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: James Long <list@museum.rain.com> Cc: freebsd-questions@freebsd.org Subject: Re: Poor throughput with natd Message-ID: <20091124144854.H61869@sola.nimnet.asn.au> In-Reply-To: <20091124024906.D807B10656EC@hub.freebsd.org> References: <20091124024906.D807B10656EC@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In freebsd-questions Digest, Vol 286, Issue 4, Message 16
On Mon, 23 Nov 2009 15:28:12 -0800 James Long <list@museum.rain.com> wrote:
> Please copy me on replies.
>
> I am testing ipfw and natd on a gateway machine running FreeBSD
> 7.2-STABLE #0: Tue Oct 27 00:12:39 PDT 2009 with the generic
> kernel. ipfw.ko and ipdivert.ko are loaded as modules, since
> they're not part of the GENERIC kernel.
>
> The symptom is that scp uploads from the gateway machine have
> very poor throughput, often showing "stalled" status in the scp
> progress output.
>
> Machines on the LAN do not suffer this problem, and can upload
> their traffic via NAT with no observed degradation in throughput.
> That's why I haven't noticed this problem until recently, when I
> tried rsync-ing some files outbound from the gateway to a remote
> machine.
>
> I can work around the problem, but this problem has never cropped
> up in the past. Is there a problem in my configuration, or in
> recent natd?
>
> Thanks for your time!
Hi Jim,
among the over-copious notes in my rc.firewall is:
#% Julian Elischer, 22Oct06 in freebsd-net:
# "one thing that you need to name sure of is that only the packets that
# have potential of being on interest to natd are passed to natd.
# i.e. be VERY specific in your natd rules..
#
# ipfw add 1000 divert natd ip from any to any out recv {inner-ineterface}
# xmit {outer-interface}.
# ipfw add 1001 divert natd ip from any to {inner-interface-address} in
# recv {outer-interface}.
#
# don't waste natd's time with packets it doesn't care about."
1001 is actually not quite right, I'll get to that, but the principle is
correct; the only packets natd can do anything useful with are these:
a) going OUT on the external interface that were received on internal
interface, so needing source address translation to the outside address.
b) coming IN on the external interface, which MAY match previous (a)
packets, so requiring destination address remapping to an internal IP.
In the case you outline, the scp is happening between this box itself
and an outside host so are of no interest to natd, costing extra time.
> All commands below were executed on the gateway machine that is
> running natd with very basic options:
>
> 15:07:37 /root# findps natd
> root 480 0.0 0.1 3388 1252 ?? Ss 12Nov09 4:32.81 natd -n fxp1
Here I rather use -a ${ext_ip} but that probably doesn't matter.
> Here are the ipfw rules:
>
> 14:55:41 /root# ipfw show
> 00100 949306 56746770 allow ip from any to any via lo0
> 00200 0 0 deny ip from any to 127.0.0.0/8
> 00300 0 0 deny ip from 127.0.0.0/8 to any
> 00400 77293 8699526 divert 8668 ip from any to any via fxp1
> 00500 35245946 28535731864 allow ip from any to any
> 65535 0 0 deny ip from any to any
Try, where ext_if=fxp1, int_if=$your_internal_if and ext_ip=$yours
ipfw add 400 divert natd ip from any to any out recv $int_if xmit $ext_if
ipfw add 410 divert natd ip from any to $ext_ip in recv $ext_if
Apart from not passing natd undivertable packets, use of 'via' here has
natd being called at least once and maybe twice on each packet coming or
going on the outside interface, including those from the host itself.
> Downloading, scp has no trouble:
>
> 14:55:59 /root# scp -p remote:public_html/video/tatra1.mpg .
> tatra1.mpg 100% 85MB 559.4KB/s 02:36
>
> But uploads stall. This scp process was killed after about 60 seconds:
Might there be an MTU issue as well? Anything in /etc/natd.conf?
Despite that the above divert rules will prevent outbound host traffic
being diverted at all, I'm still surprised natd's impact was so severe?
> 14:58:40 /root# scp -p tatra1.mpg remote:/tmp/
> tatra1.mpg 0% 320KB 1.8KB/s - stalled -
> ^CKilled by signal 2.
>
> Deleting the DIVERT rule eliminates the stalling:
>
> 14:59:54 /root# ipfw delete 400
> 15:00:04 /root# scp -p tatra1.mpg remote:/tmp/
> tatra1.mpg 27% 23MB 248.2KB/s 04:14 ETA
> ^CKilled by signal 2.
>
> But of course, it also eliminates NAT.
>
> 15:01:14 /root# ipfw add 400 divert 8668 ip from any to any via fxp1
> 00400 divert 8668 ip from any to any via fxp1
>
> Adding this rule works around the natd throughput problem:
>
> 15:01:29 /root# ipfw add 350 allow all from me to any via fxp1
> 00350 allow ip from me to any via fxp1
>
> 15:02:03 /root# scp -p tatra1.mpg remote:/tmp/
> tatra1.mpg 100% 85MB 266.9KB/s 05:27
350 has same effect as putting the selective requirements on outbound
divert. You still need to check inbound packets for possible NAT'ing.
cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091124144854.H61869>