Date: Wed, 11 Mar 2015 15:32:08 +0000 From: Malcolm Matalka <mmatalka@gmail.com> To: Atom Powers <atom.powers@gmail.com> Cc: freebsd-performance@freebsd.org Subject: Re: NAT under performing direct connection by 10x Message-ID: <874mprim87.fsf@gmail.com> In-Reply-To: <CAF-H=OnMTAphHKd62BdtXYntSMaEdHDsCLZw0n74ogF-i9uJMw@mail.gmail.com> (Atom Powers's message of "Wed, 11 Mar 2015 08:20:28 -0700") References: <87fv9bin7t.fsf@gmail.com> <CAF-H=OnMTAphHKd62BdtXYntSMaEdHDsCLZw0n74ogF-i9uJMw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Atom Powers <atom.powers@gmail.com> writes: > What are you using to implement NAT? ipf? iptables? Are you Whoops, apologies, I knew I forgot something. I'm using ipfw. My rules are: # ipfw list 00100 nat 1 ip from any to any via ue0 09999 allow ip from any to any 65535 deny ip from any to any > reassembling packets or passing them straight through? Are your memory > buffers big enough? I have gateway_enable="YES". I'm not sure if that answers your question, if it doesn't please let me know what I should look at in order to answer your question. Here is the output of netstat -m while doing a transfer: $ netstat -m 12/498/510 mbufs in use (current/cache/total) 1/255/256/126174 mbuf clusters in use (current/cache/total/max) 1/252 mbuf+clusters out of packet secondary zone in use (current/cache) 1/47/48/4925 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1459 9k jumbo clusters in use (current/cache/total/max) 0/0/0/820 16k jumbo clusters in use (current/cache/total/max) 9K/822K/831K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/4272 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile While playing around, I also have add the following to /boot/loader.conf: kern.ipc.nmbclusters=126174 And to /etc/sysctl.conf: kern.ipc.nmbjumbop=63087 Neither had any effect. These values were taken from another FreeBSD machine to see if perhaps the defaults for the RPi were too low. > > Who is your Internet provider and do you see those same rates for > repeated tests? (Some provider will give you an early burst of speed > and then throttle you back.) I have done the tests multiple times over multiple files and jumping between going through the RPi and bypassing it, all with the result that my performance from my ISP is great except when I go through the RPi. > > The window size could be significant. What happens if you manually set > the window size to different values? I'm afraid I don't know how to do this, how do I? I have tired turning window scaling off with sysctl -w net.inet.tcp.rfc1323=0, however that had no effect. > > > > > On Wed, Mar 11, 2015 at 8:10 AM, Malcolm Matalka <mmatalka@gmail.com> wrote: >> I am setting up my Raspberry Pi Model B+ as a router and am experience >> a throughput bottleneck that I cannot track down. >> >> I'd like to preface my question with two things: >> >> - I'm new to this so it is quite possible this is something obvious and >> I simply lack the knowledge to see it. If so, sorry for wasting >> anyones time and thank you very much to whoever points it out to me. >> >> - It is quite possible that my Raspberry Pi does not have the hardware >> to support the desired throughput, but I'm having trouble finding >> evidence to back up that hypothesis. >> >> The setup I have is: >> >> Laptop -> (wlan0 - RPi - ue0) -> Router >> >> I am using a USB dongle with a 'run0' interface running as a hostap. >> The RPi is then connected to a router which is connected to the >> internet. >> >> I have run the following command in different experiments: >> >> wget -O /dev/null >> ftp://ftp.freebsd.org/pub/FreeBSD/releases/VM-IMAGES/10.1-RELEASE/amd64/Latest/FreeBSD-10.1-RELEASE-amd64.raw.xz >> >> On the RPi: download speed of around 500 KB/s >> >> On the laptop: download speed of around 80 KB/s >> >> I have also scp'd a file from the laptop to another computer on the other >> side of the router: >> >> Laptop -> RPi -> Router -> Computer >> >> And I have a throughput of around 1 MB/s. This exercises both network >> interfaces similar to downloading from the internet. >> >> The only difference I am aware of is that NAT is happening in the >> Internet case. >> >> I have: >> >> - Checked CPU, in the NAT case the load average does not break 0.2. The >> CPU does not seem stressed. >> >> - Interrupt %, in top, doesn't go above a peak of 4% in either case. >> >> - netstat -i and netstat -s do not show any errors happening (that I can >> see). >> >> - While I have no idea if it is significant, I have noticed that in the >> case of a local download (on the RPi), in tcpdump the "win" goes down >> to around 1000. In the NAT case it stays at around 4096. >> >> Currently I am at a loss to explain the bottleneck. Is there a >> suggested next place to explore to track this down? As I said, it could >> be that the RPi is not capable of these throughputs, I'm using this as >> an excuse to learn how to investigate a performance bottleneck in >> FreeBSD though so I'm more interested in the meta-discussion of how to >> determine where the problem is. >> >> Thank you, >> /Malcolm >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?874mprim87.fsf>