Date: Sat, 27 Oct 2007 07:51:13 -0700 (PDT) From: Abdullah Ibn Hamad Al-Marri <wearabnet@yahoo.ca> To: Matthew Reimer <mattjreimer@gmail.com> Cc: FreeBSD STABLE <freebsd-stable@freebsd.org> Subject: Re: Network throughput problems in RELENG_7 Message-ID: <619150.6364.qm@web33713.mail.mud.yahoo.com>
next in thread | raw e-mail | index | archive | help
----- Original Message ----=0A> From: Matthew Reimer <mattjreimer@gmail.com=
>=0A> To: current@freebsd.org=0A> Sent: Friday, October 26, 2007 7:10:55 PM=
=0A> Subject: Network throughput problems in RELENG_7=0A> =0A> I'm seeing a=
problem where a much faster quad-core host=0A> running=0A> =0A RELENG_7=0A=
> serves many fewer netrate/http requests per second (175/sec) than an=0A> =
old, busy, UP 6.0 host (828/sec). The problem seems to be related to=0A> la=
tency and connection setup, as it shows up dramatically over a link=0A> wit=
h 50-60 ms latency. Can you help?=0A> =0A> "Gandalf" is the problem RELENG_=
7 host connected via em0 on a 100baseTX=0A> full-duplex switch port.=0A> "B=
oromir" is a 6.0 host for reference, connected via bge0 to another=0A> 100b=
aseTX full-duplex port on the same switch.=0A> "Theoden" is a 6.2 host remo=
te (50-60 ms away) from gandalf=0A> and=0A> =0A boromir.=0A> =0A> Running t=
ools/netrate/http against tools/netrate/httpd on gandalf and=0A> boromir, s=
erving a document with 35 bytes of content, I get this=0A> (details includi=
ng tcpdumps following my sig):=0A> =0A> dest dest=0A> =
source boromir gandalf=0A> ------ ------- -------=0A> =
theoden 828 175=0A> boromir - 18092=0A> gandal=
f 11493 -=0A> =0A> The problem occurs using ab as well, but is much less =
pronounced=0A> when=0A> =0A -k=0A> is used (enabling keepalive and presumab=
ly HTTP pipelining). This makes=0A> me wonder if the problem might have som=
ething to do with connection=0A> setups.=0A> =0A> I've verified that the sw=
itch port and the interface are configured=0A> correctly (100baseTX full-du=
plex). I've also tried toggling polling and=0A> the following sysctls, with=
no significant change:=0A> =0A> net.inet.tcp.tso=0A> net.i=
net.tcp.newreno=0A> net.inet.tcp.sack.enable=0A> net.inet.t=
cp.inflight.enable=0A> net.inet.tcp.syncookies_only=0A> net=
.inet.tcp.syncookies=0A> =0A> What else can I try?=0A> =0A> Thanks for any =
help you can give.=0A> =0A> Matt=0A> =0A> =0A> =0A> =0A> theoden-gandalf=0A=
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A> =0A> theoden:~$ /usr/ob=
j/usr/src/tools/tools/netrate/http/http -s 2=0A> 216.193.240.4=0A> 1280 /sh=
ort.html=0A> 187 transfers/second=0A> 0 errors/second=0A> =0A> tcpdump:=0A>=
http://gandalf.vpop.net/~mreimer/theoden-gandalf-netrate.dump.gz=0A>; =0A> =
---=0A> =0A> theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -n 1=
00 -s 2=0A> 216.193.240.4 1280 /short.html=0A> 175 transfers/second=0A> 0 e=
rrors/second=0A> =0A> tcpdump:=0A> http://gandalf.vpop.net/~mreimer/theoden=
-gandalf-netrate-n100.dump.gz=0A> =0A> =0A> =0A> theoden-boromir=0A> =3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A> =0A> theoden:~$ /usr/obj/usr/sr=
c/tools/tools/netrate/http/http -s 2=0A> 216.193.240.2=0A> 1280 /short.html=
=0A> 272 transfers/second=0A> 0 errors/second=0A> =0A> tcpdump:=0A> http://=
boromir.vpop.net/~mreimer/theoden-boromir-netrate.dump.gz=0A> =0A> ---=0A> =
=0A> theoden:~$ /usr/obj/usr/src/tools/tools/netrate/http/http -n 100 -s 2=
=0A> 216.193.240.2 1280 /short.html=0A> 828 transfers/second=0A> 0 errors/s=
econd=0A> =0A> tcpdump:=0A> http://boromir.vpop.net/~mreimer/theoden-boromi=
r-netrate-n100.dump.gz=0A> =0A> =0A> boromir-gandalf=0A> =3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A> =0A> boromir: $ /usr/obj/usr/src/tools/tool=
s/netrate/http/http -n 100 -s 2=0A> 216.193.240.4 1280 /short.html=0A> 1809=
2 transfers/second=0A> 0 errors/second=0A> =0A> gandalf-boromir=0A> =3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A> =0A> gandalf: $ /usr/obj/usr/sr=
c/tools/tools/netrate/http/http -n 100 -s 2=0A> 216.193.240.2 1280 /short.h=
tml=0A> 11493 transfers/second=0A> 0 errors/second=0A> =0A> =0A> =0A> =0A> =
boromir sysctl and ifconfig:=0A> =0A> http://boromir.vpop.net/~mreimer/boro=
mir.sysctl.gz=0A> =0A> =0A> # ifconfig bge0=0A> bge0: flags=3D8843 mtu 1500=
=0A> options=3D1a=0A> inet6 fe80::2e0:81ff:fe60:3925%bge0 p=
refixlen 64 scopeid 0x1 =0A> inet 216.193.240.33 netmask 0xffffffff=
broadcast 216.193.240.33=0A> inet 216.193.240.34 netmask 0xfffffff=
f broadcast 216.193.240.34=0A> inet 216.193.240.35 netmask 0xffffff=
ff broadcast 216.193.240.35=0A> inet 216.193.240.36 netmask 0xfffff=
fff broadcast 216.193.240.36=0A> inet 216.193.240.37 netmask 0xffff=
ffff broadcast 216.193.240.37=0A> inet 216.193.240.17 netmask 0xfff=
fffff broadcast 216.193.240.17=0A> inet 216.193.240.38 netmask 0xff=
ffffff broadcast 216.193.240.38=0A> inet 216.193.240.39 netmask 0xf=
fffffff broadcast 216.193.240.39=0A> inet 216.193.240.2 netmask 0xf=
ffff800 broadcast 216.193.247.255=0A> inet 216.193.240.40 netmask 0=
xffffffff broadcast 216.193.240.40=0A> ether 00:e0:81:60:39:25=0A> =
media: Ethernet autoselect (100baseTX )=0A> status: active=
=0A> =0A> gandalf sysctl and ifconfig:=0A> http://gandalf.vpop.net/~mreimer=
/gandalf.sysctl.gz=0A> =0A> # ifconfig em0=0A> em0: flags=3D8843 metric 0 m=
tu=0A> 1500=0A> options=3D1cb=0A> ether 00:30:48:33:96:a4=
=0A> inet 216.193.240.4 netmask 0xfffff800 broadcast 216.193.247.25=
5=0A> inet 216.193.240.65 netmask 0xffffffff broadcast 216.193.240.=
65=0A> inet 216.193.240.66 netmask 0xffffffff broadcast 216.193.240=
.66=0A> inet 216.193.240.67 netmask 0xffffffff broadcast 216.193.24=
0.67=0A> inet 216.193.240.68 netmask 0xffffffff broadcast 216.193.2=
40.68=0A> media: Ethernet 100baseTX =0A> status: active=0A>=
=0A=0AHello,=0A=0AWhen did you last time did you csup and buildworld?=0A=
=0AI saw some changes in tcp few days ago.=0A=0A =0ARegards, =0A-Abdullah I=
bn Hamad Al-Marri=0AArab Portal=0Ahttp://www.WeArab.Net/=0A=0A=0A=0A=0A=0A_=
_________________________________________________=0ADo You Yahoo!?=0ATired =
of spam? Yahoo! Mail has the best spam protection around =0Ahttp://mail.ya=
hoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?619150.6364.qm>