Date: Sat, 18 Sep 2004 15:54:08 +0200 From: "Raphael H. Becker" <rabe@p-i-n.com> To: Andre Oppermann <andre@freebsd.org> Cc: freebsd-current@freebsd.org Subject: Re: Strange things on GBit / 1000->100 / net.inet.tcp.inflight.* Message-ID: <20040918155408.P55054@p-i-n.com> In-Reply-To: <414B8C1B.2E7C971C@freebsd.org>; from andre@freebsd.org on Sat, Sep 18, 2004 at 03:15:07AM %2B0200 References: <20040917104356.E55054@p-i-n.com> <414ADD15.FAC42CDB@freebsd.org> <20040917231922.G55054@p-i-n.com> <414B567C.9060904@freebsd.org> <414B5777.1030901@freebsd.org> <20040918000303.J55054@p-i-n.com> <414B6F11.5070902@freebsd.org> <20040918013929.O55054@p-i-n.com> <414B8C1B.2E7C971C@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Sep 18, 2004 at 03:15:07AM +0200, Andre Oppermann wrote: > > I'm not sure this is a "physical" problem. > > The high number of out-of-order packets on your 4.10 machine is very > suspicious. You're talking about http://rhb.uugrn.org/FreeBSD/bugs/5.x/1000-to-100/check_3_target.txt ? Maybe the switch is buggy for just this testcase? Store&forward-Engine? > > Workaround: > > * Buy 1000MBit-Cards for the both "Router"-boxes > Why that? Do you have to do TCP transfer to/from these machines? The two machies are routers, but ... > Yet another test you could try: > 5.3 GE ---> FE 4.10 "Router" FE ---> 4.10 FE This was my initial testcase, I tried to transfer data from the GE-net via the "internal" router to another internal FE-4.x-Box --> 700kb/sec IIRC (Intranet) That failed. My first suspicion was the internal router (10.101.240.1, 4.10, FE) is not able to route a higher bandwidth for any reason. I tried something via the router. To mask some other unknown bottlenecks I tried direkt from the router (as target, not as router) ... now here we are. >From my point of view ther's no significant difference between "router just routes" or "router is target". Maybe I'm wrong here. Just tested the case via the 2nd router to another net. wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null Länge: 1,160,773,632 (unmaßgeblich) 2% [=> ] 28,918,448 582.63K/s ETA 31:37^C 1 145.2xx.xxx.1x6 0.243 ms 0.120 ms 0.133 ms 2 10.101.240.54 0.357 ms 0.229 ms 0.343 ms Same switch, different irons (target, router, server), another nets, ... Maybe interesting: wget starts with about 1-2MBytes/sec then converging down to some 100kBytes/sec. Another comparison, differrent targets (4.10 FE, 5.3GE), different servers (4.x,5.3 GE), see PS for a detailed list: 1) target: external router (10.101.240.254, 4.10, FE) $ wget -O - --proxy=off ftp://10.101.240.52/pub/1GB > /dev/null --> 9.42M/s $ wget -O - --proxy=off ftp://10.101.240.1/pub/250M > /dev/null --> 10.22M/s $ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 258.47K/s $ wget -O - --proxy=off ftp://10.101.240.55/pub/1GB > /dev/null --> 384.70K/s $ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 195.71K/s 2) target: neighbor-server (10.101.240.55, 5.3,GE) $ wget -O - --proxy=off ftp://10.101.240.52/pub/1GB > /dev/null --> 55.55M/s (uncached, the .52 cannot cache 1GByte file) $ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 34.12M/s (uncached from RAID) $ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 76.96M/s (cached) $ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 36.61M/s (uncached from RAID) $ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 77.40M/s (cached) $ wget -O - --proxy=off ftp://10.101.240.254/pub/250M > /dev/null --> 10.07M/s $ wget -O - --proxy=off ftp://10.101.240.1/pub/250M > /dev/null --> 10.37M/s I guess the switch has some troubles with its store&forward engine when transferring data from GE to FE and(!) the GE is a 5.3, if GE is 4.10 then it works. So the only significant difference is the OS (thats why I discuss it here and not with the guys of netgear this time). I have a spare "router-box", running a offline-hot-standby copy of our external packet filter (outside our external DMZ). I may install 5.3 on that box to have a 5.3 running on FE, even though I don't belive that will make a difference what kind of system is the target. Who knows. Regards Raphael Becker PS: all players on GE: IP OS IF desc .1 4.10 fxp1,FE internal router, PE350, 128MB RAM .254 4.10 fxp1,FE external router, PE350, 128MB RAM .52 4.10 bge1,GE normal host, PE2650, 1024MB RAM .54 5.3 bge0,GE dito, 2048MB RAM .55 5.3 bge0,GE dito, 2048MB RAM .56 5.3 bge0,GE dito, 2048MB RAM
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040918155408.P55054>