Date: Thu, 30 Aug 2001 20:43:38 -0700 From: "Hans Christensen" <hansc@techserve.datamatrix.com> To: <freebsd-hackers@freebsd.org> Subject: SLOW ftp transfers one way Message-ID: <006c01c131cf$1ea67020$523e5042@datamatrix.com>
next in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
------=_NextPart_000_0069_01C13194.720D9C60
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
I have recently redefined a problem which has been plaguing me for =
close
to a year now. I have several FBSD boxes at a site fed by a Sprint T1 =
(Site
A). Each of these boxes is capable of ftp'ing to each other on the same
subnet at speeds approaching the limits of the disk subsystem. In short,
transfers on the LAN between FBSD boxen appear to be fine. In addition, =
I
have enlisted the help of the folks at sprint to ftp in and out of these
boxes with speeds approaching the limits of the T1 line - no problem =
there.
It should be noted that the sprint guys have done their transfers from
within sprint's network and are therefore NOT crossing their own network
access points.
Here is where it gets weird. If I ftp into one of my boxes at Site A
across the WAN (in this case from a colocation facility) and put a large
file onto my server in Site A, I get speeds of about 10KB/s. This may
fluctuate from 4KB/s to 16KB/s, but it far below what one would normally =
see
across a T1 line. Interestingly enough, sending ftp traffic out of Site =
A
seems to move five to ten time faster - not perfect, but workable. Below =
are
example of the same file transferred first out of Site A to the =
colocation
facility, and then the same file just transferred, back into Site A. You
will note the difference in speeds... This colocation facility is NOT on =
the
same network as sprint and therefore DOES have to cross one of Sprint's
network access points. Furthermore, to rule out the possibility that the
colo facility is to blame, I ftp'ed from a linux box on yet another =
ISP's
network. This linux box had the same type of performance problems. Slow =
puts
to Site A and reasonable gets from Site A.
I have seen this before as well, between boxes at the colocation
facility and again across different class c subnets. Sprint claims that =
the
problem lies with the MTU settings of the boxes at the "linux side" and =
the
"colo side." This smells wrong to me, but I confess that I don't really =
know
that it is wrong. I have looked in the FBSD bug reports for any =
indication
of a similar problem and do not see any so far, but I have seen several
questions on the mailing list archives. Most of these are dismissed as
improper configuration of ethernet cards. I have tried these suggestions =
but
found no relief.
I ftp close to a GB of info every night into Site A and I need it to =
go
faster than it has been going, but I'm stumped. Anybody got any clues =
for
the clueless?
Hans Christensen
hansc@datamatrix.com
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> put jdk-1_2_2_006-win.exe
local: jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe
227 Entering Passive Mode (************).
150 Opening BINARY mode data connection for jdk-1_2_2_006-win.exe
100%
|************************************************************************=
***
***************************| 298 KB 00:00 ETA
226 Transfer complete.
305152 bytes sent in 3.66 seconds (81.33 KB/s)
ftp> get jdk-1_2_2_006-win.exe
local: jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe
227 Entering Passive Mode (*************).
150 Opening BINARY mode data connection for jdk-1_2_2_006-win.exe =
(305152
bytes).
100%
|************************************************************************=
***
***************************| 298 KB 00:00 ETA
226 Transfer complete.
305152 bytes received in 25.77 seconds (11.56 KB/s)
ftp>
Here is a dmesg:
Copyright (c) 1992-2001 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights =
reserved.
FreeBSD 4.3-STABLE #0: Fri Jun 1 06:59:28 PDT 2001
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium III/Pentium III Xeon/Celeron (501.14-MHz 686-class CPU)
Origin =3D "GenuineIntel" Id =3D 0x673 Stepping =3D 3
Features=3D0x387f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,=
CMOV,
PAT,PSE36,PN,MMX,FXSR,SSE>
real memory =3D 201261056 (196544K bytes)
avail memory =3D 192282624 (187776K bytes)
Preloaded elf kernel "kernel" at 0xc035e000.
VESA: v2.0, 8192k memory, flags:0x0, mode table:0xc02fd882 (1000022)
VESA: ATI MACH64
Pentium Pro MTRR support enabled
md0: Malloc disk
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on =
pci0
pci1: <PCI bus> on pcib1
pci1: <ATI Mach64-GB graphics accelerator> at 0.0 irq 11
isab0: <Intel 82371AB PCI to ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 ATA33 controller> port 0xf000-0xf00f at device 7.1 =
on
pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
pci0: <Intel 82371AB/EB (PIIX4) USB controller> at 7.2
chip1: <Intel 82371AB Power management controller> port 0x5000-0x500f at
device 7.3 on pci0
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xe400-0xe43f mem
0xeb000000-0xeb0fffff,0xeb202000-0xeb202fff irq 10 at device 9.0 on pci0
fxp0: Ethernet address 00:90:27:9a:47:1e
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: <Intel Pro 10/100B/100+ Ethernet> port 0xe800-0xe83f mem
0xeb100000-0xeb1fffff,0xeb200000-0xeb200fff irq 5 at device 10.0 on pci0
fxp1: Ethernet address 00:90:27:9a:47:10
inphy1: <i82555 10/100 media interface> on miibus1
inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ahc0: <Adaptec 2940 Ultra SCSI adapter> port 0xec00-0xecff mem
0xeb201000-0xeb201fff irq 11 at device 12.0 on pci0
aic7880: Wide Channel A, SCSI Id=3D7, 16/255 SCBs
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on =
isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
psm0: failed to get data.
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model Generic PS/2 mouse, device ID 0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on =
isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=3D0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
IP packet filtering initialized, divert disabled, rule-based forwarding
enabled, default to accept, logging disabled
ad0: 9787MB <WDC WD102AA> [19885/16/63] at ata0-master UDMA33
ad2: 19546MB <FUJITSU MPF3204AT> [39714/16/63] at ata1-master UDMA33
acd0: CD-RW <MATSHITA CD-RW CW-7586> at ata0-slave using PIO4
Waiting 5 seconds for SCSI devices to settle
no devsw da0 at ahc0 bus 0 target 0 lun 0
da0: <RAID INC COBRA-2 0223> Fixed Direct Access SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged Queueing
Enabled
da0: 105010MB (215061120 512 byte sectors: 255H 63S/T 13386C)
(majdev=3D0 bootdev=3D0xa0200000)
Mounting root from ufs:/dev/ad0s1
------=_NextPart_000_0069_01C13194.720D9C60
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 5.50.4207.2601" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2><FONT face=3D"Times New Roman"=20
size=3D3> I have recently redefined a problem which =
has been=20
plaguing me for close<BR>to a year now. I have several FBSD boxes at a =
site fed=20
by a Sprint T1 (Site<BR>A). Each of these boxes is capable of ftp'ing to =
each=20
other on the same<BR>subnet at speeds approaching the limits of the disk =
subsystem. In short,<BR>transfers on the LAN between FBSD boxen appear =
to be=20
fine. In addition, I<BR>have enlisted the help of the folks at sprint to =
ftp in=20
and out of these<BR>boxes with speeds approaching the limits of the T1 =
line - no=20
problem there.<BR>It should be noted that the sprint guys have done =
their=20
transfers from<BR>within sprint's network and are therefore NOT crossing =
their=20
own network<BR>access points.<BR> Here is where it =
gets weird.=20
If I ftp into one of my boxes at Site A<BR>across the WAN (in this case =
from a=20
colocation facility) and put a large<BR>file onto my server in Site A, I =
get=20
speeds of about 10KB/s. This may<BR>fluctuate from 4KB/s to 16KB/s, but =
it far=20
below what one would normally see<BR>across a T1 line. Interestingly =
enough,=20
sending ftp traffic out of Site A<BR>seems to move five to ten time =
faster - not=20
perfect, but workable. Below are<BR>example of the same file transferred =
first=20
out of Site A to the colocation<BR>facility, and then the same file just =
transferred, back into Site A. You<BR>will note the difference in =
speeds... This=20
colocation facility is NOT on the<BR>same network as sprint and =
therefore DOES=20
have to cross one of Sprint's<BR>network access points. Furthermore, to =
rule out=20
the possibility that the<BR>colo facility is to blame, I ftp'ed from a =
linux box=20
on yet another ISP's<BR>network. This linux box had the same type of =
performance=20
problems. Slow puts<BR>to Site A and reasonable gets from Site=20
A.<BR> I have seen this before as well, between boxes =
at the=20
colocation<BR>facility and again across different class c subnets. =
Sprint claims=20
that the<BR>problem lies with the MTU settings of the boxes at the =
"linux side"=20
and the<BR>"colo side." This smells wrong to me, but I confess that I =
don't=20
really know<BR>that it is wrong. I have looked in the FBSD bug reports =
for any=20
indication<BR>of a similar problem and do not see any so far, but I have =
seen=20
several<BR>questions on the mailing list archives. Most of these are =
dismissed=20
as<BR>improper configuration of ethernet cards. I have tried these =
suggestions=20
but<BR>found no relief.<BR> I ftp close to a GB of =
info every=20
night into Site A and I need it to go<BR>faster than it has been going, =
but I'm=20
stumped. Anybody got any clues for<BR>the clueless?<BR><BR>Hans=20
Christensen<BR></FONT><A href=3D"mailto:hansc@datamatrix.com"><FONT=20
face=3D"Times New Roman" =
size=3D3>hansc@datamatrix.com</FONT></A><BR><BR><FONT=20
face=3D"Times New Roman" size=3D3>Remote system type is UNIX.<BR>Using =
binary mode=20
to transfer files.<BR>ftp> put jdk-1_2_2_006-win.exe<BR>local:=20
jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe<BR>227 Entering =
Passive Mode=20
(************).<BR>150 Opening BINARY mode data connection for=20
jdk-1_2_2_006-win.exe<BR>100%<BR>|***************************************=
************************************<BR>***************************| =
; =20
298 KB 00:00 ETA<BR>226 Transfer complete.<BR>305152 =
bytes=20
sent in 3.66 seconds (81.33 KB/s)<BR>ftp> get =
jdk-1_2_2_006-win.exe<BR>local:=20
jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe<BR>227 Entering =
Passive Mode=20
(*************).<BR>150 Opening BINARY mode data connection for=20
jdk-1_2_2_006-win.exe=20
(305152<BR>bytes).<BR>100%<BR>|******************************************=
*********************************<BR>***************************| &n=
bsp;=20
298 KB 00:00 ETA<BR>226 Transfer complete.<BR>305152 =
bytes=20
received in 25.77 seconds (11.56 KB/s)<BR>ftp><BR><BR><BR> Here =
is a=20
dmesg:<BR><BR>Copyright (c) 1992-2001 The FreeBSD Project.<BR>Copyright =
(c)=20
1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,=20
1994<BR> The Regents of the =
University=20
of California. All rights reserved.<BR>FreeBSD 4.3-STABLE #0: Fri =
Jun 1=20
06:59:28 PDT 2001<BR>Timecounter "i8254" frequency 1193182 =
Hz<BR>CPU:=20
Pentium III/Pentium III Xeon/Celeron (501.14-MHz 686-class =
CPU)<BR> Origin=20
=3D "GenuineIntel" Id =3D 0x673 Stepping =3D=20
3<BR><BR>Features=3D0x387f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,M=
TRR,PGE,MCA,CMOV,<BR>PAT,PSE36,PN,MMX,FXSR,SSE><BR>real=20
memory =3D 201261056 (196544K bytes)<BR>avail memory =3D 192282624 =
(187776K=20
bytes)<BR>Preloaded elf kernel "kernel" at 0xc035e000.<BR>VESA: v2.0, =
8192k=20
memory, flags:0x0, mode table:0xc02fd882 (1000022)<BR>VESA: ATI=20
MACH64<BR>Pentium Pro MTRR support enabled<BR>md0: Malloc disk<BR>npx0: =
<math=20
processor> on motherboard<BR>npx0: INT 16 interface<BR>pcib0: =
<Intel=20
82443BX (440 BX) host to PCI bridge> on motherboard<BR>pci0: <PCI =
bus>=20
on pcib0<BR>pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> =
at device=20
1.0 on pci0<BR>pci1: <PCI bus> on pcib1<BR>pci1: <ATI Mach64-GB =
graphics accelerator> at 0.0 irq 11<BR>isab0: <Intel 82371AB PCI =
to ISA=20
bridge> at device 7.0 on pci0<BR>isa0: <ISA bus> on =
isab0<BR>atapci0:=20
<Intel PIIX4 ATA33 controller> port 0xf000-0xf00f at device 7.1=20
on<BR>pci0<BR>ata0: at 0x1f0 irq 14 on atapci0<BR>ata1: at 0x170 irq 15 =
on=20
atapci0<BR>pci0: <Intel 82371AB/EB (PIIX4) USB controller> at=20
7.2<BR>chip1: <Intel 82371AB Power management controller> port=20
0x5000-0x500f at<BR>device 7.3 on pci0<BR>fxp0: <Intel Pro =
10/100B/100+=20
Ethernet> port 0xe400-0xe43f=20
mem<BR>0xeb000000-0xeb0fffff,0xeb202000-0xeb202fff irq 10 at device 9.0 =
on=20
pci0<BR>fxp0: Ethernet address 00:90:27:9a:47:1e<BR>inphy0: <i82555 =
10/100=20
media interface> on miibus0<BR>inphy0: 10baseT, 10baseT-FDX, =
100baseTX,=20
100baseTX-FDX, auto<BR>fxp1: <Intel Pro 10/100B/100+ Ethernet> =
port=20
0xe800-0xe83f mem<BR>0xeb100000-0xeb1fffff,0xeb200000-0xeb200fff irq 5 =
at device=20
10.0 on pci0<BR>fxp1: Ethernet address 00:90:27:9a:47:10<BR>inphy1: =
<i82555=20
10/100 media interface> on miibus1<BR>inphy1: 10baseT, =
10baseT-FDX,=20
100baseTX, 100baseTX-FDX, auto<BR>ahc0: <Adaptec 2940 Ultra SCSI =
adapter>=20
port 0xec00-0xecff mem<BR>0xeb201000-0xeb201fff irq 11 at device 12.0 on =
pci0<BR>aic7880: Wide Channel A, SCSI Id=3D7, 16/255 SCBs<BR>fdc0: =
<NEC 72065B=20
or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0<BR>atkbdc0:=20
<Keyboard controller (i8042)> at port 0x60,0x64 on isa0<BR>atkbd0: =
<AT=20
Keyboard> flags 0x1 irq 1 on atkbdc0<BR>kbd0 at atkbd0<BR>psm0: =
failed to get=20
data.<BR>psm0: <PS/2 Mouse> irq 12 on atkbdc0<BR>psm0: model =
Generic PS/2=20
mouse, device ID 0<BR>vga0: <Generic ISA VGA> at port 0x3c0-0x3df =
iomem=20
0xa0000-0xbffff on isa0<BR>sc0: <System console> at flags 0x100 on =
isa0<BR>sc0: VGA <16 virtual consoles, flags=3D0x300><BR>sio0 at =
port=20
0x3f8-0x3ff irq 4 flags 0x10 on isa0<BR>sio0: type 16550A<BR>sio1 at =
port=20
0x2f8-0x2ff irq 3 on isa0<BR>sio1: type 16550A<BR>ppc0: <Parallel =
port> at=20
port 0x378-0x37f irq 7 on isa0<BR>ppc0: Generic chipset (NIBBLE-only) in =
COMPATIBLE mode<BR>lpt0: <Printer> on ppbus0<BR>lpt0: =
Interrupt-driven=20
port<BR>ppi0: <Parallel I/O> on ppbus0<BR>IP packet filtering =
initialized,=20
divert disabled, rule-based forwarding<BR>enabled, default to accept, =
logging=20
disabled<BR>ad0: 9787MB <WDC WD102AA> [19885/16/63] at ata0-master =
UDMA33<BR>ad2: 19546MB <FUJITSU MPF3204AT> [39714/16/63] at =
ata1-master=20
UDMA33<BR>acd0: CD-RW <MATSHITA CD-RW CW-7586> at ata0-slave using =
PIO4<BR>Waiting 5 seconds for SCSI devices to settle<BR>no devsw da0 at =
ahc0 bus=20
0 target 0 lun 0<BR>da0: <RAID INC COBRA-2 0223> Fixed Direct =
Access=20
SCSI-2 device<BR>da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), =
Tagged=20
Queueing<BR>Enabled<BR>da0: 105010MB (215061120 512 byte sectors: 255H =
63S/T=20
13386C)<BR>(majdev=3D0 bootdev=3D0xa0200000)<BR>Mounting root from=20
ufs:/dev/ad0s1</FONT><BR></FONT></DIV></BODY></HTML>
------=_NextPart_000_0069_01C13194.720D9C60--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?006c01c131cf$1ea67020$523e5042>