From owner-freebsd-hackers Thu Aug 30 20:46:33 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from loki.datamatrix.com (loki.datamatrix.com [63.163.67.137]) by hub.freebsd.org (Postfix) with ESMTP id 225DA37B401 for ; Thu, 30 Aug 2001 20:46:19 -0700 (PDT) (envelope-from hansc@techserve.datamatrix.com) Received: from techserve.datamatrix.com (techserve.datamatrix.com [63.163.67.141]) by loki.datamatrix.com (8.11.3/8.9.3) with ESMTP id f7V3geZ18481 for ; Thu, 30 Aug 2001 20:42:40 -0700 (PDT) (envelope-from hansc@techserve.datamatrix.com) Received: from fernis (sdsl-66-80-62-82.dsl.sca.megapath.net [66.80.62.82]) by techserve.datamatrix.com (8.11.3/8.11.1) with SMTP id f7V3gch32596 for ; Thu, 30 Aug 2001 20:42:38 -0700 (PDT) (envelope-from hansc@techserve.datamatrix.com) Message-ID: <006c01c131cf$1ea67020$523e5042@datamatrix.com> From: "Hans Christensen" To: Subject: SLOW ftp transfers one way Date: Thu, 30 Aug 2001 20:43:38 -0700 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0069_01C13194.720D9C60" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_0069_01C13194.720D9C60 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have recently redefined a problem which has been plaguing me for = close to a year now. I have several FBSD boxes at a site fed by a Sprint T1 = (Site A). Each of these boxes is capable of ftp'ing to each other on the same subnet at speeds approaching the limits of the disk subsystem. In short, transfers on the LAN between FBSD boxen appear to be fine. In addition, = I have enlisted the help of the folks at sprint to ftp in and out of these boxes with speeds approaching the limits of the T1 line - no problem = there. It should be noted that the sprint guys have done their transfers from within sprint's network and are therefore NOT crossing their own network access points. Here is where it gets weird. If I ftp into one of my boxes at Site A across the WAN (in this case from a colocation facility) and put a large file onto my server in Site A, I get speeds of about 10KB/s. This may fluctuate from 4KB/s to 16KB/s, but it far below what one would normally = see across a T1 line. Interestingly enough, sending ftp traffic out of Site = A seems to move five to ten time faster - not perfect, but workable. Below = are example of the same file transferred first out of Site A to the = colocation facility, and then the same file just transferred, back into Site A. You will note the difference in speeds... This colocation facility is NOT on = the same network as sprint and therefore DOES have to cross one of Sprint's network access points. Furthermore, to rule out the possibility that the colo facility is to blame, I ftp'ed from a linux box on yet another = ISP's network. This linux box had the same type of performance problems. Slow = puts to Site A and reasonable gets from Site A. I have seen this before as well, between boxes at the colocation facility and again across different class c subnets. Sprint claims that = the problem lies with the MTU settings of the boxes at the "linux side" and = the "colo side." This smells wrong to me, but I confess that I don't really = know that it is wrong. I have looked in the FBSD bug reports for any = indication of a similar problem and do not see any so far, but I have seen several questions on the mailing list archives. Most of these are dismissed as improper configuration of ethernet cards. I have tried these suggestions = but found no relief. I ftp close to a GB of info every night into Site A and I need it to = go faster than it has been going, but I'm stumped. Anybody got any clues = for the clueless? Hans Christensen hansc@datamatrix.com Remote system type is UNIX. Using binary mode to transfer files. ftp> put jdk-1_2_2_006-win.exe local: jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe 227 Entering Passive Mode (************). 150 Opening BINARY mode data connection for jdk-1_2_2_006-win.exe 100% |************************************************************************= *** ***************************| 298 KB 00:00 ETA 226 Transfer complete. 305152 bytes sent in 3.66 seconds (81.33 KB/s) ftp> get jdk-1_2_2_006-win.exe local: jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe 227 Entering Passive Mode (*************). 150 Opening BINARY mode data connection for jdk-1_2_2_006-win.exe = (305152 bytes). 100% |************************************************************************= *** ***************************| 298 KB 00:00 ETA 226 Transfer complete. 305152 bytes received in 25.77 seconds (11.56 KB/s) ftp> Here is a dmesg: Copyright (c) 1992-2001 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD 4.3-STABLE #0: Fri Jun 1 06:59:28 PDT 2001 Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (501.14-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0x673 Stepping =3D 3 Features=3D0x387f9ff real memory =3D 201261056 (196544K bytes) avail memory =3D 192282624 (187776K bytes) Preloaded elf kernel "kernel" at 0xc035e000. VESA: v2.0, 8192k memory, flags:0x0, mode table:0xc02fd882 (1000022) VESA: ATI MACH64 Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pcib1: at device 1.0 on = pci0 pci1: on pcib1 pci1: at 0.0 irq 11 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0xf000-0xf00f at device 7.1 = on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 pci0: at 7.2 chip1: port 0x5000-0x500f at device 7.3 on pci0 fxp0: port 0xe400-0xe43f mem 0xeb000000-0xeb0fffff,0xeb202000-0xeb202fff irq 10 at device 9.0 on pci0 fxp0: Ethernet address 00:90:27:9a:47:1e inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp1: port 0xe800-0xe83f mem 0xeb100000-0xeb1fffff,0xeb200000-0xeb200fff irq 5 at device 10.0 on pci0 fxp1: Ethernet address 00:90:27:9a:47:10 inphy1: on miibus1 inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto ahc0: port 0xec00-0xecff mem 0xeb201000-0xeb201fff irq 11 at device 12.0 on pci0 aic7880: Wide Channel A, SCSI Id=3D7, 16/255 SCBs fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on = isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 psm0: failed to get data. psm0: irq 12 on atkbdc0 psm0: model Generic PS/2 mouse, device ID 0 vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 IP packet filtering initialized, divert disabled, rule-based forwarding enabled, default to accept, logging disabled ad0: 9787MB [19885/16/63] at ata0-master UDMA33 ad2: 19546MB [39714/16/63] at ata1-master UDMA33 acd0: CD-RW at ata0-slave using PIO4 Waiting 5 seconds for SCSI devices to settle no devsw da0 at ahc0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged Queueing Enabled da0: 105010MB (215061120 512 byte sectors: 255H 63S/T 13386C) (majdev=3D0 bootdev=3D0xa0200000) Mounting root from ufs:/dev/ad0s1 ------=_NextPart_000_0069_01C13194.720D9C60 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
    I have recently redefined a problem which = has been=20 plaguing me for close
to a year now. I have several FBSD boxes at a = site fed=20 by a Sprint T1 (Site
A). Each of these boxes is capable of ftp'ing to = each=20 other on the same
subnet at speeds approaching the limits of the disk = subsystem. In short,
transfers on the LAN between FBSD boxen appear = to be=20 fine. In addition, I
have enlisted the help of the folks at sprint to = ftp in=20 and out of these
boxes with speeds approaching the limits of the T1 = line - no=20 problem there.
It should be noted that the sprint guys have done = their=20 transfers from
within sprint's network and are therefore NOT crossing = their=20 own network
access points.
    Here is where it = gets weird.=20 If I ftp into one of my boxes at Site A
across the WAN (in this case = from a=20 colocation facility) and put a large
file onto my server in Site A, I = get=20 speeds of about 10KB/s. This may
fluctuate from 4KB/s to 16KB/s, but = it far=20 below what one would normally see
across a T1 line. Interestingly = enough,=20 sending ftp traffic out of Site A
seems to move five to ten time = faster - not=20 perfect, but workable. Below are
example of the same file transferred = first=20 out of Site A to the colocation
facility, and then the same file just = transferred, back into Site A. You
will note the difference in = speeds... This=20 colocation facility is NOT on the
same network as sprint and = therefore DOES=20 have to cross one of Sprint's
network access points. Furthermore, to = rule out=20 the possibility that the
colo facility is to blame, I ftp'ed from a = linux box=20 on yet another ISP's
network. This linux box had the same type of = performance=20 problems. Slow puts
to Site A and reasonable gets from Site=20 A.
    I have seen this before as well, between boxes = at the=20 colocation
facility and again across different class c subnets. = Sprint claims=20 that the
problem lies with the MTU settings of the boxes at the = "linux side"=20 and the
"colo side." This smells wrong to me, but I confess that I = don't=20 really know
that it is wrong. I have looked in the FBSD bug reports = for any=20 indication
of a similar problem and do not see any so far, but I have = seen=20 several
questions on the mailing list archives. Most of these are = dismissed=20 as
improper configuration of ethernet cards. I have tried these = suggestions=20 but
found no relief.
    I ftp close to a GB of = info every=20 night into Site A and I need it to go
faster than it has been going, = but I'm=20 stumped. Anybody got any clues for
the clueless?

Hans=20 Christensen
hansc@datamatrix.com

Remote system type is UNIX.
Using = binary mode=20 to transfer files.
ftp> put jdk-1_2_2_006-win.exe
local:=20 jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe
227 Entering = Passive Mode=20 (************).
150 Opening BINARY mode data connection for=20 jdk-1_2_2_006-win.exe
100%
|***************************************= ************************************
***************************| = ; =20 298 KB    00:00 ETA
226 Transfer complete.
305152 = bytes=20 sent in 3.66 seconds (81.33 KB/s)
ftp> get = jdk-1_2_2_006-win.exe
local:=20 jdk-1_2_2_006-win.exe remote: jdk-1_2_2_006-win.exe
227 Entering = Passive Mode=20 (*************).
150 Opening BINARY mode data connection for=20 jdk-1_2_2_006-win.exe=20 (305152
bytes).
100%
|******************************************= *********************************
***************************| &n= bsp;=20 298 KB    00:00 ETA
226 Transfer complete.
305152 = bytes=20 received in 25.77 seconds (11.56 KB/s)
ftp>


 Here = is a=20 dmesg:

Copyright (c) 1992-2001 The FreeBSD Project.
Copyright = (c)=20 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,=20 1994
        The Regents of the = University=20 of California. All rights reserved.
FreeBSD 4.3-STABLE #0: Fri = Jun  1=20 06:59:28 PDT 2001
Timecounter "i8254"  frequency 1193182 = Hz
CPU:=20 Pentium III/Pentium III Xeon/Celeron (501.14-MHz 686-class = CPU)
  Origin=20 =3D "GenuineIntel"  Id =3D 0x673  Stepping =3D=20 3

Features=3D0x387f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,M= TRR,PGE,MCA,CMOV,
PAT,PSE36,PN,MMX,FXSR,SSE>
real=20 memory  =3D 201261056 (196544K bytes)
avail memory =3D 192282624 = (187776K=20 bytes)
Preloaded elf kernel "kernel" at 0xc035e000.
VESA: v2.0, = 8192k=20 memory, flags:0x0, mode table:0xc02fd882 (1000022)
VESA: ATI=20 MACH64
Pentium Pro MTRR support enabled
md0: Malloc disk
npx0: = <math=20 processor> on motherboard
npx0: INT 16 interface
pcib0: = <Intel=20 82443BX (440 BX) host to PCI bridge> on motherboard
pci0: <PCI = bus>=20 on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> = at device=20 1.0 on pci0
pci1: <PCI bus> on pcib1
pci1: <ATI Mach64-GB = graphics accelerator> at 0.0 irq 11
isab0: <Intel 82371AB PCI = to ISA=20 bridge> at device 7.0 on pci0
isa0: <ISA bus> on = isab0
atapci0:=20 <Intel PIIX4 ATA33 controller> port 0xf000-0xf00f at device 7.1=20 on
pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 = on=20 atapci0
pci0: <Intel 82371AB/EB (PIIX4) USB controller> at=20 7.2
chip1: <Intel 82371AB Power management controller> port=20 0x5000-0x500f at
device 7.3 on pci0
fxp0: <Intel Pro = 10/100B/100+=20 Ethernet> port 0xe400-0xe43f=20 mem
0xeb000000-0xeb0fffff,0xeb202000-0xeb202fff irq 10 at device 9.0 = on=20 pci0
fxp0: Ethernet address 00:90:27:9a:47:1e
inphy0: <i82555 = 10/100=20 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, = 100baseTX,=20 100baseTX-FDX, auto
fxp1: <Intel Pro 10/100B/100+ Ethernet> = port=20 0xe800-0xe83f mem
0xeb100000-0xeb1fffff,0xeb200000-0xeb200fff irq 5 = at device=20 10.0 on pci0
fxp1: Ethernet address 00:90:27:9a:47:10
inphy1: = <i82555=20 10/100 media interface> on miibus1
inphy1:  10baseT, = 10baseT-FDX,=20 100baseTX, 100baseTX-FDX, auto
ahc0: <Adaptec 2940 Ultra SCSI = adapter>=20 port 0xec00-0xecff mem
0xeb201000-0xeb201fff irq 11 at device 12.0 on = pci0
aic7880: Wide Channel A, SCSI Id=3D7, 16/255 SCBs
fdc0: = <NEC 72065B=20 or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
atkbdc0:=20 <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: = <AT=20 Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
psm0: = failed to get=20 data.
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model = Generic PS/2=20 mouse, device ID 0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df = iomem=20 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on = isa0
sc0: VGA <16 virtual consoles, flags=3D0x300>
sio0 at = port=20 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at = port=20 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel = port> at=20 port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (NIBBLE-only) in = COMPATIBLE mode
lpt0: <Printer> on ppbus0
lpt0: = Interrupt-driven=20 port
ppi0: <Parallel I/O> on ppbus0
IP packet filtering = initialized,=20 divert disabled, rule-based forwarding
enabled, default to accept, = logging=20 disabled
ad0: 9787MB <WDC WD102AA> [19885/16/63] at ata0-master = UDMA33
ad2: 19546MB <FUJITSU MPF3204AT> [39714/16/63] at = ata1-master=20 UDMA33
acd0: CD-RW <MATSHITA CD-RW CW-7586> at ata0-slave using = PIO4
Waiting 5 seconds for SCSI devices to settle
no devsw da0 at = ahc0 bus=20 0 target 0 lun 0
da0: <RAID INC COBRA-2 0223> Fixed Direct = Access=20 SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), = Tagged=20 Queueing
Enabled
da0: 105010MB (215061120 512 byte sectors: 255H = 63S/T=20 13386C)
(majdev=3D0 bootdev=3D0xa0200000)
Mounting root from=20 ufs:/dev/ad0s1
------=_NextPart_000_0069_01C13194.720D9C60-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message