Date: Thu, 9 Aug 2001 15:19:42 -0600 From: Nate Williams <nate@yogotech.com> To: stable@FreeBSD.org Subject: Strange I/O behavior Message-ID: <15218.65134.426946.295368@nomad.yogotech.com>
next in thread | raw e-mail | index | archive | help
I've got a dual-CPU box that's acting very strangely. At various times
throughout the day, the box starts to crawl.
In analyzing it, the only thing that jumps out at me is that the first
fixed disk is running a huge number of I/O transactions, to the point
that it's almost saturating the disk.
However, the amount transferred is almost nil, but since this disk
contains / and /usr (which means all the files are on it), interactive
performance goes to the dump during these sessions.
The box is an all SCSI system, with the first disk being a 17GB
Seagate. The dmesg is below.
Here's the output of iostat 2, which shows alot of transactions going
on. However, there are no active users doing anything as far as I can
see, and top shows the system as being mostly idle.
Any ideas how I can determine which application(s) are causing the huge
I/O loads?
Nate
turret:~ % iostat 2
tty da0 da1 cd0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 14 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 1 0 99
0 19 7.82 111 0.85 0.00 0 0.00 0.00 0 0.00 0 0 4 1 95
0 19 7.92 58 0.45 0.00 0 0.00 0.00 0 0.00 0 0 0 1 99
0 19 7.83 103 0.79 0.00 0 0.00 0.00 0 0.00 0 0 1 0 98
0 19 8.20 108 0.86 0.00 0 0.00 0.00 0 0.00 0 0 2 2 96
0 19 7.63 165 1.23 0.00 0 0.00 0.00 0 0.00 0 0 2 2 97
0 19 6.85 90 0.60 0.00 0 0.00 0.00 0 0.00 0 0 3 1 96
0 19 12.06 75 0.88 0.00 0 0.00 0.00 0 0.00 0 0 4 2 93
0 19 5.85 23 0.13 0.00 0 0.00 0.00 0 0.00 0 0 1 1 99
0 19 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0100
0 19 8.12 129 1.02 0.00 0 0.00 0.00 0 0.00 0 0 1 0 98
0 19 10.15 33 0.33 0.00 0 0.00 0.00 0 0.00 0 0 1 1 98
0 19 11.28 40 0.44 0.00 0 0.00 0.00 0 0.00 0 0 2 1 97
0 19 7.11 69 0.48 0.00 0 0.00 0.00 0 0.00 0 0 3 1 97
0 19 10.89 116 1.24 0.00 0 0.00 0.00 0 0.00 0 0 1 0 98
0 19 8.84 22 0.19 0.00 0 0.00 0.00 0 0.00 0 0 2 1 97
0 19 8.18 83 0.66 0.00 0 0.00 0.00 0 0.00 0 0 1 1 98
0 19 7.76 14 0.11 0.00 0 0.00 0.00 0 0.00 0 0 2 0 98
0 19 7.95 37 0.29 0.00 0 0.00 0.00 0 0.00 0 0 1 0 99
0 19 8.24 20 0.16 0.00 0 0.00 0.00 0 0.00 0 0 1 1 98
Copyright (c) 1992-2001 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.3-RELEASE #1: Mon Jul 23 14:52:32 PDT 2001
nate@turret.americas.nokia.com:/usr/src/sys/compile/TURRET
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium III/Pentium III Xeon/Celeron (601.37-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x673 Stepping = 3
Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory = 268423168 (262132K bytes)
avail memory = 258195456 (252144K bytes)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
IOAPIC #0 intpin 16 -> irq 11
IOAPIC #0 intpin 17 -> irq 10
IOAPIC #0 intpin 18 -> irq 12
IOAPIC #0 intpin 19 -> irq 5
FreeBSD/SMP: Multiprocessor motherboard
cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000
cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000
io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000
Preloaded elf kernel "kernel" at 0xc0321000.
Pentium Pro MTRR support enabled
md0: Malloc disk
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pci1: <Matrox MGA G200 AGP graphics accelerator> at 0.0 irq 11
isab0: <Intel 82371AB PCI to ISA bridge> at device 4.0 on pci0
isa0: <ISA bus> on isab0
pci0: <Intel PIIX4 ATA controller> at 4.1
pci0: <Intel 82371AB/EB (PIIX4) USB controller> at 4.2 irq 5
Timecounter "PIIX" frequency 3579545 Hz
chip1: <Intel 82371AB Power management controller> port 0xe800-0xe80f at device
4.3 on pci0
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xd000-0xd03f mem 0xe0800000-0xe081ffff,0xe1000000-0xe1000fff irq 5 at device 9.0 on pci0
fxp0: Ethernet address 00:02:b3:36:b1:93
fxp1: <Intel Pro 10/100B/100+ Ethernet> port 0xb800-0xb83f mem 0xdf800000-0xdf8fffff,0xe0000000-0xe0000fff irq 12 at device 10.0 on pci0
fxp1: Ethernet address 00:90:27:a5:68:11
ahc0: <Adaptec 2940 Ultra2 SCSI adapter> port 0xb400-0xb4ff mem 0xdf000000-0xdf0
00fff irq 10 at device 11.0 on pci0
aic7890/91: Wide Channel A, SCSI Id=7, 32/255 SCBs
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: routing 8254 via IOAPIC #0 intpin 2
Waiting 2 seconds for SCSI devices to settle
SMP: AP CPU #1 Launched!
sa0 at ahc0 bus 0 target 4 lun 0
sa0: <Quantum DLT4000 CD50> Removable Sequential Access SCSI-2 device
sa0: 10.000MB/s transfers (10.000MHz, offset 15)
da0 at ahc0 bus 0 target 0 lun 0
da0: <SEAGATE ST318275LW 0001> Fixed Direct Access SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 17366MB (35566480 512 byte sectors: 255H 63S/T 2213C)
Mounting root from ufs:/dev/da0s1a
ch0 at ahc0 bus 0 target 6 lun 0
ch0: <OVERLAND LXB 0409> Removable Changer SCSI-2 device
ch0: 10.000MB/s transfers (10.000MHz, offset 15)
ch0: 10 slots, 1 drive, 1 picker, 0 portals
cd0 at ahc0 bus 0 target 1 lun 0
cd0: <TOSHIBA CD-ROM XM-6401TA 1015> Removable CD-ROM SCSI-2 device
cd0: 20.000MB/s transfers (20.000MHz, offset 16)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da1 at ahc0 bus 0 target 3 lun 0
da1: <WDIGTL WDE18310 ULTRA2 1.30> Fixed Direct Access SCSI-2 device
da1: 40.000MB/s transfers (20.000MHz, offset 31, 16bit)
da1: 17461MB (35761710 512 byte sectors: 255H 63S/T 2226C)
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15218.65134.426946.295368>