Date: Fri, 16 Jul 2004 07:38:51 +0200 From: "Daniel Eriksson" <daniel_k_eriksson@telia.com> To: <noackjr@alumni.rice.edu> Cc: freebsd-current@freebsd.org Subject: RE: consistent hard lockup with recent kernels Message-ID: <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAASwXyhQrhq0KvaGmo0/NC3gEAAAAA@telia.com> In-Reply-To: <40F748F3.6020702@alumni.rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
------=_NextPart_000_0000_01C46B07.F3CD7D10
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Jon Noack wrote:
> Yeah, it is most likely bugs uncovered by native preemption. For now=20
> you can remask then by commenting out the "#define PREEMPTION" in=20
> src/sys/<ARCH>/include/param.h and rebuilding your kernel. Robert=20
> Watson's comments on this:
> "This results in a quite usable kernel, although quite probably more=20
> poor interrupt handling latency, etc."
I just tried a new kernel (2004.07.16.01.00.00) with preemption turned =
on,
and my problems with interrupt storms and SATA still remains. With
preemption turned off the machine seems stable, but somewhat sluggish.
With preemption enabled I get interrupt storms and DMA problems with the
SATA devices like below when trying to access all discs at once (running
fsck on all 11 file systems simultaneously). ad20 is one of two SATA =
devices
in the system.
Interrupt storm detected on "irq20: atapci4"; throttling interrupt =
source
ad20: FAILURE - already active DMA on this device
ad20: setting up DMA failed
Attached is the dmesg.boot from the old kernel (2004.07.12.22.00.00) =
with
preemption turned off that seems to be stable.
I'm using HZ=3D2000, DEVICE_POLLING and ZERO_COPY_SOCKETS, plus device =
apic. I
also have net.isr enabled.
'vmstat -i' gives the following output:
interrupt total rate
irq1: atkbd0 248 0
irq0: clk 8176853 1993
irq4: sio0 12 0
irq6: fdc0 6 0
irq8: rtc 524902 127
irq13: npx0 1 0
irq14: ata0 33148 8
irq15: ata1 24641 6
irq16: atapci0+ 867398 211
irq17: atapci2+ 296877 72
irq18: re0 2 0
irq19: ahc0 94035 22
irq20: atapci4 184381 44
irq23: vr0 2 0
Total 10202506 2487
/Daniel Eriksson
------=_NextPart_000_0000_01C46B07.F3CD7D10
Content-Type: application/octet-stream;
name="dmesg.boot"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="dmesg.boot"
Copyright (c) 1992-2004 The FreeBSD Project.=0A=
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994=0A=
The Regents of the University of California. All rights reserved.=0A=
FreeBSD 5.2-CURRENT #0: Tue Jul 13 03:17:24 CEST 2004=0A=
daniel@xxx.xxx.xxx:/usr/obj/usr/src/sys/FORTIFY=0A=
WARNING: Kernel preemption is disabled, expect reduced performance.=0A=
Timecounter "i8254" frequency 1193182 Hz quality 0=0A=
CPU: AMD Athlon(TM) XP 2500+ (1999.78-MHz 686-class CPU)=0A=
Origin =3D "AuthenticAMD" Id =3D 0x6a0 Stepping =3D 0=0A=
=
Features=3D0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE=
,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>=0A=
AMD Features=3D0xc0400000<AMIE,DSP,3DNow!>=0A=
real memory =3D 1342156800 (1279 MB)=0A=
avail memory =3D 1304948736 (1244 MB)=0A=
ACPI APIC Table: <ASUS A7V600-X>=0A=
ioapic0: Changing APIC ID to 2=0A=
ioapic0 <Version 0.3> irqs 0-23 on motherboard=0A=
npx0: [FAST]=0A=
npx0: <math processor> on motherboard=0A=
npx0: INT 16 interface=0A=
acpi0: <ASUS A7V600-X> on motherboard=0A=
acpi0: [GIANT-LOCKED]=0A=
acpi0: Power Button (fixed)=0A=
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000=0A=
acpi_timer0: <32-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0=0A=
cpu0: <ACPI CPU> on acpi0=0A=
acpi_button0: <Power Button> on acpi0=0A=
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0=0A=
pci0: <ACPI PCI bus> on pcib0=0A=
agp0: <VIA 8377 (Apollo KT400/KT400A/KT600) host to PCI bridge> mem =
0xf8000000-0xfbffffff at device 0.0 on pci0=0A=
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0=0A=
pci1: <ACPI PCI bus> on pcib1=0A=
pci1: <display, VGA> at device 0.0 (no driver attached)=0A=
atapci0: <HighPoint HPT374 (channel 0+1) UDMA133 controller> port =
0xb400-0xb4ff,0xb800-0xb803,0xd000-0xd007,0xd400-0xd403,0xd800-0xd807 =
irq 16 at device 10.0 on pci0=0A=
ata2: at 0xd800 on atapci0=0A=
ata3: at 0xd000 on atapci0=0A=
atapci1: <HighPoint HPT374 (channel 2+3) UDMA133 controller> port =
0x9800-0x98ff,0xa000-0xa003,0xa400-0xa407,0xa800-0xa803,0xb000-0xb007 =
irq 16 at device 10.1 on pci0=0A=
ata4: at 0xb000 on atapci1=0A=
ata5: at 0xa400 on atapci1=0A=
ahc0: <Adaptec 29160 Ultra160 SCSI adapter> port 0x9400-0x94ff mem =
0xed800000-0xed800fff irq 19 at device 12.0 on pci0=0A=
ahc0: [GIANT-LOCKED]=0A=
aic7892: Ultra160 Wide Channel A, SCSI Id=3D7, 32/253 SCBs=0A=
atapci2: <HighPoint HPT374 (channel 0+1) UDMA133 controller> port =
0x7800-0x78ff,0x8000-0x8003,0x8400-0x8407,0x8800-0x8803,0x9000-0x9007 =
irq 17 at device 14.0 on pci0=0A=
ata6: at 0x9000 on atapci2=0A=
ata7: at 0x8400 on atapci2=0A=
atapci3: <HighPoint HPT374 (channel 2+3) UDMA133 controller> port =
0x6000-0x60ff,0x6400-0x6403,0x6800-0x6807,0x7000-0x7003,0x7400-0x7407 =
irq 17 at device 14.1 on pci0=0A=
ata8: at 0x7400 on atapci3=0A=
ata9: at 0x6800 on atapci3=0A=
atapci4: <VIA 6420 SATA150 controller> port =
0x4000-0x40ff,0x4400-0x440f,0x4800-0x4803,0x5000-0x5007,0x5400-0x5403,0x5=
800-0x5807 irq 20 at device 15.0 on pci0=0A=
ata10: at 0x5800 on atapci4=0A=
ata11: at 0x5000 on atapci4=0A=
atapci5: <VIA 8237 UDMA133 controller> port =
0x3800-0x380f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 irq 20 at device 15.1 =
on pci0=0A=
ata0: at 0x1f0 irq 14 on atapci5=0A=
ata1: at 0x170 irq 15 on atapci5=0A=
uhci0: <VIA 83C572 USB controller> port 0x3400-0x341f irq 21 at device =
16.0 on pci0=0A=
uhci0: [GIANT-LOCKED]=0A=
usb0: <VIA 83C572 USB controller> on uhci0=0A=
usb0: USB revision 1.0=0A=
uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A=
uhub0: 2 ports with 2 removable, self powered=0A=
uhci1: <VIA 83C572 USB controller> port 0x3000-0x301f irq 21 at device =
16.1 on pci0=0A=
uhci1: [GIANT-LOCKED]=0A=
usb1: <VIA 83C572 USB controller> on uhci1=0A=
usb1: USB revision 1.0=0A=
uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A=
uhub1: 2 ports with 2 removable, self powered=0A=
uhci2: <VIA 83C572 USB controller> port 0x2800-0x281f irq 21 at device =
16.2 on pci0=0A=
uhci2: [GIANT-LOCKED]=0A=
usb2: <VIA 83C572 USB controller> on uhci2=0A=
usb2: USB revision 1.0=0A=
uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A=
uhub2: 2 ports with 2 removable, self powered=0A=
uhci3: <VIA 83C572 USB controller> port 0x2400-0x241f irq 21 at device =
16.3 on pci0=0A=
uhci3: [GIANT-LOCKED]=0A=
usb3: <VIA 83C572 USB controller> on uhci3=0A=
usb3: USB revision 1.0=0A=
uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A=
uhub3: 2 ports with 2 removable, self powered=0A=
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xed000000-0xed0000ff irq =
21 at device 16.4 on pci0=0A=
ehci0: [GIANT-LOCKED]=0A=
ehci_pci_attach: companion usb0=0A=
ehci_pci_attach: companion usb1=0A=
ehci_pci_attach: companion usb2=0A=
ehci_pci_attach: companion usb3=0A=
usb4: EHCI version 1.0=0A=
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3=0A=
usb4: <EHCI (generic) USB 2.0 controller> on ehci0=0A=
usb4: USB revision 2.0=0A=
uhub4: (0x1106) EHCI root hub, class 9/0, rev 2.00/1.00, addr 1=0A=
uhub4: 8 ports with 8 removable, self powered=0A=
isab0: <PCI-ISA bridge> at device 17.0 on pci0=0A=
isa0: <ISA bus> on isab0=0A=
vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0x2000-0x20ff mem =
0xec800000-0xec8000ff irq 23 at device 18.0 on pci0=0A=
miibus0: <MII bus> on vr0=0A=
rlphy0: <RTL8201L 10/100 media interface> on miibus0=0A=
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto=0A=
vr0: Ethernet address: 00:0e:a6:1f:29:1e=0A=
vr0: [GIANT-LOCKED]=0A=
re0: <RealTek 8110S Single-chip Gigabit Ethernet> port 0x1800-0x18ff mem =
0xec000000-0xec0000ff irq 18 at device 19.0 on pci0=0A=
miibus1: <MII bus> on re0=0A=
rgephy0: <RTL8169S/8110S media interface> on miibus1=0A=
rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, =
1000baseTX-FDX, auto=0A=
re0: Ethernet address: 00:50:fc:f8:c6:81=0A=
re0: [GIANT-LOCKED]=0A=
fdc0: <Enhanced floppy controller (i82077, NE72065 or clone)> port =
0x3f7,0x3f2-0x3f5 irq 6 drq 2 on acpi0=0A=
fdc0: FIFO enabled, 8 bytes threshold=0A=
fd0: <1440-KB 3.5" drive> on fdc0 drive 0=0A=
sio0: configured irq 4 not in bitmap of probed irqs 0=0A=
sio0: port may not be enabled=0A=
sio0 port 0x3f8-0x3ff irq 4 on acpi0=0A=
sio0: type 16550A=0A=
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0=0A=
atkbd0: <AT Keyboard> irq 1 on atkbdc0=0A=
kbd0 at atkbd0=0A=
atkbd0: [GIANT-LOCKED]=0A=
orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0=0A=
pmtimer0 on isa0=0A=
sc0: <System console> at flags 0x100 on isa0=0A=
sc0: VGA <16 virtual consoles, flags=3D0x300>=0A=
sio1: configured irq 3 not in bitmap of probed irqs 0=0A=
sio1: port may not be enabled=0A=
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0=0A=
Timecounter "TSC" frequency 1999782834 Hz quality 800=0A=
Timecounters tick every 1.000 msec=0A=
ipfw2 initialized, divert enabled, rule-based forwarding enabled, =
default to accept, logging unlimited=0A=
ad0: 114473MB <ST3120022A> [232581/16/63] at ata0-master UDMA100=0A=
ad1: 114473MB <ST3120022A> [232581/16/63] at ata0-slave UDMA100=0A=
ad2: 117800MB <IC35L120AVV207-0> [239340/16/63] at ata1-master UDMA100=0A=
ad3: 117800MB <IC35L120AVV207-0> [239340/16/63] at ata1-slave UDMA100=0A=
ad4: 238475MB <WDC WD2500JB-00FUA0> [484521/16/63] at ata2-master UDMA100=0A=
ad5: 238475MB <WDC WD2500JB-00FUA0> [484521/16/63] at ata2-slave UDMA100=0A=
ad6: 239372MB <Maxtor 4A250J0> [486344/16/63] at ata3-master UDMA133=0A=
ad7: 239372MB <Maxtor 5A250J0> [486344/16/63] at ata3-slave UDMA133=0A=
ad8: 194481MB <Maxtor 6Y200P0> [395136/16/63] at ata4-master UDMA133=0A=
ad9: 194481MB <Maxtor 6Y200P0> [395136/16/63] at ata4-slave UDMA133=0A=
ad10: 194481MB <Maxtor 6Y200P0> [395136/16/63] at ata5-master UDMA133=0A=
ad11: 239372MB <Maxtor 7Y250P0> [486344/16/63] at ata5-slave UDMA133=0A=
ad12: 117800MB <IC35L120AVV207-0> [239340/16/63] at ata6-master UDMA100=0A=
ad13: 117800MB <HDS722512VLAT20> [239340/16/63] at ata6-slave UDMA100=0A=
ad14: 117246MB <Maxtor 6Y120L0> [238216/16/63] at ata7-master UDMA133=0A=
ad15: 117246MB <Maxtor 6Y120L0> [238216/16/63] at ata7-slave UDMA133=0A=
ad16: 114473MB <WDC WD1200JB-00DUA3> [232581/16/63] at ata8-master =
UDMA100=0A=
ad18: 26059MB <Maxtor 92732U8> [52946/16/63] at ata9-master UDMA66=0A=
ad20: 239372MB <Maxtor 7Y250M0> [486344/16/63] at ata10-master SATA150=0A=
ad22: 238475MB <WDC WD2500JD-00FYB0> [484521/16/63] at ata11-master =
SATA150=0A=
ar0: 476950MB <ATA RAID0 array> [60802/255/63] status: READY subdisks:=0A=
disk0 READY on ad4 at ata2-master=0A=
disk1 READY on ad5 at ata2-slave=0A=
ar1: 478744MB <ATA RAID0 array> [61031/255/63] status: READY subdisks:=0A=
disk0 READY on ad6 at ata3-master=0A=
disk1 READY on ad7 at ata3-slave=0A=
ar2: 388962MB <ATA RAID0 array> [49585/255/63] status: READY subdisks:=0A=
disk0 READY on ad9 at ata4-slave=0A=
disk1 READY on ad8 at ata4-master=0A=
Waiting 5 seconds for SCSI devices to settle=0A=
sa0 at ahc0 bus 0 target 5 lun 0=0A=
sa0: <Seagate STT20000N 6A51> Removable Sequential Access SCSI-2 device =0A=
sa0: 10.000MB/s transfers (10.000MHz, offset 15)=0A=
da0 at ahc0 bus 0 target 0 lun 0=0A=
da0: <QUANTUM ATLAS 10K 9WLS UCH0> Fixed Direct Access SCSI-3 device =0A=
da0: 40.000MB/s transfers (20.000MHz, offset 31, 16bit), Tagged Queueing =
Enabled=0A=
da0: 8759MB (17938986 512 byte sectors: 255H 63S/T 1116C)=0A=
da1 at ahc0 bus 0 target 1 lun 0=0A=
da1: <QUANTUM ATLAS 10K 9WLS UCH0> Fixed Direct Access SCSI-3 device =0A=
da1: 40.000MB/s transfers (20.000MHz, offset 31, 16bit), Tagged Queueing =
Enabled=0A=
da1: 8759MB (17938986 512 byte sectors: 255H 63S/T 1116C)=0A=
da2 at ahc0 bus 0 target 10 lun 0=0A=
da2: <IBM DDYS-T36950N S93E> Fixed Direct Access SCSI-3 device =0A=
da2: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged =
Queueing Enabled=0A=
da2: 35003MB (71687340 512 byte sectors: 255H 63S/T 4462C)=0A=
da3 at ahc0 bus 0 target 11 lun 0=0A=
da3: <IBM IC35L036UWD210-0 S5CQ> Fixed Direct Access SCSI-3 device =0A=
da3: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged =
Queueing Enabled=0A=
da3: 35003MB (71687340 512 byte sectors: 255H 63S/T 4462C)=0A=
da4 at ahc0 bus 0 target 12 lun 0=0A=
da4: <IBM DDYS-T36950N S93E> Fixed Direct Access SCSI-3 device =0A=
da4: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged =
Queueing Enabled=0A=
da4: 35003MB (71687340 512 byte sectors: 255H 63S/T 4462C)=0A=
da5 at ahc0 bus 0 target 13 lun 0=0A=
da5: <IBM DDYS-T36950N S93E> Fixed Direct Access SCSI-3 device =0A=
da5: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged =
Queueing Enabled=0A=
da5: 35003MB (71687340 512 byte sectors: 255H 63S/T 4462C)=0A=
Mounting root from ufs:/dev/ad0s1a=0A=
------=_NextPart_000_0000_01C46B07.F3CD7D10--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAASwXyhQrhq0KvaGmo0/NC3gEAAAAA>