From owner-freebsd-bugs@FreeBSD.ORG Tue Feb 17 22:50:03 2009 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93F4B1065680 for ; Tue, 17 Feb 2009 22:50:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6F43A8FC12 for ; Tue, 17 Feb 2009 22:50:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n1HMo3Mo022313 for ; Tue, 17 Feb 2009 22:50:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n1HMo39b022312; Tue, 17 Feb 2009 22:50:03 GMT (envelope-from gnats) Resent-Date: Tue, 17 Feb 2009 22:50:03 GMT Resent-Message-Id: <200902172250.n1HMo39b022312@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Colin Faber Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 384EF1065678 for ; Tue, 17 Feb 2009 22:41:01 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 264A58FC1B for ; Tue, 17 Feb 2009 22:41:01 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n1HMf0Ej089059 for ; Tue, 17 Feb 2009 22:41:00 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id n1HMf0MW089058; Tue, 17 Feb 2009 22:41:00 GMT (envelope-from nobody) Message-Id: <200902172241.n1HMf0MW089058@www.freebsd.org> Date: Tue, 17 Feb 2009 22:41:00 GMT From: Colin Faber To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/131796: Introducing new USB or eSATA disks results in atapci1+ irq storm until reboot X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Feb 2009 22:50:04 -0000 >Number: 131796 >Category: kern >Synopsis: Introducing new USB or eSATA disks results in atapci1+ irq storm until reboot >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Feb 17 22:50:03 UTC 2009 >Closed-Date: >Last-Modified: >Originator: Colin Faber >Release: 7.1-RELENG-p2 amd64 kernel >Organization: n/a >Environment: System: FreeBSD hal.fpsn 7.1-RELEASE-p2 FreeBSD 7.1-RELEASE-p2 #2: Sun Feb 15 12:21:19 MST 2009 cfaber@hal.fpsn:/usr/obj/usr/src/sys/HAL amd64 >Description: It appears that when plugging in any USB, or eSATA disk drive, atapci1+ gets stuck in a massive irq storm, running about 13k irqs a second. This results in about 90% interrupt load on the system and makes everything unstable and slow. Note - this load appears even with ZERO activity on the device in question and little or no geom activity. Also Note - this machine is running two geom_raid5 array's made of geom labeled providers Non-storm: cfaber@hal:~$ vmstat -i interrupt total rate irq1: atkbd0 6 0 irq14: ata0 4316 0 irq16: mpt0 1782986 243 irq21: ohci0 213266 29 irq23: atapci1+ 2182108 298 cpu0: timer 14628721 1999 irq256: nfe0 215036 29 Total 19026439 2601 Storm: cfaber@hal:~$ vmstat -i interrupt total rate irq1: atkbd0 6 0 irq14: ata0 1534 1 irq16: mpt0 674 0 irq21: ohci0 1169 0 irq22: ehci0 193 0 irq23: atapci1+ 20420307 14230 cpu0: timer 2868072 1998 irq256: nfe0 1171 0 Total 23293126 16232 >How-To-Repeat: cvsup to RELENG_7_1 configure an amd64 kernel with the following parameters: cpu HAMMER ident HAL options GEOM_JOURNAL options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journaling options MD_ROOT # MD is a potential root device options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options NFSLOCKD # Network Lock Manager options NTFS # NT File System options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization options COMPAT_43TTY # BSD 4.3 TTY compat [KEEP THIS!] options COMPAT_IA32 # Compatible with i386 binaries options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options ADAPTIVE_GIANT # Giant mutex is adaptive. options STOP_NMI # Stop CPUS using NMI instead of IPI options AUDIT # Security event auditing options COMPAT_LINUX32 options LINPROCFS options SMP # Symmetric MultiProcessor Kernel device cpufreq device acpi device pci device ata device atadisk # ATA disk drives device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives options ATA_STATIC_ID # Static device numbering device mpt # LSI-Logic MPT-Fusion device scbus # SCSI bus (required for SCSI) device ch # SCSI media changers device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) device ses # SCSI Environmental Services (and SAF-TE) device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver device splash # Splash screen and screen saver support device sc device miibus # MII bus support device nfe # nVidia nForce MCP on-board Ethernet device loop # Network loopback device random # Entropy device device ether # Ethernet support device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" device firmware # firmware assist module device bpf # Berkeley packet filter device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device usb # USB Bus (required) device ugen # Generic device uhid # "Human Interface Devices" device ukbd # Keyboard device ulpt # Printer device umass # Disks/Mass storage - Requires scbus and da device ums # Mouse device uscanner # Scanners device ucom # Generic com ttys device sound # Provide general sound support Next, connect any umass device, or sata device. atapci1+ will go nuts the second the kernel sees the device online. >Fix: So far, the only way to get the problem to stop is to reboot the system. If the device is connected already, then there are no problems with irq storms on atapci1+. I'm willing to try and debugging procedures provided. As well as provide additional information once requested. >Release-Note: >Audit-Trail: >Unformatted: