From owner-freebsd-bugs@FreeBSD.ORG Fri Aug 17 16:50:00 2007 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD00316A41B for ; Fri, 17 Aug 2007 16:50:00 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B885D13C48A for ; Fri, 17 Aug 2007 16:50:00 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l7HGo055038727 for ; Fri, 17 Aug 2007 16:50:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l7HGo0xi038726; Fri, 17 Aug 2007 16:50:00 GMT (envelope-from gnats) Resent-Date: Fri, 17 Aug 2007 16:50:00 GMT Resent-Message-Id: <200708171650.l7HGo0xi038726@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Paolo Pisati Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1DA1116A419 for ; Fri, 17 Aug 2007 16:40:21 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 0C78613C4A6 for ; Fri, 17 Aug 2007 16:40:21 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.1/8.14.1) with ESMTP id l7HGeK1r060078 for ; Fri, 17 Aug 2007 16:40:20 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.1/8.14.1/Submit) id l7HGeKfj060077; Fri, 17 Aug 2007 16:40:20 GMT (envelope-from nobody) Message-Id: <200708171640.l7HGeKfj060077@www.freebsd.org> Date: Fri, 17 Aug 2007 16:40:20 GMT From: Paolo Pisati To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/115606: Panic while using mpt controller X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2007 16:50:01 -0000 >Number: 115606 >Category: kern >Synopsis: Panic while using mpt controller >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Aug 17 16:50:00 GMT 2007 >Closed-Date: >Last-Modified: >Originator: Paolo Pisati >Release: 7.0-CURRENT (but the same happens on 6.x) >Organization: Tomato Interactive >Environment: FreeBSD ferrettolo.tomato.lan 7.0-CURRENT FreeBSD 7.0-CURRENT #1: Fri Aug 17 14:54:35 UTC 2007 piso@ferrettolo.tomato.lan:/usr/obj/usr/src/sys/GENERIC i386 >Description: While doing some i/o work on a disk attached to an mpt controller, FreeBSD 6.x/7.x panic with the following message: ferrettolo# kgdb /usr/obj/usr/src/sys/GENERIC/kernel.debug /var/crash/vmcore.1 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: mpt0: mpt_cam_event: 0xc0 mpt0: Unhandled Event Notify Frame. Event 0xc0afa2c0 (ACK not required). mpt0: completing timedout/aborted req 0xc406d450:5896 panic: Bad link elm 0xc406d450 next->prev != elm cpuid = 0 KDB: enter: panic Dumping 1023 MB (2 chunks) chunk 0: 1MB (157 pages) ... ok chunk 1: 1023MB (261868 pages) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc048cb29 in db_fncall (dummy1=-465581460, dummy2=0, dummy3=524358, dummy4=0xe43fc9d8 "ÐíHÀ") at /usr/src/sys/ddb/db_command.c:486 #2 0xc048d095 in db_command_loop () at /usr/src/sys/ddb/db_command.c:401 #3 0xc048e805 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:222 #4 0xc0773066 in kdb_trap (type=3, code=0, tf=0xe43fcb80) at /usr/src/sys/kern/subr_kdb.c:502 #5 0xc09fd15b in trap (frame=0xe43fcb80) at /usr/src/sys/i386/i386/trap.c:621 #6 0xc09e2b7b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc07731e2 in kdb_enter (msg=0xc0a93413 "panic") at cpufunc.h:60 #8 0xc074c3e4 in panic (fmt=0xc0a49d68 "Bad link elm %p next->prev != elm") at /usr/src/sys/kern/kern_shutdown.c:547 #9 0xc06065c5 in mpt_scsi_reply_handler (mpt=0xc4066000, req=0xc406d450, reply_desc=65820, reply_frame=0xe43fcc58) at /usr/src/sys/dev/mpt/mpt_cam.c:2661 #10 0xc06033be in mpt_complete_request_chain (mpt=0xc4066000, chain=0xc40663c4, iocstatus=8) at /usr/src/sys/dev/mpt/mpt.c:849 #11 0xc0606734 in mpt_cam_ioc_reset (mpt=0xc4066000, type=0) at /usr/src/sys/dev/mpt/mpt_cam.c:3041 #12 0xc0604acf in mpt_reset (mpt=0xc4066000, reinit=1) at /usr/src/sys/dev/mpt/mpt.c:1145 #13 0xc0607bf2 in mpt_recovery_thread (arg=0xc4066000) at /usr/src/sys/dev/mpt/mpt_cam.c:4176 #14 0xc072e258 in fork_exit (callout=0xc0607920 , arg=0xc4066000, frame=0xe43fcd38) at /usr/src/sys/kern/kern_fork.c:797 #15 0xc09e2bf0 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:205 dmesg: ferrettolo# dmesg Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-CURRENT #1: Fri Aug 17 14:54:35 UTC 2007 piso@ferrettolo.tomato.lan:/usr/obj/usr/src/sys/GENERIC WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2790.72-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Features2=0x4400 Logical CPUs per core: 2 real memory = 1073659904 (1023 MB) avail memory = 1037049856 (989 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 MADT: Forcing active-low polarity and level trigger for SCI ioapic2 irqs 32-47 on motherboard ioapic1 irqs 16-31 on motherboard ioapic0 irqs 0-15 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 460, 2 (4) failed Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x488-0x48b on acpi0 cpu0: on acpi0 p4tcc0: on cpu0 cpu1: on acpi0 p4tcc1: on cpu1 pcib0: on acpi0 pci0: on pcib0 vgapci0: port 0x2200-0x22ff mem 0xfd000000-0xfdffffff,0xfebff000-0xfebfffff irq 26 at device 1.0 on pci0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x700-0x70f at device 15.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] ohci0: mem 0xfebfe000-0xfebfefff irq 11 at device 15.2 on pci0 ohci0: [GIANT-LOCKED] ohci0: [ITHREAD] usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 uhub0: <(0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 uhub0: 4 ports with 4 removable, self powered isab0: at device 15.3 on pci0 isa0: on isab0 pcib1: on acpi0 pci1: on pcib1 mpt0: port 0x2300-0x23ff mem 0xfbff0000-0xfbffffff,0xfbfe0000-0xfbfeffff irq 22 at device 1.0 on pci1 mpt0: [ITHREAD] mpt0: MPI Version=1.2.6.0 mpt0: Capabilities: ( RAID-1 SAFTE ) mpt0: 1 Active Volume (1 Max) mpt0: 2 Hidden Drive Members (3 Max) pcib2: on acpi0 pci2: on pcib2 pci2:1:0: bad VPD cksum, remain 14 bge0: mem 0xf9ff0000-0xf9ffffff irq 24 at device 1.0 on pci2 miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: Ethernet address: 00:11:25:3e:95:1e bge0: [ITHREAD] pci2:2:0: bad VPD cksum, remain 14 bge1: mem 0xf9fe0000-0xf9feffff irq 25 at device 2.0 on pci2 miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge1: Ethernet address: 00:11:25:3e:95:1f bge1: [ITHREAD] fdc0: port 0x3f0-0x3f5 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc97ff,0xc9800-0xcd7ff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 8250 or not responding sio0: [FILTER] sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec mpt0:vol0(mpt0:0ad2: 98166MB at ata1-master UDMA100 :0): Settings ( Hot-Plug-Spares ) mpt0:vol0(mpt0:0:0): Using Spare Pool: 0 mpt0:vol0(mpt0:0:0): 2 Members: (mpt0:1:0:0): Primary Online (mpt0:1:1:0): Secondary Online mpt0:vol0(mpt0:0:0): RAID-1 - Optimal mpt0:vol0(mpt0:0:0): Status ( Enabled ) (mpt0:vol0:0): Physical (mpt0:0:0:0), Pass-thru (mpt0:1:0:0) (mpt0:vol0:0): Online (mpt0:vol0:1): Physical (mpt0:0:1:0), Pass-thru (mpt0:1:1:0) (mpt0:vol0:1): Online uma_zalloc_arg: zone "1024" with the following non-sleepable locks held: exclusive sleep mutex mpt r = 0 (0xc4066004) locked @ /usr/src/sys/dev/mpt/mpt_raid.c:671 KDB: stack backtrace: db_trace_self_wrapper(c0a95e89,e449ac44,c0782a3d,c0a9624c,e449ac58,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9624c,e449ac58,4,1,0,...) at kdb_backtrace+0x29 witness_warn(5,0,c0ab40fa,c0a493c8,142e,...) at witness_warn+0x1cd uma_zalloc_arg(c146d5a0,0,2,2,c4066000,...) at uma_zalloc_arg+0x34 malloc(260,c0b43880,2,29f,e449acf8,...) at malloc+0xd2 mpt_raid_thread(c4066000,e449ad38,c0a8fda1,315,c4075ab0,...) at mpt_raid_thread+0xfb fork_exit(c0613610,c4066000,e449ad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe449ad70, ebp = 0 --- acd0: DVDROM at ata1-slave UDMA33 Waiting 5 seconds for SCSI devices to settle (probe0:mpt0:1:0:0): Vendor Specific Command. CDB: 12 0 0 0 a4 0 (probe0:mpt0:1:0:0): CAM Status: SCSI Status Error (probe0:mpt0:1:0:0): SCSI Status: Check Condition (probe0:mpt0:1:0:0): ABORTED COMMAND asc:48,0 (probe0:mpt0:1:0:0): Initiator detected error message received (probe0:mpt0:1:0:0): Retrying Command (per Sense Data) (probe0:mpt0:1:0:0): Vendor Specific Command. CDB: 12 0 0 0 a4 0 (probe0:mpt0:1:0:0): CAM Status: SCSI Status Error (probe0:mpt0:1:0:0): SCSI Status: Check Condition (probe0:mpt0:1:0:0): ABORTED COMMAND asc:48,0 (probe0:mpt0:1:0:0): Initiator detected error message received (probe0:mpt0:1:0:0): Retrying Command (per Sense Data) (probe0:mpt0:1:0:0): Vendor Specific Command. CDB: 12 0 0 0 a4 0 (probe0:mpt0:1:0:0): CAM Status: SCSI Status Error (probe0:mpt0:1:0:0): SCSI Status: Check Condition (probe0:mpt0:1:0:0): ABORTED COMMAND asc:48,0 (probe0:mpt0:1:0:0): Initiator detected error message received (probe0:mpt0:1:0:0): Retrying Command (per Sense Data) ses0 at mpt0 bus 0 target 8 lun 0 ses0: Fixed Processor SCSI-2 device ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 80, 16bit) da0: Command Queueing Enabled da0: 139898MB (286511105 512 byte sectors: 255H 63S/T 17834C) SMP: AP CPU #1 Launched! WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ad2s1a WARNING: / was not properly dismounted WARNING: /tmp was not properly dismounted WARNING: /usr was not properly dismounted WARNING: /var was not properly dismounted bge0: link state changed to UP Notice the WITNESS warning: uma_zalloc_arg: zone "1024" with the following non-sleepable locks held: exclusive sleep mutex mpt r = 0 (0xc4066004) locked @ /usr/src/sys/dev/mpt/mpt_raid.c:671 KDB: stack backtrace: db_trace_self_wrapper(c0a95e89,e449ac44,c0782a3d,c0a9624c,e449ac58,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9624c,e449ac58,4,1,0,...) at kdb_backtrace+0x29 witness_warn(5,0,c0ab40fa,c0a493c8,142e,...) at witness_warn+0x1cd uma_zalloc_arg(c146d5a0,0,2,2,c4066000,...) at uma_zalloc_arg+0x34 malloc(260,c0b43880,2,29f,e449acf8,...) at malloc+0xd2 mpt_raid_thread(c4066000,e449ad38,c0a8fda1,315,c4075ab0,...) at mpt_raid_thread+0xfb fork_exit(c0613610,c4066000,e449ad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe449ad70, ebp = 0 --- Hw: IBM X335 with SCSI U320 FreeBSD 4.x works flawlessly on the same hw. You can grab kernel.debug and vmcore.1 this way: fetch http://people.tomato.it/piso/vmcore.1.gz fetch http://people.tomato.it/piso/kernel.debug.gz >How-To-Repeat: Mount a disk on an mpt controller, copy /usr/src over it and enjoy the panic... :) >Fix: >Release-Note: >Audit-Trail: >Unformatted: