From owner-freebsd-scsi@FreeBSD.ORG Mon Oct 13 11:06:56 2008 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9774F1065692 for ; Mon, 13 Oct 2008 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 835DB8FC1E for ; Mon, 13 Oct 2008 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id m9DB6u1m029561 for ; Mon, 13 Oct 2008 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id m9DB6u5F029557 for freebsd-scsi@FreeBSD.org; Mon, 13 Oct 2008 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 13 Oct 2008 11:06:56 GMT Message-Id: <200810131106.m9DB6u5F029557@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2008 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o i386/127927 scsi isp(4) target driver crashes kernel when set up dma fo o kern/127901 scsi [scsi] "inquiry data fails comparison at DV1 step" [re o kern/126866 scsi [isp] [panic] kernel panic on card initialization o kern/124667 scsi [amd] [panic] FreeBSD-7 kernel page faults at amd-scsi o kern/123674 scsi [ahc] ahc driver dumping o kern/123666 scsi [aac] attach fails with Adaptec SAS RAID 3805 controll o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc o kern/120487 scsi [sg] scsi_sg incompatible with scanners o kern/120247 scsi [mpt] FreeBSD 6.3 and LSI Logic 1030 = only 3.300MB/s o kern/119668 scsi [cam] [patch] certain errors are too verbose comparing o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks o kern/99954 scsi [ahc] reading from DVD failes on 6.x [regression] o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load o kern/60598 scsi wire down of scsi devices conflicts with config s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/40895 scsi wierd kernel / device driver bug o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/38828 scsi [dpt] [request] DPT PM2012B/90 doesn't work o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce 28 problems total. From owner-freebsd-scsi@FreeBSD.ORG Tue Oct 14 22:23:48 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28DDC1065691; Tue, 14 Oct 2008 22:23:48 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from achilles.noc.ntua.gr (achilles.noc.ntua.gr [IPv6:2001:648:2000:de::210]) by mx1.freebsd.org (Postfix) with ESMTP id 4576C8FC1D; Tue, 14 Oct 2008 22:23:46 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (ajax6.noc.ntua.gr [IPv6:2001:648:2000:dc::1]) by achilles.noc.ntua.gr (8.14.3/8.14.3) with ESMTP id m9EMNi7S021241 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 01:23:44 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (localhost.noc.ntua.gr [127.0.0.1]) by ajax.noc.ntua.gr (8.13.8/8.13.8) with ESMTP id m9EMNhsC010061 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 01:23:44 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: (from christia@localhost) by ajax.noc.ntua.gr (8.13.8/8.13.8/Submit) id m9EMNh6o010060; Wed, 15 Oct 2008 01:23:43 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) X-Authentication-Warning: ajax.noc.ntua.gr: christia set sender to p.christias@noc.ntua.gr using -f Date: Wed, 15 Oct 2008 01:23:43 +0300 From: Panagiotis Christias To: freebsd-scsi@freebsd.org, freebsd-stable@freebsd.org Message-ID: <20081014222343.GA8706@noc.ntua.gr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-09) X-Virus-Scanned: ClamAV version 0.94, clamav-milter version 0.94 on achilles.noc.ntua.gr X-Virus-Status: Clean Cc: Subject: FreeBSD 7-STABLE, isp(4), QLE2462: panic & deadlocks X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Oct 2008 22:23:48 -0000 Hello everybody, we recently got three Dell PowerEdge servers equipped with Qlogic QLE2462 cards (dual FC 4Gbps ports) and an EMC CLARiiON CX3-40 SAN. The servers with the FC cards were successfully and extensively tested with an older SUN StorEdge T3 SAN. However, when we connect them to the CX3-40, create and mount a new partition and then do something as simple as "tar -C /san -xf ports.tgz" the system panics and deadlocks. We have tried several FreeBSD versions (6.3 i386/adm64, 7.0 i386/adm64, 7.1 i386/adm64 and lastly 7-STABLE i386 - we also tried the latest 8-CURRENT snapshot but it panicked too soon). The result is always the same; panic and deadlock. In case it helps, we have also tried all available connection speeds (4, 2 and 1Gbps), the QLE2462 cards have installed the latest firwmare version (4.03.10) and we have verified our hardware setup by testing it with Linux (CentOS 5.2), although we would try anything in order to keep the whole of our server base to FreeBSD (we have succeeded till now). Currently, we have installed the latest version of 7-STABLE (csuped today) along with a debug-enabled kernel and collected as much output as we can. See below. We are open to any suggestion and any kind of tests in order to help resolving the problem. Thanks! COLLECTED DATA: 1. uname -a: FreeBSD 7x86.noc.ntua.gr 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Tue Oct 14 21:12:57 EEST 2008 root@7x86.noc.ntua.gr:/usr/obj/usr/src/sys/GENERIC-DEBUG i386 --------------------------------------------------------------- 2. Kernel configuration file /sys/i386/conf/GENERIC-DEBUG: include GENERIC ident GENERIC-DEBUG # force isp firmware to fully loaded # device ispfw options KDB options DDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options DEBUG_LOCKS options DEBUG_VFS_LOCKS options DIAGNOSTIC options CAMDEBUG options CAM_DEBUG_BUS=-1 options CAM_DEBUG_TARGET=-1 options CAM_DEBUG_LUN=-1 options CAM_DEBUG_FLAGS=CAM_DEBUG_INFO --------------------------------------------------------------- 3. dmesg output, also available at http://noc.ntua.gr/~christia/san/7-STABLE.dmesg.out : Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-PRERELEASE #0: Tue Oct 14 21:12:57 EEST 2008 root@7x86.noc.ntua.gr:/usr/obj/usr/src/sys/GENERIC-DEBUG WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU E5405 @ 2.00GHz (1997.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x10676 Stepping = 6 Features=0xbfebfbff Features2=0xce33d> AMD Features=0x20100000 AMD Features2=0x1 Cores per package: 4 real memory = 3484745728 (3323 MB) avail memory = 3405070336 (3247 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 WITNESS: spin lock cpuset not in order list WITNESS: spin lock intrcnt not in order list ioapic0: Changing APIC ID to 8 ioapic1: Changing APIC ID to 9 ioapic2: Changing APIC ID to 10 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 160-183 on motherboard ioapic2 irqs 224-247 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci4: on pcib1 pcib2: at device 0.0 on pci4 pci5: on pcib2 pcib3: at device 0.0 on pci5 pci6: on pcib3 pcib4: at device 0.0 on pci6 pci7: on pcib4 bce0: mem 0xf4000000-0xf5ffffff irq 16 at device 0.0 on pci7 miibus0: on bce0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce0: Ethernet address: 00:1e:c9:e8:13:e9 bce0: [ITHREAD] bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); F/W (0x04000305); Flags( MFW MSI ) pcib5: at device 1.0 on pci5 pci8: on pcib5 pcib6: at device 0.3 on pci4 pci9: on pcib6 pcib7: at device 3.0 on pci0 pci1: on pcib7 mfi0: port 0xec00-0xecff mem 0xfc780000-0xfc7bffff,0xfc740000-0xfc77ffff irq 16 at device 0.0 on pci1 mfi0: Megaraid SAS driver Ver 2.00 mfi0: 3299 (277344669s/0x0020/info) - Shutdown command received from host mfi0: 3300 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0060/1000/1f0c/1028) mfi0: 3301 (boot + 3s/0x0020/info) - Firmware version 1.11.82-0473 mfi0: 3302 (boot + 3s/0x0008/info) - Battery Present mfi0: 3303 (boot + 3s/0x0020/info) - Package version 6.0.3-0002 mfi0: 3304 (boot + 23s/0x0004/info) - Enclosure (SES) discovered on PD 20(c None/p0) mfi0: 3305 (boot + 23s/0x0002/info) - Unexpected sense: Encl PD 20, CDB: 12 00 00 00 00 00, Sense: 70 00 06 00 00 00 00 0a 00 00 00 00 29 00 00 00 00 00 mfi0: 3306 (boot + 24s/0x0002/info) - Inserted: Encl PD 20 mfi0: 3307 (boot + 24s/0x0002/info) - Inserted: PD 20(c None/p0) Info: enclPd=20, scsiType=d, portMap=09, sasAddr=5001e090d62bfb00,0000000000000000 mfi0: 3308 (boot + 24s/0x0002/info) - Inserted: PD 00(e0x20/s0) mfi0: 3309 (boot + 24s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: enclPd=20, scsiType=0, portMap=00, sasAddr=5000cca005b1bf89,0000000000000000 mfi0: 3310 (boot + 24s/0x0002/info) - Inserted: PD 01(e0x20/s1) mfi0: 3311 (boot + 24s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=5000cca005b1427d,0000000000000000 mfi0: 3312 (boot + 24s/0x0002/info) - Inserted: PD 02(e0x20/s2) mfi0: 3313 (boot + 24s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: enclPd=20, scsiType=0, portMap=02, sasAddr=5000cca005b13c61,0000000000000000 mfi0: 3314 (boot + 24s/0x0042/info) - Global Hot Spare created on PD 01(e0x20/s1) (global) mfi0: 3315 (boot + 24s/0x0002/info) - State change on PD 01(e0x20/s1) from UNCONFIGURED_GOOD(0) to HOT SPARE(2) mfi0: 3316 (boot + 68s/0x0008/info) - Battery temperature is normal mfi0: 3317 (boot + 68s/0x0008/info) - Battery started charging mfi0: 3318 (boot + 68s/0x0008/info) - Current capacity of the battery is above threshold mfi0: [ITHREAD] pcib8: at device 4.0 on pci0 pci10: on pcib8 isp0: port 0xdc00-0xdcff mem 0xfc5bc000-0xfc5bffff irq 16 at device 0.0 on pci10 firmware_get: failed to load firmware image isp_2400 isp0: [ITHREAD] isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 4.3.1 isp1: port 0xd800-0xd8ff mem 0xfc5b8000-0xfc5bbfff irq 17 at device 0.1 on pci10 firmware_get: failed to load firmware image isp_2400 isp1: [ITHREAD] isp1: Board Type 2422, Chip Revision 0x3, resident F/W Revision 4.3.1 pcib9: at device 5.0 on pci0 pci11: on pcib9 pcib10: at device 6.0 on pci0 pci12: on pcib10 pcib11: at device 0.0 on pci12 pci13: on pcib11 pcib12: at device 0.2 on pci12 pci14: on pcib12 pcib13: at device 7.0 on pci0 pci15: on pcib13 pcib14: at device 28.0 on pci0 pci2: on pcib14 pcib15: at device 0.0 on pci2 pci3: on pcib15 bce1: mem 0xf8000000-0xf9ffffff irq 16 at device 0.0 on pci3 miibus1: on bce1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce1: Ethernet address: 00:1e:c9:e8:13:e7 bce1: [ITHREAD] bce1: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); F/W (0x04000305); Flags( MFW MSI ) uhci0: port 0xbce0-0xbcff irq 21 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xbcc0-0xbcdf irq 20 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xbca0-0xbcbf irq 21 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xbc80-0xbc9f irq 20 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xfc800000-0xfc8003ff irq 21 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered uhub5: on uhub4 uhub5: multiple transaction translators uhub5: 2 ports with 2 removable, self powered ehci_device_intr_done: not done, ex=0xc699e800 ukbd0: on uhub5 ehci_device_intr_done: not done, ex=0xc699e000 kbd2 at ukbd0 ums0: on uhub5 ums0: X report 0x0002 not supported device_attach: ums0 attach returned 6 umass0: on uhub5 umass1: on uhub5 uhub6: on uhub4 uhub6: multiple transaction translators uhub6: 4 ports with 4 removable, self powered ehci_device_intr_done: not done, ex=0xc69a9400 pcib16: at device 30.0 on pci0 pci16: on pcib16 vgapci0: port 0xcc00-0xccff mem 0xd8000000-0xdfffffff,0xfc2d0000-0xfc2dffff irq 19 at device 13.0 on pci16 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console sio0: [FILTER] sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A sio1: [FILTER] cpu0: on acpi0 p4tcc0: on cpu0 cpu1: on acpi0 p4tcc1: on cpu1 cpu2: on acpi0 p4tcc2: on cpu2 cpu3: on acpi0 p4tcc3: on cpu3 cpu4: on acpi0 p4tcc4: on cpu4 cpu5: on acpi0 p4tcc5: on cpu5 cpu6: on acpi0 p4tcc6: on cpu6 cpu7: on acpi0 p4tcc7: on cpu7 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc8fff,0xc9000-0xc9fff,0xd4800-0xd57ff,0xec000-0xeffff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec bce0: link state changed to DOWN bce1: link state changed to DOWN acd0: DVDROM at ata0-master UDMA33 mfi0: 3319 (277344766s/0x0020/info) - Time established as 10/15/08 0:12:46; (71 seconds since power on) Expensive timeout(9) function: 0xc06f9de0(0xc0c52520) 0.113017715 s mfid0: on mfi0 mfid0: 69376MB (142082048 sectors) RAID volume '' is optimal (probe376:isp0:0:376:0): cam completion status 0x4a (probe376:isp0:0:376:0): cam completion status 0x4a (probe395:isp0:0:395:0): cam completion status 0x4a (probe395:isp0:0:395:0): cam completion status 0x4a (probe400:isp0:0:400:0): cam completion status 0x4a (probe400:isp0:0:400:0): cam completion status 0x4a (probe411:isp0:0:411:0): cam completion status 0x4a (probe411:isp0:0:411:0): cam completion status 0x4a (probe423:isp0:0:423:0): cam completion status 0x4a (probe423:isp0:0:423:0): cam completion status 0x4a (probe437:isp0:0:437:0): cam completion status 0x4a (probe437:isp0:0:437:0): cam completion status 0x4a (probe449:isp0:0:449:0): cam completion status 0x4a (probe449:isp0:0:449:0): cam completion status 0x4a (probe459:isp0:0:459:0): cam completion status 0x4a (probe459:isp0:0:459:0): cam completion status 0x4a (probe464:isp0:0:464:0): cam completion status 0x4a (probe464:isp0:0:464:0): cam completion status 0x4a (probe471:isp0:0:471:0): cam completion status 0x4a (probe471:isp0:0:471:0): cam completion status 0x4a (probe478:isp0:0:478:0): cam completion status 0x4a (probe478:isp0:0:478:0): cam completion status 0x4a (probe488:isp0:0:488:0): cam completion status 0x4a (probe488:isp0:0:488:0): cam completion status 0x4a (probe504:isp0:0:504:0): cam completion status 0x4a (probe504:isp0:0:504:0): cam completion status 0x4a (probe505:isp0:0:505:0): cam completion status 0x4a (probe505:isp0:0:505:0): cam completion status 0x4a (probe506:isp0:0:506:0): cam completion status 0x4a (probe506:isp0:0:506:0): cam completion status 0x4a (probe507:isp0:0:507:0): cam completion status 0x4a (probe507:isp0:0:507:0): cam completion status 0x4a (probe508:isp0:0:508:0): cam completion status 0x4a (probe508:isp0:0:508:0): cam completion status 0x4a (probe509:isp0:0:509:0): cam completion status 0x4a (probe509:isp0:0:509:0): cam completion status 0x4a (probe510:isp0:0:510:0): cam completion status 0x4a (probe510:isp0:0:510:0): cam completion status 0x4a (probe511:isp0:0:511:0): cam completion status 0x4a (probe511:isp0:0:511:0): cam completion status 0x4a (probe1:isp0:0:1:0): cam completion status 0x4a (probe1:isp0:0:1:0): cam completion status 0x4a (probe2:isp0:0:2:0): cam completion status 0x4a (probe2:isp0:0:2:0): cam completion status 0x4a (probe3:isp0:0:3:0): cam completion status 0x4a (probe0:isp0:0:0:0): cam completion status 0xcc (probe3:isp0:0:3:0): cam completion status 0x4a (probe4:isp0:0:4:0): cam completion status 0x4a (probe4:isp0:0:4:0): cam completion status 0x4a (probe5:isp0:0:5:0): cam completion status 0x4a (probe5:isp0:0:5:0): cam completion status 0x4a (probe7:isp0:0:7:0): cam completion status 0x4a (probe7:isp0:0:7:0): cam completion status 0x4a (probe8:isp0:0:8:0): cam completion status 0x4a (probe8:isp0:0:8:0): cam completion status 0x4a (probe9:isp0:0:9:0): cam completion status 0x4a (probe9:isp0:0:9:0): cam completion status 0x4a (probe10:isp0:0:10:0): cam completion status 0x4a (probe10:isp0:0:10:0): cam completion status 0x4a (probe11:isp0:0:11:0): cam completion status 0x4a (probe11:isp0:0:11:0): cam completion status 0x4a (probe12:isp0:0:12:0): cam completion status 0x4a (probe12:isp0:0:12:0): cam completion status 0x4a (probe13:isp0:0:13:0): cam completion status 0x4a (probe13:isp0:0:13:0): cam completion status 0x4a (probe14:isp0:0:14:0): cam completion status 0x4a (probe14:isp0:0:14:0): cam completion status 0x4a (probe15:isp0:0:15:0): cam completion status 0x4a (probe15:isp0:0:15:0): cam completion status 0x4a (probe16:isp0:0:16:0): cam completion status 0x4a (probe16:isp0:0:16:0): cam completion status 0x4a (probe17:isp0:0:17:0): cam completion status 0x4a (probe17:isp0:0:17:0): cam completion status 0x4a (probe18:isp0:0:18:0): cam completion status 0x4a (probe18:isp0:0:18:0): cam completion status 0x4a (probe19:isp0:0:19:0): cam completion status 0x4a (probe19:isp0:0:19:0): cam completion status 0x4a (probe20:isp0:0:20:0): cam completion status 0x4a (probe20:isp0:0:20:0): cam completion status 0x4a (probe21:isp0:0:21:0): cam completion status 0x4a (probe21:isp0:0:21:0): cam completion status 0x4a (probe22:isp0:0:22:0): cam completion status 0x4a (probe22:isp0:0:22:0): cam completion status 0x4a (probe23:isp0:0:23:0): cam completion status 0x4a (probe23:isp0:0:23:0): cam completion status 0x4a (probe24:isp0:0:24:0): cam completion status 0x4a (probe24:isp0:0:24:0): cam completion status 0x4a (probe26:isp0:0:26:0): cam completion status 0x4a (probe26:isp0:0:26:0): cam completion status 0x4a (probe27:isp0:0:27:0): cam completion status 0x4a (probe27:isp0:0:27:0): cam completion status 0x4a (probe28:isp0:0:28:0): cam completion status 0x4a (probe28:isp0:0:28:0): cam completion status 0x4a (probe29:isp0:0:29:0): cam completion status 0x4a (probe29:isp0:0:29:0): cam completion status 0x4a (probe30:isp0:0:30:0): cam completion status 0x4a (probe30:isp0:0:30:0): cam completion status 0x4a (probe31:isp0:0:31:0): cam completion status 0x4a (probe31:isp0:0:31:0): cam completion status 0x4a (probe32:isp0:0:32:0): cam completion status 0x4a (probe32:isp0:0:32:0): cam completion status 0x4a (probe33:isp0:0:33:0): cam completion status 0x4a (probe33:isp0:0:33:0): cam completion status 0x4a (probe34:isp0:0:34:0): cam completion status 0x4a (probe34:isp0:0:34:0): cam completion status 0x4a (probe35:isp0:0:35:0): cam completion status 0x4a (probe35:isp0:0:35:0): cam completion status 0x4a (probe36:isp0:0:36:0): cam completion status 0x4a (probe36:isp0:0:36:0): cam completion status 0x4a (probe37:isp0:0:37:0): cam completion status 0x4a (probe37:isp0:0:37:0): cam completion status 0x4a (probe38:isp0:0:38:0): cam completion status 0x4a (probe38:isp0:0:38:0): cam completion status 0x4a (probe39:isp0:0:39:0): cam completion status 0x4a (probe39:isp0:0:39:0): cam completion status 0x4a (probe40:isp0:0:40:0): cam completion status 0x4a (probe40:isp0:0:40:0): cam completion status 0x4a (probe41:isp0:0:41:0): cam completion status 0x4a (probe41:isp0:0:41:0): cam completion status 0x4a (probe42:isp0:0:42:0): cam completion status 0x4a (probe42:isp0:0:42:0): cam completion status 0x4a (probe43:isp0:0:43:0): cam completion status 0x4a (probe43:isp0:0:43:0): cam completion status 0x4a (probe44:isp0:0:44:0): cam completion status 0x4a (probe44:isp0:0:44:0): cam completion status 0x4a (probe45:isp0:0:45:0): cam completion status 0x4a (probe45:isp0:0:45:0): cam completion status 0x4a (probe46:isp0:0:46:0): cam completion status 0x4a (probe46:isp0:0:46:0): cam completion status 0x4a (probe47:isp0:0:47:0): cam completion status 0x4a (probe47:isp0:0:47:0): cam completion status 0x4a (probe48:isp0:0:48:0): cam completion status 0x4a (probe48:isp0:0:48:0): cam completion status 0x4a (probe49:isp0:0:49:0): cam completion status 0x4a (probe49:isp0:0:49:0): cam completion status 0x4a (probe50:isp0:0:50:0): cam completion status 0x4a (probe50:isp0:0:50:0): cam completion status 0x4a (probe51:isp0:0:51:0): cam completion status 0x4a (probe51:isp0:0:51:0): cam completion status 0x4a (probe52:isp0:0:52:0): cam completion status 0x4a (probe52:isp0:0:52:0): cam completion status 0x4a (probe53:isp0:0:53:0): cam completion status 0x4a (probe53:isp0:0:53:0): cam completion status 0x4a (probe54:isp0:0:54:0): cam completion status 0x4a (probe54:isp0:0:54:0): cam completion status 0x4a (probe55:isp0:0:55:0): cam completion status 0x4a (probe55:isp0:0:55:0): cam completion status 0x4a (probe56:isp0:0:56:0): cam completion status 0x4a (probe56:isp0:0:56:0): cam completion status 0x4a (probe57:isp0:0:57:0): cam completion status 0x4a (probe57:isp0:0:57:0): cam completion status 0x4a (probe58:isp0:0:58:0): cam completion status 0x4a (probe58:isp0:0:58:0): cam completion status 0x4a (probe59:isp0:0:59:0): cam completion status 0x4a (probe59:isp0:0:59:0): cam completion status 0x4a (probe60:isp0:0:60:0): cam completion status 0x4a (probe60:isp0:0:60:0): cam completion status 0x4a (probe61:isp0:0:61:0): cam completion status 0x4a (probe61:isp0:0:61:0): cam completion status 0x4a (probe63:isp0:0:63:0): cam completion status 0x4a (probe63:isp0:0:63:0): cam completion status 0x4a (probe64:isp0:0:64:0): cam completion status 0x4a (probe64:isp0:0:64:0): cam completion status 0x4a (probe65:isp0:0:65:0): cam completion status 0x4a (probe65:isp0:0:65:0): cam completion status 0x4a (probe66:isp0:0:66:0): cam completion status 0x4a (probe66:isp0:0:66:0): cam completion status 0x4a (probe67:isp0:0:67:0): cam completion status 0x4a (probe67:isp0:0:67:0): cam completion status 0x4a (probe68:isp0:0:68:0): cam completion status 0x4a (probe68:isp0:0:68:0): cam completion status 0x4a (probe69:isp0:0:69:0): cam completion status 0x4a (probe69:isp0:0:69:0): cam completion status 0x4a (probe70:isp0:0:70:0): cam completion status 0x4a (probe70:isp0:0:70:0): cam completion status 0x4a (probe71:isp0:0:71:0): cam completion status 0x4a (probe71:isp0:0:71:0): cam completion status 0x4a (probe72:isp0:0:72:0): cam completion status 0x4a (probe72:isp0:0:72:0): cam completion status 0x4a (probe73:isp0:0:73:0): cam completion status 0x4a (probe73:isp0:0:73:0): cam completion status 0x4a (probe75:isp0:0:75:0): cam completion status 0x4a (probe75:isp0:0:75:0): cam completion status 0x4a (probe76:isp0:0:76:0): cam completion status 0x4a (probe76:isp0:0:76:0): cam completion status 0x4a (probe77:isp0:0:77:0): cam completion status 0x4a (probe77:isp0:0:77:0): cam completion status 0x4a (probe78:isp0:0:78:0): cam completion status 0x4a (probe78:isp0:0:78:0): cam completion status 0x4a (probe79:isp0:0:79:0): cam completion status 0x4a (probe79:isp0:0:79:0): cam completion status 0x4a (probe81:isp0:0:81:0): cam completion status 0x4a (probe81:isp0:0:81:0): cam completion status 0x4a (probe82:isp0:0:82:0): cam completion status 0x4a (probe82:isp0:0:82:0): cam completion status 0x4a (probe83:isp0:0:83:0): cam completion status 0x4a (probe83:isp0:0:83:0): cam completion status 0x4a (probe84:isp0:0:84:0): cam completion status 0x4a (probe84:isp0:0:84:0): cam completion status 0x4a (probe85:isp0:0:85:0): cam completion status 0x4a (probe85:isp0:0:85:0): cam completion status 0x4a (probe86:isp0:0:86:0): cam completion status 0x4a (probe86:isp0:0:86:0): cam completion status 0x4a (probe87:isp0:0:87:0): cam completion status 0x4a (probe87:isp0:0:87:0): cam completion status 0x4a (probe88:isp0:0:88:0): cam completion status 0x4a (probe88:isp0:0:88:0): cam completion status 0x4a (probe89:isp0:0:89:0): cam completion status 0x4a (probe89:isp0:0:89:0): cam completion status 0x4a (probe90:isp0:0:90:0): cam completion status 0x4a (probe90:isp0:0:90:0): cam completion status 0x4a (probe91:isp0:0:91:0): cam completion status 0x4a (probe91:isp0:0:91:0): cam completion status 0x4a (probe92:isp0:0:92:0): cam completion status 0x4a (probe92:isp0:0:92:0): cam completion status 0x4a (probe93:isp0:0:93:0): cam completion status 0x4a (probe93:isp0:0:93:0): cam completion status 0x4a (probe94:isp0:0:94:0): cam completion status 0x4a (probe94:isp0:0:94:0): cam completion status 0x4a (probe95:isp0:0:95:0): cam completion status 0x4a (probe95:isp0:0:95:0): cam completion status 0x4a (probe96:isp0:0:96:0): cam completion status 0x4a (probe96:isp0:0:96:0): cam completion status 0x4a (probe97:isp0:0:97:0): cam completion status 0x4a (probe97:isp0:0:97:0): cam completion status 0x4a (probe98:isp0:0:98:0): cam completion status 0x4a (probe98:isp0:0:98:0): cam completion status 0x4a (probe99:isp0:0:99:0): cam completion status 0x4a (probe99:isp0:0:99:0): cam completion status 0x4a (probe100:isp0:0:100:0): cam completion status 0x4a (probe100:isp0:0:100:0): cam completion status 0x4a (probe101:isp0:0:101:0): cam completion status 0x4a (probe101:isp0:0:101:0): cam completion status 0x4a (probe102:isp0:0:102:0): cam completion status 0x4a (probe102:isp0:0:102:0): cam completion status 0x4a (probe103:isp0:0:103:0): cam completion status 0x4a (probe103:isp0:0:103:0): cam completion status 0x4a (probe104:isp0:0:104:0): cam completion status 0x4a (probe104:isp0:0:104:0): cam completion status 0x4a (probe105:isp0:0:105:0): cam completion status 0x4a (probe105:isp0:0:105:0): cam completion status 0x4a (probe106:isp0:0:106:0): cam completion status 0x4a (probe106:isp0:0:106:0): cam completion status 0x4a (probe107:isp0:0:107:0): cam completion status 0x4a (probe107:isp0:0:107:0): cam completion status 0x4a (probe108:isp0:0:108:0): cam completion status 0x4a (probe108:isp0:0:108:0): cam completion status 0x4a (probe110:isp0:0:110:0): cam completion status 0x4a (probe110:isp0:0:110:0): cam completion status 0x4a (probe111:isp0:0:111:0): cam completion status 0x4a (probe111:isp0:0:111:0): cam completion status 0x4a (probe112:isp0:0:112:0): cam completion status 0x4a (probe112:isp0:0:112:0): cam completion status 0x4a (probe113:isp0:0:113:0): cam completion status 0x4a (probe113:isp0:0:113:0): cam completion status 0x4a (probe114:isp0:0:114:0): cam completion status 0x4a (probe114:isp0:0:114:0): cam completion status 0x4a (probe115:isp0:0:115:0): cam completion status 0x4a (probe115:isp0:0:115:0): cam completion status 0x4a (probe116:isp0:0:116:0): cam completion status 0x4a (probe116:isp0:0:116:0): cam completion status 0x4a (probe117:isp0:0:117:0): cam completion status 0x4a (probe117:isp0:0:117:0): cam completion status 0x4a (probe118:isp0:0:118:0): cam completion status 0x4a (probe118:isp0:0:118:0): cam completion status 0x4a (probe119:isp0:0:119:0): cam completion status 0x4a (probe119:isp0:0:119:0): cam completion status 0x4a (probe120:isp0:0:120:0): cam completion status 0x4a (probe120:isp0:0:120:0): cam completion status 0x4a (probe121:isp0:0:121:0): cam completion status 0x4a (probe121:isp0:0:121:0): cam completion status 0x4a (probe122:isp0:0:122:0): cam completion status 0x4a (probe122:isp0:0:122:0): cam completion status 0x4a (probe123:isp0:0:123:0): cam completion status 0x4a (probe123:isp0:0:123:0): cam completion status 0x4a (probe124:isp0:0:124:0): cam completion status 0x4a (probe124:isp0:0:124:0): cam completion status 0x4a (probe125:isp0:0:125:0): cam completion status 0x4a (probe125:isp0:0:125:0): cam completion status 0x4a (probe126:isp0:0:126:0): cam completion status 0x4a (probe126:isp0:0:126:0): cam completion status 0x4a (probe127:isp0:0:127:0): cam completion status 0x4a (probe127:isp0:0:127:0): cam completion status 0x4a (probe128:isp0:0:128:0): cam completion status 0x4a (probe128:isp0:0:128:0): cam completion status 0x4a (probe129:isp0:0:129:0): cam completion status 0x4a (probe129:isp0:0:129:0): cam completion status 0x4a (probe130:isp0:0:130:0): cam completion status 0x4a (probe130:isp0:0:130:0): cam completion status 0x4a (probe132:isp0:0:132:0): cam completion status 0x4a (probe132:isp0:0:132:0): cam completion status 0x4a (probe133:isp0:0:133:0): cam completion status 0x4a (probe133:isp0:0:133:0): cam completion status 0x4a (probe134:isp0:0:134:0): cam completion status 0x4a (probe134:isp0:0:134:0): cam completion status 0x4a (probe135:isp0:0:135:0): cam completion status 0x4a (probe135:isp0:0:135:0): cam completion status 0x4a (probe136:isp0:0:136:0): cam completion status 0x4a (probe136:isp0:0:136:0): cam completion status 0x4a (probe137:isp0:0:137:0): cam completion status 0x4a (probe137:isp0:0:137:0): cam completion status 0x4a (probe138:isp0:0:138:0): cam completion status 0x4a (probe138:isp0:0:138:0): cam completion status 0x4a (probe139:isp0:0:139:0): cam completion status 0x4a (probe139:isp0:0:139:0): cam completion status 0x4a (probe140:isp0:0:140:0): cam completion status 0x4a (probe140:isp0:0:140:0): cam completion status 0x4a (probe141:isp0:0:141:0): cam completion status 0x4a (probe141:isp0:0:141:0): cam completion status 0x4a (probe142:isp0:0:142:0): cam completion status 0x4a (probe142:isp0:0:142:0): cam completion status 0x4a (probe143:isp0:0:143:0): cam completion status 0x4a (probe143:isp0:0:143:0): cam completion status 0x4a (probe144:isp0:0:144:0): cam completion status 0x4a (probe144:isp0:0:144:0): cam completion status 0x4a (probe145:isp0:0:145:0): cam completion status 0x4a (probe145:isp0:0:145:0): cam completion status 0x4a (probe146:isp0:0:146:0): cam completion status 0x4a (probe146:isp0:0:146:0): cam completion status 0x4a (probe148:isp0:0:148:0): cam completion status 0x4a (probe148:isp0:0:148:0): cam completion status 0x4a (probe149:isp0:0:149:0): cam completion status 0x4a (probe149:isp0:0:149:0): cam completion status 0x4a (probe150:isp0:0:150:0): cam completion status 0x4a (probe150:isp0:0:150:0): cam completion status 0x4a (probe151:isp0:0:151:0): cam completion status 0x4a (probe151:isp0:0:151:0): cam completion status 0x4a (probe152:isp0:0:152:0): cam completion status 0x4a (probe152:isp0:0:152:0): cam completion status 0x4a (probe153:isp0:0:153:0): cam completion status 0x4a (probe153:isp0:0:153:0): cam completion status 0x4a (probe154:isp0:0:154:0): cam completion status 0x4a (probe154:isp0:0:154:0): cam completion status 0x4a (probe155:isp0:0:155:0): cam completion status 0x4a (probe155:isp0:0:155:0): cam completion status 0x4a (probe156:isp0:0:156:0): cam completion status 0x4a (probe156:isp0:0:156:0): cam completion status 0x4a (probe157:isp0:0:157:0): cam completion status 0x4a (probe157:isp0:0:157:0): cam completion status 0x4a (probe158:isp0:0:158:0): cam completion status 0x4a (probe158:isp0:0:158:0): cam completion status 0x4a (probe159:isp0:0:159:0): cam completion status 0x4a (probe159:isp0:0:159:0): cam completion status 0x4a (probe160:isp0:0:160:0): cam completion status 0x4a (probe160:isp0:0:160:0): cam completion status 0x4a (probe161:isp0:0:161:0): cam completion status 0x4a (probe161:isp0:0:161:0): cam completion status 0x4a (probe162:isp0:0:162:0): cam completion status 0x4a (probe162:isp0:0:162:0): cam completion status 0x4a (probe163:isp0:0:163:0): cam completion status 0x4a (probe163:isp0:0:163:0): cam completion status 0x4a (probe164:isp0:0:164:0): cam completion status 0x4a (probe164:isp0:0:164:0): cam completion status 0x4a (probe165:isp0:0:165:0): cam completion status 0x4a (probe165:isp0:0:165:0): cam completion status 0x4a (probe166:isp0:0:166:0): cam completion status 0x4a (probe166:isp0:0:166:0): cam completion status 0x4a (probe167:isp0:0:167:0): cam completion status 0x4a (probe167:isp0:0:167:0): cam completion status 0x4a (probe168:isp0:0:168:0): cam completion status 0x4a (probe168:isp0:0:168:0): cam completion status 0x4a (probe169:isp0:0:169:0): cam completion status 0x4a (probe169:isp0:0:169:0): cam completion status 0x4a (probe170:isp0:0:170:0): cam completion status 0x4a (probe170:isp0:0:170:0): cam completion status 0x4a (probe171:isp0:0:171:0): cam completion status 0x4a (probe171:isp0:0:171:0): cam completion status 0x4a (probe172:isp0:0:172:0): cam completion status 0x4a (probe172:isp0:0:172:0): cam completion status 0x4a (probe173:isp0:0:173:0): cam completion status 0x4a (probe173:isp0:0:173:0): cam completion status 0x4a (probe174:isp0:0:174:0): cam completion status 0x4a (probe174:isp0:0:174:0): cam completion status 0x4a (probe175:isp0:0:175:0): cam completion status 0x4a (probe175:isp0:0:175:0): cam completion status 0x4a (probe176:isp0:0:176:0): cam completion status 0x4a (probe176:isp0:0:176:0): cam completion status 0x4a (probe177:isp0:0:177:0): cam completion status 0x4a (probe177:isp0:0:177:0): cam completion status 0x4a (probe178:isp0:0:178:0): cam completion status 0x4a (probe178:isp0:0:178:0): cam completion status 0x4a (probe179:isp0:0:179:0): cam completion status 0x4a (probe179:isp0:0:179:0): cam completion status 0x4a (probe180:isp0:0:180:0): cam completion status 0x4a (probe180:isp0:0:180:0): cam completion status 0x4a (probe181:isp0:0:181:0): cam completion status 0x4a (probe181:isp0:0:181:0): cam completion status 0x4a (probe182:isp0:0:182:0): cam completion status 0x4a (probe182:isp0:0:182:0): cam completion status 0x4a (probe183:isp0:0:183:0): cam completion status 0x4a (probe183:isp0:0:183:0): cam completion status 0x4a (probe184:isp0:0:184:0): cam completion status 0x4a (probe184:isp0:0:184:0): cam completion status 0x4a (probe185:isp0:0:185:0): cam completion status 0x4a (probe185:isp0:0:185:0): cam completion status 0x4a (probe186:isp0:0:186:0): cam completion status 0x4a (probe186:isp0:0:186:0): cam completion status 0x4a (probe187:isp0:0:187:0): cam completion status 0x4a (probe187:isp0:0:187:0): cam completion status 0x4a (probe188:isp0:0:188:0): cam completion status 0x4a (probe188:isp0:0:188:0): cam completion status 0x4a (probe189:isp0:0:189:0): cam completion status 0x4a (probe189:isp0:0:189:0): cam completion status 0x4a (probe190:isp0:0:190:0): cam completion status 0x4a (probe190:isp0:0:190:0): cam completion status 0x4a (probe191:isp0:0:191:0): cam completion status 0x4a (probe191:isp0:0:191:0): cam completion status 0x4a (probe192:isp0:0:192:0): cam completion status 0x4a (probe192:isp0:0:192:0): cam completion status 0x4a (probe193:isp0:0:193:0): cam completion status 0x4a (probe193:isp0:0:193:0): cam completion status 0x4a (probe194:isp0:0:194:0): cam completion status 0x4a (probe194:isp0:0:194:0): cam completion status 0x4a (probe195:isp0:0:195:0): cam completion status 0x4a (probe195:isp0:0:195:0): cam completion status 0x4a (probe196:isp0:0:196:0): cam completion status 0x4a (probe196:isp0:0:196:0): cam completion status 0x4a (probe197:isp0:0:197:0): cam completion status 0x4a (probe197:isp0:0:197:0): cam completion status 0x4a (probe198:isp0:0:198:0): cam completion status 0x4a (probe198:isp0:0:198:0): cam completion status 0x4a (probe199:isp0:0:199:0): cam completion status 0x4a (probe199:isp0:0:199:0): cam completion status 0x4a (probe200:isp0:0:200:0): cam completion status 0x4a (probe200:isp0:0:200:0): cam completion status 0x4a (probe201:isp0:0:201:0): cam completion status 0x4a (probe201:isp0:0:201:0): cam completion status 0x4a (probe202:isp0:0:202:0): cam completion status 0x4a (probe202:isp0:0:202:0): cam completion status 0x4a (probe203:isp0:0:203:0): cam completion status 0x4a (probe203:isp0:0:203:0): cam completion status 0x4a (probe204:isp0:0:204:0): cam completion status 0x4a (probe204:isp0:0:204:0): cam completion status 0x4a (probe205:isp0:0:205:0): cam completion status 0x4a (probe205:isp0:0:205:0): cam completion status 0x4a (probe207:isp0:0:207:0): cam completion status 0x4a (probe207:isp0:0:207:0): cam completion status 0x4a (probe208:isp0:0:208:0): cam completion status 0x4a (probe208:isp0:0:208:0): cam completion status 0x4a (probe209:isp0:0:209:0): cam completion status 0x4a (probe209:isp0:0:209:0): cam completion status 0x4a (probe210:isp0:0:210:0): cam completion status 0x4a (probe210:isp0:0:210:0): cam completion status 0x4a (probe211:isp0:0:211:0): cam completion status 0x4a (probe211:isp0:0:211:0): cam completion status 0x4a (probe212:isp0:0:212:0): cam completion status 0x4a (probe212:isp0:0:212:0): cam completion status 0x4a (probe213:isp0:0:213:0): cam completion status 0x4a (probe213:isp0:0:213:0): cam completion status 0x4a (probe214:isp0:0:214:0): cam completion status 0x4a (probe214:isp0:0:214:0): cam completion status 0x4a (probe215:isp0:0:215:0): cam completion status 0x4a (probe215:isp0:0:215:0): cam completion status 0x4a (probe216:isp0:0:216:0): cam completion status 0x4a (probe216:isp0:0:216:0): cam completion status 0x4a (probe217:isp0:0:217:0): cam completion status 0x4a (probe217:isp0:0:217:0): cam completion status 0x4a (probe218:isp0:0:218:0): cam completion status 0x4a (probe218:isp0:0:218:0): cam completion status 0x4a (probe219:isp0:0:219:0): cam completion status 0x4a (probe219:isp0:0:219:0): cam completion status 0x4a (probe220:isp0:0:220:0): cam completion status 0x4a (probe220:isp0:0:220:0): cam completion status 0x4a (probe221:isp0:0:221:0): cam completion status 0x4a (probe221:isp0:0:221:0): cam completion status 0x4a (probe222:isp0:0:222:0): cam completion status 0x4a (probe222:isp0:0:222:0): cam completion status 0x4a (probe223:isp0:0:223:0): cam completion status 0x4a (probe223:isp0:0:223:0): cam completion status 0x4a (probe224:isp0:0:224:0): cam completion status 0x4a (probe224:isp0:0:224:0): cam completion status 0x4a (probe225:isp0:0:225:0): cam completion status 0x4a (probe225:isp0:0:225:0): cam completion status 0x4a (probe226:isp0:0:226:0): cam completion status 0x4a (probe226:isp0:0:226:0): cam completion status 0x4a (probe227:isp0:0:227:0): cam completion status 0x4a (probe227:isp0:0:227:0): cam completion status 0x4a (probe228:isp0:0:228:0): cam completion status 0x4a (probe228:isp0:0:228:0): cam completion status 0x4a (probe229:isp0:0:229:0): cam completion status 0x4a (probe229:isp0:0:229:0): cam completion status 0x4a (probe230:isp0:0:230:0): cam completion status 0x4a (probe230:isp0:0:230:0): cam completion status 0x4a (probe231:isp0:0:231:0): cam completion status 0x4a (probe231:isp0:0:231:0): cam completion status 0x4a (probe232:isp0:0:232:0): cam completion status 0x4a (probe232:isp0:0:232:0): cam completion status 0x4a (probe233:isp0:0:233:0): cam completion status 0x4a (probe233:isp0:0:233:0): cam completion status 0x4a (probe234:isp0:0:234:0): cam completion status 0x4a (probe234:isp0:0:234:0): cam completion status 0x4a (probe235:isp0:0:235:0): cam completion status 0x4a (probe235:isp0:0:235:0): cam completion status 0x4a (probe236:isp0:0:236:0): cam completion status 0x4a (probe236:isp0:0:236:0): cam completion status 0x4a (probe237:isp0:0:237:0): cam completion status 0x4a (probe237:isp0:0:237:0): cam completion status 0x4a (probe238:isp0:0:238:0): cam completion status 0x4a (probe238:isp0:0:238:0): cam completion status 0x4a (probe239:isp0:0:239:0): cam completion status 0x4a (probe239:isp0:0:239:0): cam completion status 0x4a (probe240:isp0:0:240:0): cam completion status 0x4a (probe240:isp0:0:240:0): cam completion status 0x4a (probe241:isp0:0:241:0): cam completion status 0x4a (probe241:isp0:0:241:0): cam completion status 0x4a (probe242:isp0:0:242:0): cam completion status 0x4a (probe242:isp0:0:242:0): cam completion status 0x4a (probe243:isp0:0:243:0): cam completion status 0x4a (probe243:isp0:0:243:0): cam completion status 0x4a (probe244:isp0:0:244:0): cam completion status 0x4a (probe244:isp0:0:244:0): cam completion status 0x4a (probe245:isp0:0:245:0): cam completion status 0x4a (probe245:isp0:0:245:0): cam completion status 0x4a (probe246:isp0:0:246:0): cam completion status 0x4a (probe246:isp0:0:246:0): cam completion status 0x4a (probe247:isp0:0:247:0): cam completion status 0x4a (probe247:isp0:0:247:0): cam completion status 0x4a (probe248:isp0:0:248:0): cam completion status 0x4a (probe248:isp0:0:248:0): cam completion status 0x4a (probe249:isp0:0:249:0): cam completion status 0x4a (probe249:isp0:0:249:0): cam completion status 0x4a (probe250:isp0:0:250:0): cam completion status 0x4a (probe250:isp0:0:250:0): cam completion status 0x4a (probe251:isp0:0:251:0): cam completion status 0x4a (probe251:isp0:0:251:0): cam completion status 0x4a (probe252:isp0:0:252:0): cam completion status 0x4a (probe252:isp0:0:252:0): cam completion status 0x4a (probe253:isp0:0:253:0): cam completion status 0x4a (probe253:isp0:0:253:0): cam completion status 0x4a (probe254:isp0:0:254:0): cam completion status 0x4a (probe254:isp0:0:254:0): cam completion status 0x4a (probe255:isp0:0:255:0): cam completion status 0x4a (probe255:isp0:0:255:0): cam completion status 0x4a (probe256:isp0:0:256:0): cam completion status 0x4a (probe256:isp0:0:256:0): cam completion status 0x4a (probe257:isp0:0:257:0): cam completion status 0x4a (probe257:isp0:0:257:0): cam completion status 0x4a (probe258:isp0:0:258:0): cam completion status 0x4a (probe258:isp0:0:258:0): cam completion status 0x4a (probe259:isp0:0:259:0): cam completion status 0x4a (probe259:isp0:0:259:0): cam completion status 0x4a (probe260:isp0:0:260:0): cam completion status 0x4a (probe260:isp0:0:260:0): cam completion status 0x4a (probe261:isp0:0:261:0): cam completion status 0x4a (probe261:isp0:0:261:0): cam completion status 0x4a (probe262:isp0:0:262:0): cam completion status 0x4a (probe262:isp0:0:262:0): cam completion status 0x4a (probe263:isp0:0:263:0): cam completion status 0x4a (probe263:isp0:0:263:0): cam completion status 0x4a (probe264:isp0:0:264:0): cam completion status 0x4a (probe264:isp0:0:264:0): cam completion status 0x4a (probe265:isp0:0:265:0): cam completion status 0x4a (probe265:isp0:0:265:0): cam completion status 0x4a (probe266:isp0:0:266:0): cam completion status 0x4a (probe266:isp0:0:266:0): cam completion status 0x4a (probe267:isp0:0:267:0): cam completion status 0x4a (probe267:isp0:0:267:0): cam completion status 0x4a (probe268:isp0:0:268:0): cam completion status 0x4a (probe268:isp0:0:268:0): cam completion status 0x4a (probe269:isp0:0:269:0): cam completion status 0x4a (probe269:isp0:0:269:0): cam completion status 0x4a (probe270:isp0:0:270:0): cam completion status 0x4a (probe270:isp0:0:270:0): cam completion status 0x4a (probe271:isp0:0:271:0): cam completion status 0x4a (probe271:isp0:0:271:0): cam completion status 0x4a (probe272:isp0:0:272:0): cam completion status 0x4a (probe272:isp0:0:272:0): cam completion status 0x4a (probe273:isp0:0:273:0): cam completion status 0x4a (probe273:isp0:0:273:0): cam completion status 0x4a (probe274:isp0:0:274:0): cam completion status 0x4a (probe274:isp0:0:274:0): cam completion status 0x4a (probe275:isp0:0:275:0): cam completion status 0x4a (probe275:isp0:0:275:0): cam completion status 0x4a (probe276:isp0:0:276:0): cam completion status 0x4a (probe276:isp0:0:276:0): cam completion status 0x4a (probe277:isp0:0:277:0): cam completion status 0x4a (probe277:isp0:0:277:0): cam completion status 0x4a (probe278:isp0:0:278:0): cam completion status 0x4a (probe278:isp0:0:278:0): cam completion status 0x4a (probe279:isp0:0:279:0): cam completion status 0x4a (probe279:isp0:0:279:0): cam completion status 0x4a (probe280:isp0:0:280:0): cam completion status 0x4a (probe280:isp0:0:280:0): cam completion status 0x4a (probe281:isp0:0:281:0): cam completion status 0x4a (probe281:isp0:0:281:0): cam completion status 0x4a (probe282:isp0:0:282:0): cam completion status 0x4a (probe282:isp0:0:282:0): cam completion status 0x4a (probe283:isp0:0:283:0): cam completion status 0x4a (probe283:isp0:0:283:0): cam completion status 0x4a (probe284:isp0:0:284:0): cam completion status 0x4a (probe284:isp0:0:284:0): cam completion status 0x4a (probe285:isp0:0:285:0): cam completion status 0x4a (probe285:isp0:0:285:0): cam completion status 0x4a (probe286:isp0:0:286:0): cam completion status 0x4a (probe286:isp0:0:286:0): cam completion status 0x4a (probe287:isp0:0:287:0): cam completion status 0x4a (probe287:isp0:0:287:0): cam completion status 0x4a (probe288:isp0:0:288:0): cam completion status 0x4a (probe288:isp0:0:288:0): cam completion status 0x4a (probe289:isp0:0:289:0): cam completion status 0x4a (probe289:isp0:0:289:0): cam completion status 0x4a (probe290:isp0:0:290:0): cam completion status 0x4a (probe290:isp0:0:290:0): cam completion status 0x4a (probe291:isp0:0:291:0): cam completion status 0x4a (probe291:isp0:0:291:0): cam completion status 0x4a (probe292:isp0:0:292:0): cam completion status 0x4a (probe292:isp0:0:292:0): cam completion status 0x4a (probe293:isp0:0:293:0): cam completion status 0x4a (probe293:isp0:0:293:0): cam completion status 0x4a (probe294:isp0:0:294:0): cam completion status 0x4a (probe294:isp0:0:294:0): cam completion status 0x4a (probe295:isp0:0:295:0): cam completion status 0x4a (probe295:isp0:0:295:0): cam completion status 0x4a (probe296:isp0:0:296:0): cam completion status 0x4a (probe296:isp0:0:296:0): cam completion status 0x4a (probe297:isp0:0:297:0): cam completion status 0x4a (probe297:isp0:0:297:0): cam completion status 0x4a (probe298:isp0:0:298:0): cam completion status 0x4a (probe298:isp0:0:298:0): cam completion status 0x4a (probe299:isp0:0:299:0): cam completion status 0x4a (probe299:isp0:0:299:0): cam completion status 0x4a (probe300:isp0:0:300:0): cam completion status 0x4a (probe300:isp0:0:300:0): cam completion status 0x4a (probe301:isp0:0:301:0): cam completion status 0x4a (probe301:isp0:0:301:0): cam completion status 0x4a (probe302:isp0:0:302:0): cam completion status 0x4a (probe302:isp0:0:302:0): cam completion status 0x4a (probe303:isp0:0:303:0): cam completion status 0x4a (probe303:isp0:0:303:0): cam completion status 0x4a (probe304:isp0:0:304:0): cam completion status 0x4a (probe304:isp0:0:304:0): cam completion status 0x4a (probe305:isp0:0:305:0): cam completion status 0x4a (probe305:isp0:0:305:0): cam completion status 0x4a (probe306:isp0:0:306:0): cam completion status 0x4a (probe306:isp0:0:306:0): cam completion status 0x4a (probe307:isp0:0:307:0): cam completion status 0x4a (probe307:isp0:0:307:0): cam completion status 0x4a (probe308:isp0:0:308:0): cam completion status 0x4a (probe308:isp0:0:308:0): cam completion status 0x4a (probe309:isp0:0:309:0): cam completion status 0x4a (probe309:isp0:0:309:0): cam completion status 0x4a (probe310:isp0:0:310:0): cam completion status 0x4a (probe310:isp0:0:310:0): cam completion status 0x4a (probe311:isp0:0:311:0): cam completion status 0x4a (probe311:isp0:0:311:0): cam completion status 0x4a (probe312:isp0:0:312:0): cam completion status 0x4a (probe312:isp0:0:312:0): cam completion status 0x4a (probe313:isp0:0:313:0): cam completion status 0x4a (probe313:isp0:0:313:0): cam completion status 0x4a (probe314:isp0:0:314:0): cam completion status 0x4a (probe314:isp0:0:314:0): cam completion status 0x4a (probe315:isp0:0:315:0): cam completion status 0x4a (probe315:isp0:0:315:0): cam completion status 0x4a (probe316:isp0:0:316:0): cam completion status 0x4a (probe316:isp0:0:316:0): cam completion status 0x4a (probe317:isp0:0:317:0): cam completion status 0x4a (probe317:isp0:0:317:0): cam completion status 0x4a (probe318:isp0:0:318:0): cam completion status 0x4a (probe318:isp0:0:318:0): cam completion status 0x4a (probe319:isp0:0:319:0): cam completion status 0x4a (probe319:isp0:0:319:0): cam completion status 0x4a (probe320:isp0:0:320:0): cam completion status 0x4a (probe320:isp0:0:320:0): cam completion status 0x4a (probe321:isp0:0:321:0): cam completion status 0x4a (probe321:isp0:0:321:0): cam completion status 0x4a (probe322:isp0:0:322:0): cam completion status 0x4a (probe322:isp0:0:322:0): cam completion status 0x4a (probe323:isp0:0:323:0): cam completion status 0x4a (probe323:isp0:0:323:0): cam completion status 0x4a (probe324:isp0:0:324:0): cam completion status 0x4a (probe324:isp0:0:324:0): cam completion status 0x4a (probe325:isp0:0:325:0): cam completion status 0x4a (probe325:isp0:0:325:0): cam completion status 0x4a (probe326:isp0:0:326:0): cam completion status 0x4a (probe326:isp0:0:326:0): cam completion status 0x4a (probe327:isp0:0:327:0): cam completion status 0x4a (probe327:isp0:0:327:0): cam completion status 0x4a (probe328:isp0:0:328:0): cam completion status 0x4a (probe328:isp0:0:328:0): cam completion status 0x4a (probe329:isp0:0:329:0): cam completion status 0x4a (probe329:isp0:0:329:0): cam completion status 0x4a (probe330:isp0:0:330:0): cam completion status 0x4a (probe330:isp0:0:330:0): cam completion status 0x4a (probe331:isp0:0:331:0): cam completion status 0x4a (probe331:isp0:0:331:0): cam completion status 0x4a (probe332:isp0:0:332:0): cam completion status 0x4a (probe332:isp0:0:332:0): cam completion status 0x4a (probe333:isp0:0:333:0): cam completion status 0x4a (probe333:isp0:0:333:0): cam completion status 0x4a (probe334:isp0:0:334:0): cam completion status 0x4a (probe334:isp0:0:334:0): cam completion status 0x4a (probe335:isp0:0:335:0): cam completion status 0x4a (probe335:isp0:0:335:0): cam completion status 0x4a (probe336:isp0:0:336:0): cam completion status 0x4a (probe336:isp0:0:336:0): cam completion status 0x4a (probe337:isp0:0:337:0): cam completion status 0x4a (probe337:isp0:0:337:0): cam completion status 0x4a (probe338:isp0:0:338:0): cam completion status 0x4a (probe338:isp0:0:338:0): cam completion status 0x4a (probe339:isp0:0:339:0): cam completion status 0x4a (probe339:isp0:0:339:0): cam completion status 0x4a (probe340:isp0:0:340:0): cam completion status 0x4a (probe340:isp0:0:340:0): cam completion status 0x4a (probe341:isp0:0:341:0): cam completion status 0x4a (probe341:isp0:0:341:0): cam completion status 0x4a (probe342:isp0:0:342:0): cam completion status 0x4a (probe342:isp0:0:342:0): cam completion status 0x4a (probe343:isp0:0:343:0): cam completion status 0x4a (probe343:isp0:0:343:0): cam completion status 0x4a (probe344:isp0:0:344:0): cam completion status 0x4a (probe344:isp0:0:344:0): cam completion status 0x4a (probe345:isp0:0:345:0): cam completion status 0x4a (probe345:isp0:0:345:0): cam completion status 0x4a (probe346:isp0:0:346:0): cam completion status 0x4a (probe346:isp0:0:346:0): cam completion status 0x4a (probe347:isp0:0:347:0): cam completion status 0x4a (probe347:isp0:0:347:0): cam completion status 0x4a (probe348:isp0:0:348:0): cam completion status 0x4a (probe348:isp0:0:348:0): cam completion status 0x4a (probe349:isp0:0:349:0): cam completion status 0x4a (probe349:isp0:0:349:0): cam completion status 0x4a (probe350:isp0:0:350:0): cam completion status 0x4a (probe350:isp0:0:350:0): cam completion status 0x4a (probe351:isp0:0:351:0): cam completion status 0x4a (probe351:isp0:0:351:0): cam completion status 0x4a (probe352:isp0:0:352:0): cam completion status 0x4a (probe352:isp0:0:352:0): cam completion status 0x4a (probe353:isp0:0:353:0): cam completion status 0x4a (probe353:isp0:0:353:0): cam completion status 0x4a (probe354:isp0:0:354:0): cam completion status 0x4a (probe354:isp0:0:354:0): cam completion status 0x4a (probe355:isp0:0:355:0): cam completion status 0x4a (probe355:isp0:0:355:0): cam completion status 0x4a (probe356:isp0:0:356:0): cam completion status 0x4a (probe356:isp0:0:356:0): cam completion status 0x4a (probe357:isp0:0:357:0): cam completion status 0x4a (probe357:isp0:0:357:0): cam completion status 0x4a (probe358:isp0:0:358:0): cam completion status 0x4a (probe358:isp0:0:358:0): cam completion status 0x4a (probe359:isp0:0:359:0): cam completion status 0x4a (probe359:isp0:0:359:0): cam completion status 0x4a (probe360:isp0:0:360:0): cam completion status 0x4a (probe360:isp0:0:360:0): cam completion status 0x4a (probe361:isp0:0:361:0): cam completion status 0x4a (probe361:isp0:0:361:0): cam completion status 0x4a (probe362:isp0:0:362:0): cam completion status 0x4a (probe362:isp0:0:362:0): cam completion status 0x4a (probe363:isp0:0:363:0): cam completion status 0x4a (probe363:isp0:0:363:0): cam completion status 0x4a (probe364:isp0:0:364:0): cam completion status 0x4a (probe364:isp0:0:364:0): cam completion status 0x4a (probe365:isp0:0:365:0): cam completion status 0x4a (probe365:isp0:0:365:0): cam completion status 0x4a (probe366:isp0:0:366:0): cam completion status 0x4a (probe366:isp0:0:366:0): cam completion status 0x4a (probe367:isp0:0:367:0): cam completion status 0x4a (probe367:isp0:0:367:0): cam completion status 0x4a (probe368:isp0:0:368:0): cam completion status 0x4a (probe368:isp0:0:368:0): cam completion status 0x4a (probe369:isp0:0:369:0): cam completion status 0x4a (probe369:isp0:0:369:0): cam completion status 0x4a (probe370:isp0:0:370:0): cam completion status 0x4a (probe370:isp0:0:370:0): cam completion status 0x4a (probe371:isp0:0:371:0): cam completion status 0x4a (probe371:isp0:0:371:0): cam completion status 0x4a (probe372:isp0:0:372:0): cam completion status 0x4a (probe372:isp0:0:372:0): cam completion status 0x4a (probe373:isp0:0:373:0): cam completion status 0x4a (probe373:isp0:0:373:0): cam completion status 0x4a (probe374:isp0:0:374:0): cam completion status 0x4a (probe374:isp0:0:374:0): cam completion status 0x4a (probe375:isp0:0:375:0): cam completion status 0x4a (probe375:isp0:0:375:0): cam completion status 0x4a (probe377:isp0:0:377:0): cam completion status 0x4a (probe377:isp0:0:377:0): cam completion status 0x4a (probe378:isp0:0:378:0): cam completion status 0x4a (probe378:isp0:0:378:0): cam completion status 0x4a (probe379:isp0:0:379:0): cam completion status 0x4a (probe379:isp0:0:379:0): cam completion status 0x4a (probe380:isp0:0:380:0): cam completion status 0x4a (probe380:isp0:0:380:0): cam completion status 0x4a (probe381:isp0:0:381:0): cam completion status 0x4a (probe381:isp0:0:381:0): cam completion status 0x4a (probe382:isp0:0:382:0): cam completion status 0x4a (probe382:isp0:0:382:0): cam completion status 0x4a (probe383:isp0:0:383:0): cam completion status 0x4a (probe383:isp0:0:383:0): cam completion status 0x4a (probe384:isp0:0:384:0): cam completion status 0x4a (probe384:isp0:0:384:0): cam completion status 0x4a (probe385:isp0:0:385:0): cam completion status 0x4a (probe385:isp0:0:385:0): cam completion status 0x4a (probe386:isp0:0:386:0): cam completion status 0x4a (probe386:isp0:0:386:0): cam completion status 0x4a (probe387:isp0:0:387:0): cam completion status 0x4a (probe387:isp0:0:387:0): cam completion status 0x4a (probe388:isp0:0:388:0): cam completion status 0x4a (probe388:isp0:0:388:0): cam completion status 0x4a (probe389:isp0:0:389:0): cam completion status 0x4a (probe389:isp0:0:389:0): cam completion status 0x4a (probe390:isp0:0:390:0): cam completion status 0x4a (probe390:isp0:0:390:0): cam completion status 0x4a (probe391:isp0:0:391:0): cam completion status 0x4a (probe391:isp0:0:391:0): cam completion status 0x4a (probe392:isp0:0:392:0): cam completion status 0x4a (probe392:isp0:0:392:0): cam completion status 0x4a (probe393:isp0:0:393:0): cam completion status 0x4a (probe393:isp0:0:393:0): cam completion status 0x4a (probe394:isp0:0:394:0): cam completion status 0x4a (probe394:isp0:0:394:0): cam completion status 0x4a (probe396:isp0:0:396:0): cam completion status 0x4a (probe396:isp0:0:396:0): cam completion status 0x4a (probe397:isp0:0:397:0): cam completion status 0x4a (probe397:isp0:0:397:0): cam completion status 0x4a (probe398:isp0:0:398:0): cam completion status 0x4a (probe398:isp0:0:398:0): cam completion status 0x4a (probe399:isp0:0:399:0): cam completion status 0x4a (probe399:isp0:0:399:0): cam completion status 0x4a (probe401:isp0:0:401:0): cam completion status 0x4a (probe401:isp0:0:401:0): cam completion status 0x4a (probe402:isp0:0:402:0): cam completion status 0x4a (probe402:isp0:0:402:0): cam completion status 0x4a (probe403:isp0:0:403:0): cam completion status 0x4a (probe403:isp0:0:403:0): cam completion status 0x4a (probe404:isp0:0:404:0): cam completion status 0x4a (probe404:isp0:0:404:0): cam completion status 0x4a (probe405:isp0:0:405:0): cam completion status 0x4a (probe405:isp0:0:405:0): cam completion status 0x4a (probe406:isp0:0:406:0): cam completion status 0x4a (probe406:isp0:0:406:0): cam completion status 0x4a (probe407:isp0:0:407:0): cam completion status 0x4a (probe407:isp0:0:407:0): cam completion status 0x4a (probe408:isp0:0:408:0): cam completion status 0x4a (probe408:isp0:0:408:0): cam completion status 0x4a (probe409:isp0:0:409:0): cam completion status 0x4a (probe409:isp0:0:409:0): cam completion status 0x4a (probe410:isp0:0:410:0): cam completion status 0x4a (probe410:isp0:0:410:0): cam completion status 0x4a (probe412:isp0:0:412:0): cam completion status 0x4a (probe412:isp0:0:412:0): cam completion status 0x4a (probe413:isp0:0:413:0): cam completion status 0x4a (probe413:isp0:0:413:0): cam completion status 0x4a (probe414:isp0:0:414:0): cam completion status 0x4a (probe414:isp0:0:414:0): cam completion status 0x4a (probe415:isp0:0:415:0): cam completion status 0x4a (probe415:isp0:0:415:0): cam completion status 0x4a (probe416:isp0:0:416:0): cam completion status 0x4a (probe416:isp0:0:416:0): cam completion status 0x4a (probe417:isp0:0:417:0): cam completion status 0x4a (probe417:isp0:0:417:0): cam completion status 0x4a (probe418:isp0:0:418:0): cam completion status 0x4a (probe418:isp0:0:418:0): cam completion status 0x4a (probe419:isp0:0:419:0): cam completion status 0x4a (probe419:isp0:0:419:0): cam completion status 0x4a (probe420:isp0:0:420:0): cam completion status 0x4a (probe420:isp0:0:420:0): cam completion status 0x4a (probe421:isp0:0:421:0): cam completion status 0x4a (probe421:isp0:0:421:0): cam completion status 0x4a (probe422:isp0:0:422:0): cam completion status 0x4a (probe422:isp0:0:422:0): cam completion status 0x4a (probe424:isp0:0:424:0): cam completion status 0x4a (probe424:isp0:0:424:0): cam completion status 0x4a (probe425:isp0:0:425:0): cam completion status 0x4a (probe425:isp0:0:425:0): cam completion status 0x4a (probe426:isp0:0:426:0): cam completion status 0x4a (probe426:isp0:0:426:0): cam completion status 0x4a (probe427:isp0:0:427:0): cam completion status 0x4a (probe427:isp0:0:427:0): cam completion status 0x4a (probe428:isp0:0:428:0): cam completion status 0x4a (probe428:isp0:0:428:0): cam completion status 0x4a (probe429:isp0:0:429:0): cam completion status 0x4a (probe429:isp0:0:429:0): cam completion status 0x4a (probe430:isp0:0:430:0): cam completion status 0x4a (probe430:isp0:0:430:0): cam completion status 0x4a (probe431:isp0:0:431:0): cam completion status 0x4a (probe431:isp0:0:431:0): cam completion status 0x4a (probe432:isp0:0:432:0): cam completion status 0x4a (probe432:isp0:0:432:0): cam completion status 0x4a (probe433:isp0:0:433:0): cam completion status 0x4a (probe433:isp0:0:433:0): cam completion status 0x4a (probe434:isp0:0:434:0): cam completion status 0x4a (probe434:isp0:0:434:0): cam completion status 0x4a (probe435:isp0:0:435:0): cam completion status 0x4a (probe435:isp0:0:435:0): cam completion status 0x4a (probe436:isp0:0:436:0): cam completion status 0x4a (probe436:isp0:0:436:0): cam completion status 0x4a (probe438:isp0:0:438:0): cam completion status 0x4a (probe438:isp0:0:438:0): cam completion status 0x4a (probe439:isp0:0:439:0): cam completion status 0x4a (probe439:isp0:0:439:0): cam completion status 0x4a (probe440:isp0:0:440:0): cam completion status 0x4a (probe440:isp0:0:440:0): cam completion status 0x4a (probe441:isp0:0:441:0): cam completion status 0x4a (probe441:isp0:0:441:0): cam completion status 0x4a (probe442:isp0:0:442:0): cam completion status 0x4a (probe442:isp0:0:442:0): cam completion status 0x4a (probe443:isp0:0:443:0): cam completion status 0x4a (probe443:isp0:0:443:0): cam completion status 0x4a (probe444:isp0:0:444:0): cam completion status 0x4a (probe444:isp0:0:444:0): cam completion status 0x4a (probe445:isp0:0:445:0): cam completion status 0x4a (probe445:isp0:0:445:0): cam completion status 0x4a (probe446:isp0:0:446:0): cam completion status 0x4a (probe446:isp0:0:446:0): cam completion status 0x4a (probe447:isp0:0:447:0): cam completion status 0x4a (probe447:isp0:0:447:0): cam completion status 0x4a (probe448:isp0:0:448:0): cam completion status 0x4a (probe448:isp0:0:448:0): cam completion status 0x4a (probe450:isp0:0:450:0): cam completion status 0x4a (probe450:isp0:0:450:0): cam completion status 0x4a (probe451:isp0:0:451:0): cam completion status 0x4a (probe451:isp0:0:451:0): cam completion status 0x4a (probe452:isp0:0:452:0): cam completion status 0x4a (probe452:isp0:0:452:0): cam completion status 0x4a (probe453:isp0:0:453:0): cam completion status 0x4a (probe453:isp0:0:453:0): cam completion status 0x4a (probe454:isp0:0:454:0): cam completion status 0x4a (probe454:isp0:0:454:0): cam completion status 0x4a (probe455:isp0:0:455:0): cam completion status 0x4a (probe455:isp0:0:455:0): cam completion status 0x4a (probe456:isp0:0:456:0): cam completion status 0x4a (probe456:isp0:0:456:0): cam completion status 0x4a (probe457:isp0:0:457:0): cam completion status 0x4a (probe457:isp0:0:457:0): cam completion status 0x4a (probe458:isp0:0:458:0): cam completion status 0x4a (probe458:isp0:0:458:0): cam completion status 0x4a (probe460:isp0:0:460:0): cam completion status 0x4a (probe460:isp0:0:460:0): cam completion status 0x4a (probe461:isp0:0:461:0): cam completion status 0x4a (probe461:isp0:0:461:0): cam completion status 0x4a (probe462:isp0:0:462:0): cam completion status 0x4a (probe462:isp0:0:462:0): cam completion status 0x4a (probe463:isp0:0:463:0): cam completion status 0x4a (probe463:isp0:0:463:0): cam completion status 0x4a (probe465:isp0:0:465:0): cam completion status 0x4a (probe465:isp0:0:465:0): cam completion status 0x4a (probe466:isp0:0:466:0): cam completion status 0x4a (probe466:isp0:0:466:0): cam completion status 0x4a (probe467:isp0:0:467:0): cam completion status 0x4a (probe467:isp0:0:467:0): cam completion status 0x4a (probe468:isp0:0:468:0): cam completion status 0x4a (probe468:isp0:0:468:0): cam completion status 0x4a (probe469:isp0:0:469:0): cam completion status 0x4a (probe469:isp0:0:469:0): cam completion status 0x4a (probe470:isp0:0:470:0): cam completion status 0x4a (probe470:isp0:0:470:0): cam completion status 0x4a (probe472:isp0:0:472:0): cam completion status 0x4a (probe472:isp0:0:472:0): cam completion status 0x4a (probe473:isp0:0:473:0): cam completion status 0x4a (probe473:isp0:0:473:0): cam completion status 0x4a (probe474:isp0:0:474:0): cam completion status 0x4a (probe474:isp0:0:474:0): cam completion status 0x4a (probe475:isp0:0:475:0): cam completion status 0x4a (probe475:isp0:0:475:0): cam completion status 0x4a (probe476:isp0:0:476:0): cam completion status 0x4a (probe476:isp0:0:476:0): cam completion status 0x4a (probe477:isp0:0:477:0): cam completion status 0x4a (probe477:isp0:0:477:0): cam completion status 0x4a (probe479:isp0:0:479:0): cam completion status 0x4a (probe479:isp0:0:479:0): cam completion status 0x4a (probe480:isp0:0:480:0): cam completion status 0x4a (probe480:isp0:0:480:0): cam completion status 0x4a (probe481:isp0:0:481:0): cam completion status 0x4a (probe481:isp0:0:481:0): cam completion status 0x4a (probe482:isp0:0:482:0): cam completion status 0x4a (probe482:isp0:0:482:0): cam completion status 0x4a (probe483:isp0:0:483:0): cam completion status 0x4a (probe483:isp0:0:483:0): cam completion status 0x4a (probe484:isp0:0:484:0): cam completion status 0x4a (probe484:isp0:0:484:0): cam completion status 0x4a (probe485:isp0:0:485:0): cam completion status 0x4a (probe485:isp0:0:485:0): cam completion status 0x4a (probe486:isp0:0:486:0): cam completion status 0x4a (probe486:isp0:0:486:0): cam completion status 0x4a (probe487:isp0:0:487:0): cam completion status 0x4a (probe487:isp0:0:487:0): cam completion status 0x4a (probe489:isp0:0:489:0): cam completion status 0x4a (probe489:isp0:0:489:0): cam completion status 0x4a (probe490:isp0:0:490:0): cam completion status 0x4a (probe490:isp0:0:490:0): cam completion status 0x4a (probe491:isp0:0:491:0): cam completion status 0x4a (probe491:isp0:0:491:0): cam completion status 0x4a (probe492:isp0:0:492:0): cam completion status 0x4a (probe492:isp0:0:492:0): cam completion status 0x4a (probe493:isp0:0:493:0): cam completion status 0x4a (probe493:isp0:0:493:0): cam completion status 0x4a (probe494:isp0:0:494:0): cam completion status 0x4a (probe494:isp0:0:494:0): cam completion status 0x4a (probe495:isp0:0:495:0): cam completion status 0x4a (probe495:isp0:0:495:0): cam completion status 0x4a (probe496:isp0:0:496:0): cam completion status 0x4a (probe496:isp0:0:496:0): cam completion status 0x4a (probe497:isp0:0:497:0): cam completion status 0x4a (probe497:isp0:0:497:0): cam completion status 0x4a (probe498:isp0:0:498:0): cam completion status 0x4a (probe498:isp0:0:498:0): cam completion status 0x4a (probe499:isp0:0:499:0): cam completion status 0x4a (probe499:isp0:0:499:0): cam completion status 0x4a (probe500:isp0:0:500:0): cam completion status 0x4a (probe500:isp0:0:500:0): cam completion status 0x4a (probe501:isp0:0:501:0): cam completion status 0x4a (probe501:isp0:0:501:0): cam completion status 0x4a (probe502:isp0:0:502:0): cam completion status 0x4a (probe502:isp0:0:502:0): cam completion status 0x4a (probe503:isp0:0:503:0): cam completion status 0x4a (probe503:isp0:0:503:0): cam completion status 0x4a (probe6:isp0:0:6:0): cam completion status 0x4a (probe6:isp0:0:6:0): cam completion status 0x4a (probe25:isp0:0:25:0): cam completion status 0x4a (probe25:isp0:0:25:0): cam completion status 0x4a (probe62:isp0:0:62:0): cam completion status 0x4a (probe62:isp0:0:62:0): cam completion status 0x4a (probe74:isp0:0:74:0): cam completion status 0x4a (probe74:isp0:0:74:0): cam completion status 0x4a (probe80:isp0:0:80:0): cam completion status 0x4a (probe80:isp0:0:80:0): cam completion status 0x4a (probe109:isp0:0:109:0): cam completion status 0x4a (probe109:isp0:0:109:0): cam completion status 0x4a (probe131:isp0:0:131:0): cam completion status 0x4a (probe131:isp0:0:131:0): cam completion status 0x4a (probe147:isp0:0:147:0): cam completion status 0x4a (probe147:isp0:0:147:0): cam completion status 0x4a (probe206:isp0:0:206:0): cam completion status 0x4a (probe206:isp0:0:206:0): cam completion status 0x4a da0 at isp0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-4 device da0: 100.000MB/s transfers da0: Command Queueing Enabled da0: 512000MB (1048576000 512 byte sectors: 255H 63S/T 65270C) cd0 at umass-sim0 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 40.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. da1 at umass-sim1 bus 1 target 0 lun 0 da1: Removable Direct Access SCSI-0 device da1: 40.000MB/s transfers da1: Attempt to query device size failed: NOT READY, Medium not present Trying to mount root from ufs:/dev/mfid0s1a bce0: link state changed to UP mfi0: 3320 (277345023s/0x0008/info) - Battery charge complete --------------------------------------------------------------- 4. panic data and ddb output, also available at http://noc.ntua.gr/~christia/san/7-STABLE.deadlock-ddb.out : KKDDpanic: ufs_dirbad: /a: bad dir ino 3370734 at ofBBfset 0: mangled entry cpuid = 1 KDB: enter: panic [thread pid 768 tid 100104 ] Stopped at kdb_enter_why+0x3a: movl $0,kdb_why db> trace Tracing pid 768 tid 100104 td 0xc74c6690 kdb_enter_why(c0b3b0a1,c0b3b0a1,c0b5d0ec,ed2dd85c,1,...) at kdb_enter_why+0x3a panic(c0b5d0ec,c70365c0,336eee,0,c0b5d164,...) at panic+0x136 ufs_dirbad(c9c40180,0,c0b5d164,0,ed2dd904,...) at ufs_dirbad+0x49 ufs_lookup(ed2dd94c,c0b6f08d,cb090160,cb090160,ed2ddba4,...) at ufs_lookup+0x377 VOP_CACHEDLOOKUP_APV(c0c0c1a0,ed2dd94c,ed2ddba4,ed2ddb90,c75dae00,...) at VOP_CACHEDLOOKUP_APV+0xc5 vfs_cache_lookup(ed2dd9d0,c0b471e4,cb090160,2,cb090160,...) at vfs_cache_lookup+0xd3 VOP_LOOKUP_APV(c0c0c1a0,ed2dd9d0,c74c6690,c0b46172,19b,...) at VOP_LOOKUP_APV+0xe5 lookup(ed2ddb7c,c0b46172,c6,bf,c703a72c,...) at lookup+0x59e namei(ed2ddb7c,c74c6690,ed2dda74,246,c0c61ab8,...) at namei+0x35e vn_open_cred(ed2ddb7c,ed2ddc78,1a4,c75dae00,c74871c8,...) at vn_open_cred+0x62 vn_open(ed2ddb7c,ed2ddc78,1a4,c74871c8,0,...) at vn_open+0x33 kern_open(c74c6690,8164100,0,a02,1a4,...) at kern_open+0xe7 open(c74c6690,ed2ddcfc,c,c0b40d50,c0beb278,...) at open+0x30 syscall(ed2ddd38) at syscall+0x2d3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip = 0x281a161b, esp = 0xbfbfe8cc, ebp = 0xbfbfe8f8 --- db> ps pid ppid pgrp uid state wmesg wchan cmd 768 728 768 0 R+ CPU 1 bsdtar 766 728 766 0 R+ CPU 4 bsdtar 728 724 728 0 Ss+ ttyin 0xc68ce810 csh 724 645 724 0 Ss select 0xc0ca821c sshd 720 711 720 0 S+ ttyin 0xc69a7810 csh 719 1 719 0 Ss+ ttyin 0xc69db410 getty 718 1 718 0 Ss+ ttyin 0xc69dc810 getty 717 1 717 0 Ss+ ttyin 0xc69de410 getty 716 1 716 0 Ss+ ttyin 0xc69d3810 getty 715 1 715 0 Ss+ ttyin 0xc69d5810 getty 714 1 714 0 Ss+ ttyin 0xc69d5c10 getty 713 1 713 0 Ss+ ttyin 0xc69dc010 getty 712 1 712 0 Ss+ ttyin 0xc69db010 getty 711 1 711 0 Ss+ wait 0xc6feb2b8 login 672 1 672 0 Ss nanslp 0xc0c56844 cron 666 1 666 25 Ss pause 0xc7049b40 sendmail 660 1 660 0 Ss select 0xc0ca821c sendmail 645 1 645 0 Ss select 0xc0ca821c sshd 527 1 527 0 Ss select 0xc0ca821c syslogd 473 1 473 0 Ss select 0xc0ca821c devd 143 1 143 0 Ss pause 0xc704ab40 adjkerntz 56 0 0 0 SL sdflush 0xc0cb4668 [softdepflush] 55 0 0 0 SL syncer 0xc0c5666c [syncer] 54 0 0 0 SL vlruwt 0xc6fec000 [vnlru] 53 0 0 0 RL CPU 5 [bufdaemon] 52 0 0 0 SL pgzero 0xc0cb5220 [pagezero] 51 0 0 0 SL psleep 0xc0cb4e38 [vmdaemon] 50 0 0 0 SL psleep 0xc0cb4e00 [pagedaemon] 49 0 0 0 SL waiting_ 0xc0caa434 [sctp_iterator] 48 0 0 0 WL [irq1: atkbd0] 47 0 0 0 WL [swi0: sio] 46 0 0 0 WL [irq15: ata1] 45 0 0 0 WL [irq14: ata0] 44 0 0 0 SL usbevt 0xc68a8210 [usb4] 43 0 0 0 SL usbevt 0xc6994210 [usb3] 42 0 0 0 SL usbevt 0xc6977210 [usb2] 41 0 0 0 SL usbevt 0xc697c210 [usb1] 40 0 0 0 WL [irq20: uhci1 uhci3] 39 0 0 0 SL usbtsk 0xc0c53e14 [usbtask-dr] 38 0 0 0 SL usbtsk 0xc0c53e00 [usbtask-hc] 37 0 0 0 SL usbevt 0xc692d210 [usb0] 36 0 0 0 WL [irq21: uhci0 uhci+] 35 0 0 0 WL [irq259: bce1] 34 0 0 0 SL ispf 0xc67e14d4 [isp1: fc_thrd] 33 0 0 0 WL [irq258: isp1] 32 0 0 0 SL ispf 0xc67e06d4 [isp0: fc_thrd] 31 0 0 0 RL [irq257: isp0] 30 0 0 0 WL [irq16: mfi0] 29 0 0 0 WL [irq256: bce0] 28 0 0 0 WL [irq9: acpi0] 27 0 0 0 SL - 0xc685e900 [thread taskq] 26 0 0 0 WL [swi5: +] 9 0 0 0 SL - 0xc685eb00 [kqueue taskq] 8 0 0 0 SL - 0xc685eb80 [acpi_task_2] 7 0 0 0 SL - 0xc685eb80 [acpi_task_1] 6 0 0 0 SL - 0xc685eb80 [acpi_task_0] 25 0 0 0 WL [swi2: cambio] 5 0 0 0 SL ccb_scan 0xc0c24394 [xpt_thrd] 24 0 0 0 WL [swi6: task queue] 23 0 0 0 WL [swi6: Giant taskq] 22 0 0 0 SL - 0xc0c56674 [yarrow] 4 0 0 0 SL - 0xc0c5420c [g_down] 3 0 0 0 RL CPU 3 [g_up] 2 0 0 0 SL - 0xc0c54200 [g_event] 21 0 0 0 WL [swi3: vm] 20 0 0 0 WL [swi4: clock sio] 19 0 0 0 WL [swi1: net] 18 0 0 0 RL CPU 0 [idle: cpu0] 17 0 0 0 RL [idle: cpu1] 16 0 0 0 RL CPU 2 [idle: cpu2] 15 0 0 0 RL [idle: cpu3] 14 0 0 0 RL [idle: cpu4] 13 0 0 0 RL [idle: cpu5] 12 0 0 0 RL CPU 6 [idle: cpu6] 11 0 0 0 RL CPU 7 [idle: cpu7] 1 0 1 0 SLs wait 0xc6793ae0 [init] 10 0 0 0 SL audit_wo 0xc0cb40a4 [audit] 0 0 0 0 SLs sched 0xc0c542c0 [swapper] db> show pcpu cpuid = 1 curthread = 0xc74c6690: pid 768 "bsdtar" curpcb = 0xed2ddd90 fpcurthread = none idlethread = 0xc67a7460: pid 17 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db> show allpcpu Current CPU: 1 cpuid = 0 curthread = 0xc67a7230: pid 18 "idle: cpu0" curpcb = 0xe5424d90 fpcurthread = none idlethread = 0xc67a7230: pid 18 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 curthread = 0xc74c6690: pid 768 "bsdtar" curpcb = 0xed2ddd90 fpcurthread = none idlethread = 0xc67a7460: pid 17 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 curthread = 0xc6795000: pid 16 "idle: cpu2" curpcb = 0xe541ed90 fpcurthread = none idlethread = 0xc6795000: pid 16 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 curthread = 0xc6797690: pid 3 "g_up" curpcb = 0xe6e36d90 fpcurthread = none idlethread = 0xc6795230: pid 15 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: cpuid = 4 curthread = 0xc75c3690: pid 766 "bsdtar" curpcb = 0xed348d90 fpcurthread = none idlethread = 0xc6795460: pid 14 "idle: cpu4" APIC ID = 4 currentldt = 0x50 spin locks held: cpuid = 5 curthread = 0xc6fea8c0: pid 53 "bufdaemon" curpcb = 0xeafead90 fpcurthread = none idlethread = 0xc6795690: pid 13 "idle: cpu5" APIC ID = 5 currentldt = 0x50 spin locks held: cpuid = 6 curthread = 0xc67958c0: pid 12 "idle: cpu6" curpcb = 0xe5412d90 fpcurthread = none idlethread = 0xc67958c0: pid 12 "idle: cpu6" APIC ID = 6 currentldt = 0x50 spin locks held: cpuid = 7 curthread = 0xc6795af0: pid 11 "idle: cpu7" curpcb = 0xe540fd90 fpcurthread = none idlethread = 0xc6795af0: pid 11 "idle: cpu7" APIC ID = 7 currentldt = 0x50 spin locks held: db> show locks db> show alllocks Process 4 (g_down) thread 0xc6797460 (100015) exclusive sleep mutex isp r = 0 (0xc67e0690) locked @ /usr/src/sys/cam/cam_periph.h:182 db> show lockedvnods Locked vnodes 0xcb090160: tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xcaa70c1c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xc74c6690 (pid 768)#0 0xc077f6c0 at _lockmgr+0x560 #1 0xc099cb51 at ffs_lock+0x91 #2 0xc0aafcc5 at VOP_LOCK1_APV+0xa5 #3 0xc081f7b8 at _vn_lock+0xf8 #4 0xc0813c20 at vget+0x110 #5 0xc08029b7 at cache_lookup+0x4e7 #6 0xc0802aea at vfs_cache_lookup+0xaa #7 0xc0ab0e65 at VOP_LOOKUP_APV+0xe5 #8 0xc0808d2e at lookup+0x59e #9 0xc08098de at namei+0x35e #10 0xc081f022 at vn_open_cred+0x62 #11 0xc081f513 at vn_open+0x33 #12 0xc081e177 at kern_open+0xe7 #13 0xc081e660 at open+0x30 #14 0xc0a98853 at syscall+0x2d3 #15 0xc0a7ea80 at Xint0x80_syscall+0x20 ino 3370734, on dev da0s1d 0xcb5906e0: tag ufs, type VDIR usecount 0, writecount 0, refcount 2 mountedhere 0 flags () v_object 0xc9b65a2c ref 0 pages 0 lock type ufs: EXCL (count 1) by thread 0xc6fea8c0 (pid 53)#0 0xc077f6c0 at _lockmgr+0x560 #1 0xc099cb51 at ffs_lock+0x91 #2 0xc0aafcc5 at VOP_LOCK1_APV+0xa5 #3 0xc081f7b8 at _vn_lock+0xf8 #4 0xc07ff5d8 at flushbufqueues+0x568 #5 0xc07ff8f0 at buf_daemon+0xc0 #6 0xc0770ea8 at fork_exit+0xb8 #7 0xc0a7ea90 at fork_trampoline+0x8 ino 24168915, on dev da0s1d db> trace 768 Tracing pid 768 tid 100104 td 0xc74c6690 kdb_enter_why(c0b3b0a1,c0b3b0a1,c0b5d0ec,ed2dd85c,1,...) at kdb_enter_why+0x3a panic(c0b5d0ec,c70365c0,336eee,0,c0b5d164,...) at panic+0x136 ufs_dirbad(c9c40180,0,c0b5d164,0,ed2dd904,...) at ufs_dirbad+0x49 ufs_lookup(ed2dd94c,c0b6f08d,cb090160,cb090160,ed2ddba4,...) at ufs_lookup+0x377 VOP_CACHEDLOOKUP_APV(c0c0c1a0,ed2dd94c,ed2ddba4,ed2ddb90,c75dae00,...) at VOP_CACHEDLOOKUP_APV+0xc5 vfs_cache_lookup(ed2dd9d0,c0b471e4,cb090160,2,cb090160,...) at vfs_cache_lookup+0xd3 VOP_LOOKUP_APV(c0c0c1a0,ed2dd9d0,c74c6690,c0b46172,19b,...) at VOP_LOOKUP_APV+0xe5 lookup(ed2ddb7c,c0b46172,c6,bf,c703a72c,...) at lookup+0x59e namei(ed2ddb7c,c74c6690,ed2dda74,246,c0c61ab8,...) at namei+0x35e vn_open_cred(ed2ddb7c,ed2ddc78,1a4,c75dae00,c74871c8,...) at vn_open_cred+0x62 vn_open(ed2ddb7c,ed2ddc78,1a4,c74871c8,0,...) at vn_open+0x33 kern_open(c74c6690,8164100,0,a02,1a4,...) at kern_open+0xe7 open(c74c6690,ed2ddcfc,c,c0b40d50,c0beb278,...) at open+0x30 syscall(ed2ddd38) at syscall+0x2d3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip = 0x281a161b, esp = 0xbfbfe8cc, ebp = 0xbfbfe8f8 --- db> trace 55 Tracing pid 55 tid 100054 td 0xc6fea460 sched_switch(c6fea460,0,1,17c,29bc06da,...) at sched_switch+0x49b mi_switch(1,0,c0b3ec35,1cb,c6fea460,...) at mi_switch+0x223 sleepq_switch(c6fea460,0,c0b3ec35,243,68,...) at sleepq_switch+0x182 sleepq_wait(c0c5666c,0,c0b3ba36,d8,0,...) at sleepq_wait+0x60 _sleep(c0c5666c,c0ca8988,68,c0b47461,0,...) at _sleep+0x369 sched_sync(0,eaff0d38,c0b37584,31c,c6febae0,...) at sched_sync+0x752 fork_exit(c0814220,0,eaff0d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xeaff0d70, ebp = 0 --- db> trace 4 Tracing pid 4 tid 100015 td 0xc6797460 sched_switch(c6797460,0,1,17c,5512f4a0,...) at sched_switch+0x49b mi_switch(1,0,c0b3ec35,1cb,4c,...) at mi_switch+0x223 sleepq_switch(c6797460,0,c0b3ec35,266,4c,...) at sleepq_switch+0x182 sleepq_timedwait(c0c5420c,64,c0b32247,0,0,...) at sleepq_timedwait+0x68 _sleep(c0c5420c,c0c54128,24c,c0b32247,64,...) at _sleep+0x345 g_io_schedule_down(c6797460,0,c0b339ab,74,0,...) at g_io_schedule_down+0x6b g_down_procbody(0,e6e39d38,c0b37584,31c,c67eb828,...) at g_down_procbody+0x8d fork_exit(c073d540,0,e6e39d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe6e39d70, ebp = 0 --- db> continue ::Uptime: 7m21s ssttPhysical memory: 3310 MB Dumping 225 MB:aacckk bbaacckkttrraaccee:: ddbb__ttrraaccee__sseellff__wwrrappppeerr((cc00bb33ee003388,,eeadf3e4a8ab400c,,cc00881100333333,,00,,eeadf3e4a8ab400c,,......)) aatt ddbb__ttrraaccee__sseellff__wwrraappppeerr++00xx2266 kkddbb__bbaacckkttrraaccee((00,,ee 4a8ab400c,,cc00aaaacc99bb55,,eeadf3e4a8ab510c,, c b4623806dec00,,......)) aatt kkddbb__bbaacckkttrraaccee++00xx2299 vv f bbaaddlloocckk((c00cc00cc11aa00,,eeadf3e4a8ab510c,,cc00cc223388aa00,,ccbb4623806dc00,,00)) aatt vvffss__bbaaddlloocckk++00xx2233 aasssseerrtt__vvoopp__lloocckkeedd((ccbb4623806dec00,,cc ) aatt aasssseerrtt__vvoopp__lloocckkeedd++00xx5500 vvoopp__lloocckk__ppoosstt((eeda3f4e8aba7a8c,,00,,11020,21,21,0c0b242c8b66e300,d.c.0.,). .a.t) vaotp _vloopc_kl_opcoks_tp+o0sxt3+60 x P O _VLOOPC_KL1O_CAKP1V_(AcP0Vc(0cc01ca00c,1eaa0f,eeada3a48,bc708b,4c709bb427,93b222,,3c2b24,2c8b76b340,e.9.4.,). .a.t) VaOtP _VLOOPC_KL1O_CAKP1V_x0PxVc+10 nc_1v __lvonc_kl(occbk4(2c8b66e300,d1c20,,c160f0e2a, c 05,cc306b9404,7cc04b,487a93b,2.,.2.4)2 ,a.t. ._)v na_tl o_ + lxofc8k h0fxlfu8s 0bvunf_qwureiutees((cc760f5e2c62d48,,e0d,3c40bc46407,cc47,58d0abe,0604,,0.,.c.7)5 ca3t6 9f0l,u.s.h.b)u faqtu uvens_+w0rxi5t6e8+ dxb1u6fa_ adeomfoinl(e0w,reiatfee(aedd3384,8cc06b03,7f5f8f4f,f3f1fcf,,cf6fffefcf2bf8f,,.0.,.c)7 0a5t2 6bdu4f,_.d.a.e)m oant+ 0dxocf0 i efworrikt_ee+x0ixfr843_0w,r0i,teeavf(eca7d53c83)6 9a0t, 4f,oerdk3_4e8xci6t0+,08x1b281 o0f0o,r1k9_at,r.a.m.p)o laitn ek(e)r na_tw rfiotrekv_+t0rxa5m8p 5lwirniet+e0(xc87 4c-3-6-9 0t,readp3 408,c feci,pc ,=c 00b,4 0edscp5 ,=c 00bxeeba2f6e0a,d.7.0.,) eabtp w=r i0t e-+-0-x OfV tPs_yLsOcCaKl:l (0exdc3b4482d8368e)0 aits snyostc allolc+k0exd2 db3u y Xsihnotu0lxd8 0b_es sKc + aBl:l (e)n taetr :X ilnotc0kx 8v0i_oslyastciaolnl [thread pid 53 tid 100052 ] Stopped at kdb_enter_why+0x3a: movl $0,kdb_why db> xD2B0: s-t-a-c ks ybsaccakltlr a(c4:, Fdrbe_etBrSaDc eE_LsFe3l2f,_ wwrraiptpee)r,( c0ibp3 e=0 308x,2e8a1fbe1a793e3c,, ce0s8p1 0=3 303x,b0f,befaef9e2ac9,e ce,b.p. .=) 0axtb fdbbf_etlr7a8ce-_-s-e 6fV_OwPr_aLpOpCeKr:+ 00xx2c6b ,3k0ddbc_0baicsk tnroatc el(o0c,keeadf ebau9te cs,hco0ualadc 9bbe5 deKaDfBe:a 9efnct,ecrb:4 2l8o6cek ,v.i.o.l)a taito nk [thread pid 766 tid 100137 ] Stopped at kdb_enter_why+0x3a: movl $0,kdb_why db> r_DbBa:c ksttraaccke +b0axc2k9t bavcfes:_ addbl_otcrka(cce0_cs0ecl1f c _w,reaapfpeear9(fcc0,bc30ec023388,ae0d,3c4b84a2d8c6,ec00,801)0 3a3t3 ,vcf7s5_cb3a6d9l0o,cekd+304x82a3d ,,a.s.s.e)r ta_tv odpb__ltorcakceed_(scebl4f2_8w6rea0p,pce0rb+60fx22465 cck0dbb3_9bca4c9k,tdraa9c3e3(0c1705,c43,6.9.0.,)e da3t4 8aasdsce,rct0_avaocp9_bl5o,cekde3d4+80axe5c0, bV6O3P0_dScT0R,A.T.E.G)Y _aAtP Vk(dcb0_c 0 , c1kat0r,aecaef+e0axa239c cvbf4 ) 2_b7abd4l,occ0kc(2c30cca00c,1cab04,2e8d63e408,a.e.c.,)c 0act2 3V8OaP0_,ScTbR6A3T0EdGcY0_,AcP7V5+c03x699b0 bautf svtfrsa_tbeag , yl(occbk4+20x72e3c daas9s3e3r0t1_0v , c 0_0e,lco0cckae8d6(3c0b,6d3a09d3c300,7c00,b.6.f.4)8 0a,te db3u4f8sbt2rca,ce0g8y1+20dx6601, e0bcu0fcw1rai0t,e.(.d.a)9 3a3t0 1a0s,seearfte_avao9p0_,ecl0oacakfecde+10,xe5a3f aV OaPc_,W0R,I.T.E._)A PaVt( c 0ucf0wcr1iat0e,+e0dx314981b 4v,fcs7_5bci3o6_9a0w,rci0tbe4(7d9ab923,320420,,.0.,.c)0 ba4t4 7VcO4P,_8WaR4I,T8Ea3A,P.V.+.0)x 9a8t 3vvfns__wbriiot_ea(wcr7i0t5e2+60dx43,2e8d 48lcu6s0h,bcu7f5qduaeeu0e0s,(0c,6cf7e5cc23b689,00,,. . 4a7tc 4v,n8_0wbr, t 6t4e,+.0.x.2)3 1a f dfolfuislhebwurfiqtuee(ueeds3+408xc5690c, fbfufff_fd f b fonf(f0f,fefaff,e0a,dc3780,5c206bd347,5.8.4.,)3 1act, cd6offeicl2ebw8r,i.t.e.+)0 xa9t5 5ukfe_rdna_ewmroint+e0vx(c c07 02cf3o6r9k0_,e4x,ietd(3c4087cf6f08,3801,201,4e0a0f,e1a9da3,8.). .a)t afto rkke_renx_iwtr+i0txebv8+ kxf5o8r 0_wtrriatmep(ocl7i5nce3(6)9 0a,te df3o4r8kc_ftcr,acm,pco0lbi4n0ed+c05x,8c =b-e-b-2 6t0r,a.p. .0), aeti pw r=i t0e,+ 0exs4pf c s0yxsecaaflela(d1e7d03,04 8edb3p8 )= a0t -s-y-s EaVlOlP+_0SxT2RdA3T 0GXYi:n t00xxc8b04_2s8y6sec0a lils( )n oatt lXoicnkte0dx 8b0u_ts ysshcoaullld+ 0bxe2 K LD-B-:- esnytsecra:l ll o(c4k, vFiroeleaBtSiDo nE [thread pid 53 tid 100052 ] Stopped at kdb_enter_why+0x3a: movl $0,kdb_why db> panic [no answer... let's power cycle the server once more] -- Panagiotis J. Christias Network Management Center P.Christias@noc.ntua.gr National Technical Univ. of Athens, GREECE From owner-freebsd-scsi@FreeBSD.ORG Wed Oct 15 06:02:04 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4002C106568C; Wed, 15 Oct 2008 06:02:04 +0000 (UTC) (envelope-from os@rsu.ru) Received: from mail.r61.net (mail.r61.net [195.208.245.249]) by mx1.freebsd.org (Postfix) with ESMTP id 1810A8FC19; Wed, 15 Oct 2008 06:02:02 +0000 (UTC) (envelope-from os@rsu.ru) Received: from [195.208.252.154] (brain.cc.rsu.ru [195.208.252.154]) (authenticated bits=0) by mail.r61.net (8.14.1/8.14.1) with ESMTP id m9F5iF7p028676 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 09:44:16 +0400 (MSD) (envelope-from os@rsu.ru) From: Oleg Sharoiko To: Panagiotis Christias In-Reply-To: <20081014222343.GA8706@noc.ntua.gr> References: <20081014222343.GA8706@noc.ntua.gr> Content-Type: text/plain Date: Wed, 15 Oct 2008 09:44:15 +0400 Message-Id: <1224049455.1277.44.camel@brain.cc.rsu.ru> Mime-Version: 1.0 X-Mailer: Evolution 2.22.3.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD 7-STABLE, isp(4), QLE2462: panic & deadlocks X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Oct 2008 06:02:04 -0000 Hi! On Wed, 2008-10-15 at 01:23 +0300, Panagiotis Christias wrote: > However, when we connect them to the CX3-40, create and mount a new > partition and then do something as simple as "tar -C /san -xf ports.tgz" > the system panics and deadlocks. We have tried several FreeBSD versions > (6.3 i386/adm64, 7.0 i386/adm64, 7.1 i386/adm64 and lastly 7-STABLE i386 > - we also tried the latest 8-CURRENT snapshot but it panicked too soon). > The result is always the same; panic and deadlock. Try reducing the number of "tagged openings" with 'camcontrol tags' down to 46. If it doesn't work try reducing it further to 2. Also be advised that I've seen panics with geom_multipath in FreeBSD-7, unfortunately I had no time to test it in -current. -- Oleg Sharoiko. Software and Network Engineer Computer Center of Rostov State University. From owner-freebsd-scsi@FreeBSD.ORG Wed Oct 15 17:54:57 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 534A1106568F; Wed, 15 Oct 2008 17:54:57 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from diomedes.noc.ntua.gr (diomedes.noc.ntua.gr [IPv6:2001:648:2000:de::220]) by mx1.freebsd.org (Postfix) with ESMTP id 5A1018FC15; Wed, 15 Oct 2008 17:54:56 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (ajax6.noc.ntua.gr [IPv6:2001:648:2000:dc::1]) by diomedes.noc.ntua.gr (8.14.0/8.14.0) with ESMTP id m9FHsrkv010592 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 20:54:54 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (localhost.noc.ntua.gr [127.0.0.1]) by ajax.noc.ntua.gr (8.13.8/8.13.8) with ESMTP id m9FHsrGP005474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 20:54:53 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: (from christia@localhost) by ajax.noc.ntua.gr (8.13.8/8.13.8/Submit) id m9FHsrHh005469; Wed, 15 Oct 2008 20:54:53 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) X-Authentication-Warning: ajax.noc.ntua.gr: christia set sender to p.christias@noc.ntua.gr using -f Date: Wed, 15 Oct 2008 20:54:53 +0300 From: Panagiotis Christias To: Oleg Sharoiko Message-ID: <20081015175453.GA3260@noc.ntua.gr> References: <20081014222343.GA8706@noc.ntua.gr> <1224049455.1277.44.camel@brain.cc.rsu.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1224049455.1277.44.camel@brain.cc.rsu.ru> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (diomedes.noc.ntua.gr [IPv6:2001:648:2000:de::220]); Wed, 15 Oct 2008 20:54:54 +0300 (EEST) X-Virus-Scanned: ClamAV version 0.93.3, clamav-milter version 0.93.3 on diomedes.noc.ntua.gr X-Virus-Status: Clean Cc: freebsd-scsi@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD 7-STABLE, isp(4), QLE2462: panic & deadlocks X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Oct 2008 17:54:57 -0000 On Wed, Oct 15, 2008 at 09:44:15AM +0400, Oleg Sharoiko wrote: > Hi! > > On Wed, 2008-10-15 at 01:23 +0300, Panagiotis Christias wrote: > > > However, when we connect them to the CX3-40, create and mount a new > > partition and then do something as simple as "tar -C /san -xf ports.tgz" > > the system panics and deadlocks. We have tried several FreeBSD versions > > (6.3 i386/adm64, 7.0 i386/adm64, 7.1 i386/adm64 and lastly 7-STABLE i386 > > - we also tried the latest 8-CURRENT snapshot but it panicked too soon). > > The result is always the same; panic and deadlock. > > Try reducing the number of "tagged openings" with 'camcontrol tags' down > to 46. If it doesn't work try reducing it further to 2. Also be advised > that I've seen panics with geom_multipath in FreeBSD-7, unfortunately I > had no time to test it in -current. Hm.. that would probably explain the fact that I was unable to panic the system when I had set the hint.isp.0.debug="0x1F" in /boot/device.hints. Currently I am stress testing the server with the tagged openings set to 44 (first value tested). Until now there is no panic or deadlock. I am trying concurrent tar extractions and rsync copies. The filesystem looks ok till now according to fsck. I will let it write/copy/delete overnight and tomorrow I will try different tagged opening values. Thank you for the hint! I am wondering what is the performance penalty with decreased tagged openings. Also, is there anything else I could try in order to get more useful debug output? I have at least three servers that I could use for any kind of tests and I am willing to spend as much time I can get to help solving the problem. Finally, the only output in the logs is: Expensive timeout(9) function: 0xc06f4210(0xc67e1200) 0.059422635 s Expensive timeout(9) function: 0xc08d4fd0(0) 0.060676147 s I suppose that is related to the CAMDEBUG kernel config options. Regards, Panagiotis -- Panagiotis J. Christias Network Management Center P.Christias@noc.ntua.gr National Technical Univ. of Athens, GREECE From owner-freebsd-scsi@FreeBSD.ORG Fri Oct 17 17:56:53 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94A6C106568D for ; Fri, 17 Oct 2008 17:56:53 +0000 (UTC) (envelope-from daniel@dgnetwork.com.br) Received: from mail.mastercabo.com.br (mail.mastercabo.com.br [200.179.179.14]) by mx1.freebsd.org (Postfix) with SMTP id BB6368FC14 for ; Fri, 17 Oct 2008 17:56:52 +0000 (UTC) (envelope-from daniel@dgnetwork.com.br) Received: (qmail 38515 invoked by uid 1008); 17 Oct 2008 17:56:28 -0000 Received: from unknown (HELO ?192.168.0.169?) (200.251.26.17) by mail.mastercabo.com.br with SMTP; 17 Oct 2008 17:56:28 -0000 Message-ID: <48F8D1C3.1080607@dgnetwork.com.br> Date: Fri, 17 Oct 2008 14:56:19 -0300 From: =?ISO-8859-1?Q?Daniel_Dias_Gon=E7alves?= User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Danny Braniss References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-scsi@freebsd.org Subject: Re: FreeBSD and ISCSI, Strange Problem X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: daniel@dgnetwork.com.br List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Oct 2008 17:56:53 -0000 Thanks Danny, The patch work fine, but i have new problem: Have two servers 7.0R with iscsi-2.1. They are mounted the same directory way iSCSI. When I create an archive inside of this directory in the server A, the server B don't show the archive, if in the server B to unmount and mount the directory again, the archive created in the server A appears It. Where it is the problem? The storage is Dell MD3000i. Thanks, Daniel Danny Braniss escreveu: >> the problem is probably that iscsi is deadlocked, so fetch >> ftp://ftp/users/danny/freebsd/iscsi-2.1.tar.gzs;/ftp/;/&.cs.huji.ac.il; >> > ftp://ftp.cs.huji.ac.il/users/danny/freebsd/iscsi-2.1.tar.gz > > >> Danny, >> >> You typed the ftp wrong. >> >> hi Daniel, >> > > oh well, it was before coffee :-) > > >> Obrigado, thanks !! =) >> >> Daniel >> >> >> > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-scsi@FreeBSD.ORG Fri Oct 17 21:10:32 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93893106568D for ; Fri, 17 Oct 2008 21:10:32 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.freebsd.org (Postfix) with ESMTP id 4A4878FC0A for ; Fri, 17 Oct 2008 21:10:32 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.3/8.14.3) with ESMTP id m9HLAUb3043191 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 17 Oct 2008 16:10:30 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.3/8.14.3/Submit) id m9HLATZ3043190; Fri, 17 Oct 2008 16:10:29 -0500 (CDT) (envelope-from dan) Date: Fri, 17 Oct 2008 16:10:29 -0500 From: Dan Nelson To: Daniel Dias =?utf-8?Q?Gon=C3=A7alves?= Message-ID: <20081017211028.GK99270@dan.emsphone.com> References: <48F8D1C3.1080607@dgnetwork.com.br> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <48F8D1C3.1080607@dgnetwork.com.br> X-OS: FreeBSD 7.1-PRERELEASE User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org, freebsd-scsi@freebsd.org Subject: Re: FreeBSD and ISCSI, Strange Problem X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Oct 2008 21:10:32 -0000 In the last episode (Oct 17), Daniel Dias Gonçalves said: > Thanks Danny, > > The patch work fine, but i have new problem: > > Have two servers 7.0R with iscsi-2.1. > They are mounted the same directory way iSCSI. When I create an archive > inside of this directory in the server A, the server B don't show the > archive, if in the server B to unmount and mount the directory again, > the archive created in the server A appears It. > > Where it is the problem? You can't mount the same UFS filesystem on two servers at once. You'll end up with a horribly-corrupted filesystem, since neither one is aware of changes the other makes. You would need a shared-storage cluster filessytem to be able to do that (or mount the volume read-only on both servers). Mount the filesystem on one server only, then access it via NFS from the other. -- Dan Nelson dnelson@allantgroup.com