From owner-freebsd-hardware@FreeBSD.ORG Fri Jun 25 12:24:25 2010 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D458F106564A for ; Fri, 25 Jun 2010 12:24:25 +0000 (UTC) (envelope-from ipluta@wp.pl) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.8]) by mx1.freebsd.org (Postfix) with ESMTP id 1023C8FC12 for ; Fri, 25 Jun 2010 12:24:24 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 26745 invoked from network); 25 Jun 2010 13:57:42 +0200 Received: from afgi78.neoplus.adsl.tpnet.pl (HELO [127.0.0.1]) (ipluta@[95.49.164.78]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES256-SHA encrypted SMTP for ; 25 Jun 2010 13:57:42 +0200 Message-ID: <4C2499B5.3030404@wp.pl> Date: Fri, 25 Jun 2010 13:57:41 +0200 From: Ireneusz Pluta User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: freebsd-hardware@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [0YM0] Subject: System hangs during heavy sequential write to mfi device X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jun 2010 12:24:26 -0000 Hello, I already posted this to freebsd-questions, with no response, so far. As this is rather a problem closer to hardware issues, so maybe here I have a better luck. Hope nobody blames me for crossposting. Below is the exact copy of what I posted there, but here let add than meanwuile I also noticed that the problem does not appear when writing directly to to /dev/mfid0 device, instead of writing to a file on a mounted filesystem. I have the following LSI adapter: # mfiutil show adapter mfi0 Adapter: Product Name: MegaRAID SAS 84016E Serial Number: P709144809 Firmware: 7.0.1-0075 RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID6, RAID10, RAID50 Battery Backup: present NVRAM: 32K Onboard Memory: 256M Minimum Stripe: 8K Maximum Stripe: 1M in my: # uname -a FreeBSD emu.xxx.xx 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 with 48GB RAM, 2xXeon E The RAID controller has a volume of raid10, consisring of 16 2TB disks, 256K stripe, (more disk details, as well as full dmesg.boot are included at the end of this message): # mfiutil show volumes && mfiutil cache 0 mfi0 Volumes: Id Size Level Stripe State Cache Name mfid0 ( 15T) RAID-10 256K OPTIMAL Writes mfi0 volume mfid0 cache settings: I/O caching: writes write caching: write-back read ahead: none drive write cache: default The machine has not been put in production yet. It will be a dedicated PostgreSQL server. So far I am figuring out the best choice of raid and filesystem configuration. When I test sequential write performance by: # dd if=/dev/zero of=/path/to/mountpoint/bigfile bs=8k count=$bigcount, or an appropriate fio job, then system perioduically hangs, stops responding for a while, and then flushes series of garbled output to the system log like this: Jun 22 15:09:21 emu kernel: N NISAM NIM I II3SS0A,A N NE3MIMS0II3, N 0I ,ENSIMA NSI IEAMASI MI A IS3S A00 0, Jun 22 15:09:21 emu kernel: I Jun 22 15:09:21 emu kernel: A33 Jun 22 15:09:21 emu kernel: Jun 22 15:09:21 emu kernel: I Jun 22 15:09:21 emu kernel: Jun 22 15:09:21 emu kernel: < Jun 22 15:09:21 emu kernel: <<2>22< >00<, Jun 22 15:09:21 emu kernel: Jun 22 15:09:21 emu kernel: <>2>EI2S2A 0 Jun 22 15:09:21 emu kernel: 0> Jun 22 15:09:21 emu kernel: ,< SE2AI>S>A 0E 3IS0IA, S A0E Jun 22 15:09:21 emu kernel: I Jun 22 15:09:21 emu kernel: Jun 22 15:09:21 emu kernel: Jun 22 15:09:21 emu kernel: <<2>2S>A 300 Jun 22 15:09:21 emu kernel: , Jun 22 15:09:21 emu kernel: EISA 0 Jun 22 15:09:24 emu kernel: NMI ISA N3M0I, IESNISAAM 0I Jun 22 15:09:24 emu kernel: 3N Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: < Jun 22 15:09:24 emu kernel: <22<>0,EN I2IMI>SASA NM0> 2INI Jun 22 15:09:24 emu kernel: >MS I Jun 22 15:09:24 emu kernel: MMAE A II2S 0I, AI IE0SIS Jun 22 15:09:24 emu kernel: SS Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: < Jun 22 15:09:24 emu kernel: 2<<><>>A2 A200 , 22E>,0I ,SE EIASI SAA 00A2 Jun 22 15:09:24 emu kernel: 0 Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: Jun 22 15:09:24 emu kernel: <<22>>,0 Jun 22 15:09:24 emu kernel: E Jun 22 15:09:24 emu kernel: ISA 0 Sometimes, also the following messages can be seen around: Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 956 usec to 437 usec for pid 14834 (su) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 2013 usec to 921 usec for pid 14833 (bash) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 60 usec to 27 usec for pid 14831 (screen) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 1159 usec to 753 usec for pid 14830 (screen) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 5149 usec to 2792 usec for pid 12477 (bash) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 58987 usec to 37622 usec for pid 12477 (bash) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 26566 usec to 13431 usec for pid 12095 (bash) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 487938869 usec to 223259876 usec for pid 12095 (bash) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 1038 usec to 475 usec for pid 12094 (su) Jun 22 15:11:23 emu kernel: calcru: runtime went backwards from 2026 usec to 927 usec for pid 12076 (bash) (they do not seem to be caused just by the situation explained in the FAQ: http://www.freebsd.org/doc/en/books/faq/troubleshoot.html#CALCRU-NEGATIVE-RUNTIME, as I already followed that tip and disabled the speed step option in BIOS) It also seems that with stripe sizes lower than 128k, like 64k, the problem does not appear. However I need to confirm that with some more extensive tests. Could anyone point me to a resolution of the problem? Thanks Irek. PS. more detailed raid config, and dmesg.boot follows: # mfiutil show config mfi0 Configuration: 8 arrays, 1 volumes, 0 spares array 0 of 2 drives: drive 16 ( 1863G) ONLINE SATA enclosure 1, slot 0 drive 24 ( 1863G) ONLINE SATA enclosure 2, slot 0 array 1 of 2 drives: drive 17 ( 1863G) ONLINE SATA enclosure 1, slot 1 drive 25 ( 1863G) ONLINE SATA enclosure 2, slot 1 array 2 of 2 drives: drive 18 ( 1863G) ONLINE SATA enclosure 1, slot 2 drive 26 ( 1863G) ONLINE SATA enclosure 2, slot 2 array 3 of 2 drives: drive 19 ( 1863G) ONLINE SATA enclosure 1, slot 3 drive 27 ( 1863G) ONLINE SATA enclosure 2, slot 3 array 4 of 2 drives: drive 20 ( 1863G) ONLINE SATA enclosure 1, slot 4 drive 28 ( 1863G) ONLINE SATA enclosure 2, slot 4 array 5 of 2 drives: drive 21 ( 1863G) ONLINE SATA enclosure 1, slot 5 drive 29 ( 1863G) ONLINE SATA enclosure 2, slot 5 array 6 of 2 drives: drive 22 ( 1863G) ONLINE SATA enclosure 1, slot 6 drive 30 ( 1863G) ONLINE SATA enclosure 2, slot 6 array 7 of 2 drives: drive 23 ( 1863G) ONLINE SATA enclosure 1, slot 7 drive 31 ( 1863G) ONLINE SATA enclosure 2, slot 7 volume mfid0 (15T) RAID-10 256K OPTIMAL spans: array 0 array 1 array 2 array 3 array 4 array 5 array 6 array 7 $ cat /var/run/dmesg.boot Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2266.65-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106a5 Stepping = 5 Features=0xbfebfbff Features2=0x9ce3bd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant real memory = 51539607552 (49152 MB) avail memory = 49656578048 (47356 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 cpu4 (AP): APIC ID: 16 cpu5 (AP): APIC ID: 18 cpu6 (AP): APIC ID: 20 cpu7 (AP): APIC ID: 22 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard lapic0: Forcing LINT1 to edge trigger kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 28 at device 1.0 on pci0 pci1: on pcib1 igb0: port 0x1020-0x103f mem 0xb1b20000-0xb1b3ffff,0xb1b44000-0xb1b47fff irq 40 at device 0.0 on pci1 igb0: Using MSIX interrupts with 3 vectors igb0: [ITHREAD] igb0: [ITHREAD] igb0: [ITHREAD] igb0: Ethernet address: 00:15:17:d7:ca:50 igb1: port 0x1000-0x101f mem 0xb1b00000-0xb1b1ffff,0xb1b40000-0xb1b43fff irq 28 at device 0.1 on pci1 igb1: Using MSIX interrupts with 3 vectors igb1: [ITHREAD] igb1: [ITHREAD] igb1: [ITHREAD] igb1: Ethernet address: 00:15:17:d7:ca:51 pcib2: irq 24 at device 3.0 on pci0 pci2: on pcib2 pcib3: at device 0.0 on pci2 pci3: on pcib3 mfi0: mem 0xb1900000-0xb190ffff,0xb1a00000-0xb1a3ffff irq 35 at device 14.0 on pci3 mfi0: Megaraid SAS driver Ver 3.00 mfi0: 6573 (330436349s/0x0020/info) - Shutdown command received from host mfi0: 6574 (boot + 0s/0x0020/info) - Firmware initialization started (PCI ID 0411/1000/1008/1000) mfi0: 6575 (boot + 0s/0x0020/info) - Firmware version 1.12.280-0826 mfi0: 6576 (boot + 45s/0x0008/info) - Battery temperature is normal mfi0: 6577 (boot + 45s/0x0008/info) - Battery Present mfi0: 6578 (boot + 45s/0x0020/info) - Board Revision ( mfi0: 6579 (boot + 93s/0x0002/info) - Inserted: PD 10(e0xff/s16) mfi0: 6580 (boot + 93s/0x0002/info) - Inserted: PD 10(e0xff/s16) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=91813c23a6a08faa,0000000000000000 mfi0: 6581 (boot + 93s/0x0002/info) - Inserted: PD 11(e0xff/s17) mfi0: 6582 (boot + 93s/0x0002/info) - Inserted: PD 11(e0xff/s17) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=91813c24a1a18ca8,0000000000000000 mfi0: 6583 (boot + 93s/0x0002/info) - Inserted: PD 12(e0xff/s18) mfi0: 6584 (boot + 93s/0x0002/info) - Inserted: PD 12(e0xff/s18) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=91813c24a29f8ca9,0000000000000000 mfi0: 6585 (boot + 93s/0x0002/info) - Inserted: PD 13(e0xff/s19) mfi0: 6586 (boot + 93s/0x0002/info) - Inserted: PD 13(e0xff/s19) Info: enclPd=ffff, scsiType=0, portMap=03, sasAddr=91813b24a5a591a3,0000000000000000 mfi0: 6587 (boot + 93s/0x0002/info) - Inserted: PD 14(e0xff/s20) mfi0: 6588 (boot + 93s/0x0002/info) - Inserted: PD 14(e0xff/s20) Info: enclPd=ffff, scsiType=0, portMap=04, sasAddr=918139229fa08fa4,0000000000000000 mfi0: 6589 (boot + 93s/0x0002/info) - Inserted: PD 15(e0xff/s21) mfi0: 6590 (boot + 93s/0x0002/info) - Inserted: PD 15(e0xff/s21) Info: enclPd=ffff, scsiType=0, portMap=05, sasAddr=91813c249fa68ca4,0000000000000000 mfi0: 6591 (boot + 93s/0x0002/info) - Inserted: PD 16(e0xff/s22) mfi0: 6592 (boot + 93s/0x0002/info) - Inserted: PD 16(e0xff/s22) Info: enclPd=ffff, scsiType=0, portMap=06, sasAddr=91813c1ea6a792a2,0000000000000000 mfi0: 6593 (boot + 93s/0x0002/info) - Inserted: PD 17(e0xff/s23) mfi0: 6594 (boot + 93s/0x0002/info) - Inserted: PD 17(e0xff/s23) Info: enclPd=ffff, scsiType=0, portMap=07, sasAddr=91813c23a6a092a6,0000000000000000 mfi0: 6595 (boot + 93s/0x0002/info) - Inserted: PD 18(e0xff/s24) mfi0: 6596 (boot + 93s/0x0002/info) - Inserted: PD 18(e0xff/s24) Info: enclPd=ffff, scsiType=0, portMap=08, sasAddr=91813b24a3a08da2,0000000000000000 mfi0: 6597 (boot + 93s/0x0002/info) - Inserted: PD 19(e0xff/s25) mfi0: 6598 (boot + 93s/0x0002/info) - Inserted: PD 19(e0xff/s25) Info: enclPd=ffff, scsiType=0, portMap=09, sasAddr=91813c23a6a08cab,0000000000000000 mfi0: 6599 (boot + 93s/0x0002/info) - Inserted: PD 1a(e0xff/s26) mfi0: 6600 (boot + 93s/0x0002/info) - Inserted: PD 1a(e0xff/s26) Info: enclPd=ffff, scsiType=0, portMap=0a, sasAddr=91813b24a5a094a8,0000000000000000 mfi0: 6601 (boot + 93s/0x0002/info) - Inserted: PD 1b(e0xff/s27) mfi0: 6602 (boot + 93s/0x0002/info) - Inserted: PD 1b(e0xff/s27) Info: enclPd=ffff, scsiType=0, portMap=0b, sasAddr=91813c21a0a88ca3,0000000000000000 mfi0: 6603 (boot + 93s/0x0002/info) - Inserted: PD 1c(e0xff/s28) mfi0: 6604 (boot + 93s/0x0002/info) - Inserted: PD 1c(e0xff/s28) Info: enclPd=ffff, scsiType=0, portMap=0c, sasAddr=91813c23a5a590a4,0000000000000000 mfi0: 6605 (boot + 93s/0x0002/info) - Inserted: PD 1d(e0xff/s29) mfi0: 6606 (boot + 93s/0x0002/info) - Inserted: PD 1d(e0xff/s29) Info: enclPd=ffff, scsiType=0, portMap=0d, sasAddr=91813b24a5a191a2,0000000000000000 mfi0: 6607 (boot + 93s/0x0002/info) - Inserted: PD 1e(e0xff/s30) mfi0: 6608 (boot + 93s/0x0002/info) - Inserted: PD 1e(e0xff/s30) Info: enclPd=ffff, scsiType=0, portMap=0e, sasAddr=91813c24a3a695a3,0000000000000000 mfi0: 6609 (boot + 93s/0x0002/info) - Inserted: PD 1f(e0xff/s31) mfi0: 6610 (boot + 93s/0x0002/info) - Inserted: PD 1f(e0xff/s31) Info: enclPd=ffff, scsiType=0, portMap=0f, sasAddr=91813a27a0a493a6,0000000000000000 mfi0: 6611 (330436454s/0x0020/info) - Time established as 06/21/10 11:54:14; (95 seconds since power on) mfi0: 6612 (boot + 0s/0x0020/info) - Firmware initialization started (PCI ID 0411/1000/1008/1000) mfi0: 6613 (boot + 0s/0x0020/info) - Firmware version 1.12.280-0826 mfi0: 6614 (boot + 48s/0x0008/info) - Battery temperature is normal mfi0: 6615 (boot + 48s/0x0008/info) - Battery Present mfi0: 6616 (boot + 48s/0x0020/info) - Board Revision ( mfi0: 6617 (boot + 95s/0x0002/info) - Inserted: PD 10(e0xff/s16) mfi0: 6618 (boot + 95s/0x0002/info) - Inserted: PD 10(e0xff/s16) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=91813c23a6a08faa,0000000000000000 mfi0: 6619 (boot + 95s/0x0002/info) - Inserted: PD 11(e0xff/s17) mfi0: 6620 (boot + 95s/0x0002/info) - Inserted: PD 11(e0xff/s17) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=91813c24a1a18ca8,0000000000000000 mfi0: 6621 (boot + 95s/0x0002/info) - Inserted: PD 12(e0xff/s18) mfi0: 6622 (boot + 95s/0x0002/info) - Inserted: PD 12(e0xff/s18) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=91813c24a29f8ca9,0000000000000000 mfi0: 6623 (boot + 95s/0x0002/info) - Inserted: PD 13(e0xff/s19) mfi0: 6624 (boot + 95s/0x0002/info) - Inserted: PD 13(e0xff/s19) Info: enclPd=ffff, scsiType=0, portMap=03, sasAddr=91813b24a5a591a3,0000000000000000 mfi0: 6625 (boot + 95s/0x0002/info) - Inserted: PD 14(e0xff/s20) mfi0: 6626 (boot + 95s/0x0002/info) - Inserted: PD 14(e0xff/s20) Info: enclPd=ffff, scsiType=0, portMap=04, sasAddr=918139229fa08fa4,0000000000000000 mfi0: 6627 (boot + 95s/0x0002/info) - Inserted: PD 15(e0xff/s21) mfi0: 6628 (boot + 95s/0x0002/info) - Inserted: PD 15(e0xff/s21) Info: enclPd=ffff, scsiType=0, portMap=05, sasAddr=91813c249fa68ca4,0000000000000000 mfi0: 6629 (boot + 95s/0x0002/info) - Inserted: PD 16(e0xff/s22) mfi0: 6630 (boot + 95s/0x0002/info) - Inserted: PD 16(e0xff/s22) Info: enclPd=ffff, scsiType=0, portMap=06, sasAddr=91813c1ea6a792a2,0000000000000000 mfi0: 6631 (boot + 95s/0x0002/info) - Inserted: PD 17(e0xff/s23) mfi0: 6632 (boot + 95s/0x0002/info) - Inserted: PD 17(e0xff/s23) Info: enclPd=ffff, scsiType=0, portMap=07, sasAddr=91813c23a6a092a6,0000000000000000 mfi0: 6633 (boot + 95s/0x0002/info) - Inserted: PD 18(e0xff/s24) mfi0: 6634 (boot + 95s/0x0002/info) - Inserted: PD 18(e0xff/s24) Info: enclPd=ffff, scsiType=0, portMap=08, sasAddr=91813b24a3a08da2,0000000000000000 mfi0: 6635 (boot + 95s/0x0002/info) - Inserted: PD 19(e0xff/s25) mfi0: 6636 (boot + 95s/0x0002/info) - Inserted: PD 19(e0xff/s25) Info: enclPd=ffff, scsiType=0, portMap=09, sasAddr=91813c23a6a08cab,0000000000000000 mfi0: 6637 (boot + 95s/0x0002/info) - Inserted: PD 1a(e0xff/s26) mfi0: 6638 (boot + 95s/0x0002/info) - Inserted: PD 1a(e0xff/s26) Info: enclPd=ffff, scsiType=0, portMap=0a, sasAddr=91813b24a5a094a8,0000000000000000 mfi0: 6639 (boot + 95s/0x0002/info) - Inserted: PD 1b(e0xff/s27) mfi0: 6640 (boot + 95s/0x0002/info) - Inserted: PD 1b(e0xff/s27) Info: enclPd=ffff, scsiType=0, portMap=0b, sasAddr=91813c21a0a88ca3,0000000000000000 mfi0: 6641 (boot + 95s/0x0002/info) - Inserted: PD 1c(e0xff/s28) mfi0: 6642 (boot + 95s/0x0002/info) - Inserted: PD 1c(e0xff/s28) Info: enclPd=ffff, scsiType=0, portMap=0c, sasAddr=91813c23a5a590a4,0000000000000000 mfi0: 6643 (boot + 95s/0x0002/info) - Inserted: PD 1d(e0xff/s29) mfi0: 6644 (boot + 95s/0x0002/info) - Inserted: PD 1d(e0xff/s29) Info: enclPd=ffff, scsiType=0, portMap=0d, sasAddr=91813b24a5a191a2,0000000000000000 mfi0: 6645 (boot + 95s/0x0002/info) - Inserted: PD 1e(e0xff/s30) mfi0: 6646 (boot + 95s/0x0002/info) - Inserted: PD 1e(e0xff/s30) Info: enclPd=ffff, scsiType=0, portMap=0e, sasAddr=91813c24a3a695a3,0000000000000000 mfi0: 6647 (boot + 95s/0x0002/info) - Inserted: PD 1f(e0xff/s31) mfi0: 6648 (boot + 95s/0x0002/info) - Inserted: PD 1f(e0xff/s31) Info: enclPd=ffff, scsiType=0, portMap=0f, sasAddr=91813a27a0a493a6,0000000000000000 mfi0: 6649 (330436590s/0x0020/info) - Time established as 06/21/10 11:56:30; (98 seconds since power on) mfi0: 6650 (boot + 0s/0x0020/info) - Firmware initialization started (PCI ID 0411/1000/1008/1000) mfi0: 6651 (boot + 0s/0x0020/info) - Firmware version 1.12.280-0826 mfi0: 6652 (boot + 46s/0x0008/info) - Battery temperature is normal mfi0: 6653 (boot + 46s/0x0008/info) - Battery Present mfi0: 6654 (boot + 47s/0x0020/info) - Board Revision ( mfi0: 6655 (boot + 94s/0x0002/info) - Inserted: PD 10(e0xff/s16) mfi0: 6656 (boot + 94s/0x0002/info) - Inserted: PD 10(e0xff/s16) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=91813c23a6a08faa,0000000000000000 mfi0: 6657 (boot + 94s/0x0002/info) - Inserted: PD 11(e0xff/s17) mfi0: 6658 (boot + 94s/0x0002/info) - Inserted: PD 11(e0xff/s17) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=91813c24a1a18ca8,0000000000000000 mfi0: 6659 (boot + 94s/0x0002/info) - Inserted: PD 12(e0xff/s18) mfi0: 6660 (boot + 94s/0x0002/info) - Inserted: PD 12(e0xff/s18) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=91813c24a29f8ca9,0000000000000000 mfi0: 6661 (boot + 94s/0x0002/info) - Inserted: PD 13(e0xff/s19) mfi0: 6662 (boot + 94s/0x0002/info) - Inserted: PD 13(e0xff/s19) Info: enclPd=ffff, scsiType=0, portMap=03, sasAddr=91813b24a5a591a3,0000000000000000 mfi0: 6663 (boot + 94s/0x0002/info) - Inserted: PD 14(e0xff/s20) mfi0: 6664 (boot + 94s/0x0002/info) - Inserted: PD 14(e0xff/s20) Info: enclPd=ffff, scsiType=0, portMap=04, sasAddr=918139229fa08fa4,0000000000000000 mfi0: 6665 (boot + 94s/0x0002/info) - Inserted: PD 15(e0xff/s21) mfi0: 6666 (boot + 94s/0x0002/info) - Inserted: PD 15(e0xff/s21) Info: enclPd=ffff, scsiType=0, portMap=05, sasAddr=91813c249fa68ca4,0000000000000000 mfi0: 6667 (boot + 94s/0x0002/info) - Inserted: PD 16(e0xff/s22) mfi0: 6668 (boot + 94s/0x0002/info) - Inserted: PD 16(e0xff/s22) Info: enclPd=ffff, scsiType=0, portMap=06, sasAddr=91813c1ea6a792a2,0000000000000000 mfi0: 6669 (boot + 94s/0x0002/info) - Inserted: PD 17(e0xff/s23) mfi0: 6670 (boot + 94s/0x0002/info) - Inserted: PD 17(e0xff/s23) Info: enclPd=ffff, scsiType=0, portMap=07, sasAddr=91813c23a6a092a6,0000000000000000 mfi0: 6671 (boot + 94s/0x0002/info) - Inserted: PD 18(e0xff/s24) mfi0: 6672 (boot + 94s/0x0002/info) - Inserted: PD 18(e0xff/s24) Info: enclPd=ffff, scsiType=0, portMap=08, sasAddr=91813b24a3a08da2,0000000000000000 mfi0: 6673 (boot + 94s/0x0002/info) - Inserted: PD 19(e0xff/s25) mfi0: 6674 (boot + 94s/0x0002/info) - Inserted: PD 19(e0xff/s25) Info: enclPd=ffff, scsiType=0, portMap=09, sasAddr=91813c23a6a08cab,0000000000000000 mfi0: 6675 (boot + 94s/0x0002/info) - Inserted: PD 1a(e0xff/s26) mfi0: 6676 (boot + 94s/0x0002/info) - Inserted: PD 1a(e0xff/s26) Info: enclPd=ffff, scsiType=0, portMap=0a, sasAddr=91813b24a5a094a8,0000000000000000 mfi0: 6677 (boot + 94s/0x0002/info) - Inserted: PD 1b(e0xff/s27) mfi0: 6678 (boot + 94s/0x0002/info) - Inserted: PD 1b(e0xff/s27) Info: enclPd=ffff, scsiType=0, portMap=0b, sasAddr=91813c21a0a88ca3,0000000000000000 mfi0: 6679 (boot + 94s/0x0002/info) - Inserted: PD 1c(e0xff/s28) mfi0: 6680 (boot + 94s/0x0002/info) - Inserted: PD 1c(e0xff/s28) Info: enclPd=ffff, scsiType=0, portMap=0c, sasAddr=91813c23a5a590a4,0000000000000000 mfi0: 6681 (boot + 94s/0x0002/info) - Inserted: PD 1d(e0xff/s29) mfi0: 6682 (boot + 94s/0x0002/info) - Inserted: PD 1d(e0xff/s29) Info: enclPd=ffff, scsiType=0, portMap=0d, sasAddr=91813b24a5a191a2,0000000000000000 mfi0: 6683 (boot + 94s/0x0002/info) - Inserted: PD 1e(e0xff/s30) mfi0: 6684 (boot + 94s/0x0002/info) - Inserted: PD 1e(e0xff/s30) Info: enclPd=ffff, scsiType=0, portMap=0e, sasAddr=91813c24a3a695a3,0000000000000000 mfi0: 6685 (boot + 94s/0x0002/info) - Inserted: PD 1f(e0xff/s31) mfi0: 6686 (boot + 94s/0x0002/info) - Inserted: PD 1f(e0xff/s31) Info: enclPd=ffff, scsiType=0, portMap=0f, sasAddr=91813a27a0a493a6,0000000000000000 mfi0: 6687 (330437870s/0x0020/info) - Time established as 06/21/10 12:17:50; (97 seconds since power on) mfi0: 6688 (boot + 0s/0x0020/info) - Firmware initialization started (PCI ID 0411/1000/1008/1000) mfi0: 6689 (boot + 0s/0x0020/info) - Firmware version 1.12.280-0826 mfi0: 6690 (boot + 51s/0x0008/info) - Battery temperature is normal mfi0: 6691 (boot + 51s/0x0008/info) - Battery Present mfi0: 6692 (boot + 51s/0x0020/info) - Board Revision ( mfi0: 6693 (boot + 98s/0x0002/info) - Inserted: PD 10(e0xff/s16) mfi0: 6694 (boot + 98s/0x0002/info) - Inserted: PD 10(e0xff/s16) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=91813c23a6a08faa,0000000000000000 mfi0: 6695 (boot + 98s/0x0002/info) - Inserted: PD 11(e0xff/s17) mfi0: 6696 (boot + 98s/0x0002/info) - Inserted: PD 11(e0xff/s17) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=91813c24a1a18ca8,0000000000000000 mfi0: 6697 (boot + 98s/0x0002/info) - Inserted: PD 12(e0xff/s18) mfi0: 6698 (boot + 98s/0x0002/info) - Inserted: PD 12(e0xff/s18) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=91813c24a29f8ca9,0000000000000000 mfi0: 6699 (boot + 98s/0x0002/info) - Inserted: PD 13(e0xff/s19) mfi0: 6700 (boot + 98s/0x0002/info) - Inserted: PD 13(e0xff/s19) Info: enclPd=ffff, scsiType=0, portMap=03, sasAddr=91813b24a5a591a3,0000000000000000 mfi0: 6701 (boot + 98s/0x0002/info) - Inserted: PD 14(e0xff/s20) mfi0: 6702 (boot + 98s/0x0002/info) - Inserted: PD 14(e0xff/s20) Info: enclPd=ffff, scsiType=0, portMap=04, sasAddr=918139229fa08fa4,0000000000000000 mfi0: 6703 (boot + 98s/0x0002/info) - Inserted: PD 15(e0xff/s21) mfi0: 6704 (boot + 98s/0x0002/info) - Inserted: PD 15(e0xff/s21) Info: enclPd=ffff, scsiType=0, portMap=05, sasAddr=91813c249fa68ca4,0000000000000000 mfi0: 6705 (boot + 98s/0x0002/info) - Inserted: PD 16(e0xff/s22) mfi0: 6706 (boot + 98s/0x0002/info) - Inserted: PD 16(e0xff/s22) Info: enclPd=ffff, scsiType=0, portMap=06, sasAddr=91813c1ea6a792a2,0000000000000000 mfi0: 6707 (boot + 98s/0x0002/info) - Inserted: PD 17(e0xff/s23) mfi0: 6708 (boot + 98s/0x0002/info) - Inserted: PD 17(e0xff/s23) Info: enclPd=ffff, scsiType=0, portMap=07, sasAddr=91813c23a6a092a6,0000000000000000 mfi0: 6709 (boot + 98s/0x0002/info) - Inserted: PD 18(e0xff/s24) mfi0: 6710 (boot + 98s/0x0002/info) - Inserted: PD 18(e0xff/s24) Info: enclPd=ffff, scsiType=0, portMap=08, sasAddr=91813b24a3a08da2,0000000000000000 mfi0: 6711 (boot + 98s/0x0002/info) - Inserted: PD 19(e0xff/s25) mfi0: 6712 (boot + 98s/0x0002/info) - Inserted: PD 19(e0xff/s25) Info: enclPd=ffff, scsiType=0, portMap=09, sasAddr=91813c23a6a08cab,0000000000000000 mfi0: 6713 (boot + 98s/0x0002/info) - Inserted: PD 1a(e0xff/s26) mfi0: 6714 (boot + 98s/0x0002/info) - Inserted: PD 1a(e0xff/s26) Info: enclPd=ffff, scsiType=0, portMap=0a, sasAddr=91813b24a5a094a8,0000000000000000 mfi0: 6715 (boot + 98s/0x0002/info) - Inserted: PD 1b(e0xff/s27) mfi0: 6716 (boot + 98s/0x0002/info) - Inserted: PD 1b(e0xff/s27) Info: enclPd=ffff, scsiType=0, portMap=0b, sasAddr=91813c21a0a88ca3,0000000000000000 mfi0: 6717 (boot + 99s/0x0002/info) - Inserted: PD 1c(e0xff/s28) mfi0: 6718 (boot + 99s/0x0002/info) - Inserted: PD 1c(e0xff/s28) Info: enclPd=ffff, scsiType=0, portMap=0c, sasAddr=91813c23a5a590a4,0000000000000000 mfi0: 6719 (boot + 99s/0x0002/info) - Inserted: PD 1d(e0xff/s29) mfi0: 6720 (boot + 99s/0x0002/info) - Inserted: PD 1d(e0xff/s29) Info: enclPd=ffff, scsiType=0, portMap=0d, sasAddr=91813b24a5a191a2,0000000000000000 mfi0: 6721 (boot + 99s/0x0002/info) - Inserted: PD 1e(e0xff/s30) mfi0: 6722 (boot + 99s/0x0002/info) - Inserted: PD 1e(e0xff/s30) Info: enclPd=ffff, scsiType=0, portMap=0e, sasAddr=91813c24a3a695a3,0000000000000000 mfi0: 6723 (boot + 99s/0x0002/info) - Inserted: PD 1f(e0xff/s31) mfi0: 6724 (boot + 99s/0x0002/info) - Inserted: PD 1f(e0xff/s31) Info: enclPd=ffff, scsiType=0, portMap=0f, sasAddr=91813a27a0a493a6,0000000000000000 mfi0: [ITHREAD] pcib4: at device 0.2 on pci2 pci4: on pcib4 pcib5: irq 26 at device 5.0 on pci0 pci5: on pcib5 pcib6: irq 30 at device 7.0 on pci0 pci6: on pcib6 pcib7: irq 32 at device 9.0 on pci0 pci7: on pcib7 pci0: at device 16.0 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 17.0 (no driver attached) pci0: at device 17.1 (no driver attached) pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) pci0: at device 22.2 (no driver attached) pci0: at device 22.3 (no driver attached) pci0: at device 22.4 (no driver attached) pci0: at device 22.5 (no driver attached) pci0: at device 22.6 (no driver attached) pci0: at device 22.7 (no driver attached) uhci0: port 0x20c0-0x20df irq 19 at device 26.0 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x1f30 usbus0: on uhci0 uhci1: port 0x20a0-0x20bf irq 19 at device 26.1 on pci0 uhci1: [ITHREAD] uhci1: LegSup = 0x0f30 usbus1: on uhci1 uhci2: port 0x2080-0x209f irq 19 at device 26.2 on pci0 uhci2: [ITHREAD] uhci2: LegSup = 0x0f30 usbus2: on uhci2 ehci0: mem 0xb1c21000-0xb1c213ff irq 19 at device 26.7 on pci0 ehci0: [ITHREAD] usbus3: EHCI version 1.0 usbus3: on ehci0 pcib8: irq 16 at device 28.0 on pci0 pci8: on pcib8 pcib9: irq 16 at device 28.4 on pci0 pci9: on pcib9 vgapci0: mem 0xb0000000-0xb0ffffff,0xb1800000-0xb1803fff,0xb1000000-0xb17fffff irq 16 at device 0.0 on pci9 uhci3: port 0x2060-0x207f irq 16 at device 29.0 on pci0 uhci3: [ITHREAD] uhci3: LegSup = 0x0f30 usbus4: on uhci3 uhci4: port 0x2040-0x205f irq 16 at device 29.1 on pci0 uhci4: [ITHREAD] uhci4: LegSup = 0x0f30 usbus5: on uhci4 uhci5: port 0x2020-0x203f irq 16 at device 29.2 on pci0 uhci5: [ITHREAD] uhci5: LegSup = 0x0f30 usbus6: on uhci5 ehci1: mem 0xb1c20000-0xb1c203ff irq 16 at device 29.7 on pci0 ehci1: [ITHREAD] usbus7: EHCI version 1.0 usbus7: on ehci1 pcib10: at device 30.0 on pci0 pci10: on pcib10 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x2110-0x211f,0x2100-0x210f irq 18 at device 31.2 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 31.3 (no driver attached) atapci1: port 0x2128-0x212f,0x2144-0x2147,0x2120-0x2127,0x2140-0x2143,0x20f0-0x20ff,0x20e0-0x20ef irq 21 at device 31.5 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] acpi_button0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] cpu0: on acpi0 ACPI Warning: \\_PR_.CPU0._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf0: invalid top level _PSS package device_attach: acpi_perf0 attach returned 6 acpi_perf0: invalid top level _PSS package device_attach: acpi_perf0 attach returned 6 est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est0 attach returned 6 p4tcc0: on cpu0 cpu1: on acpi0 ACPI Warning: \\_PR_.CPU1._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf1: invalid top level _PSS package device_attach: acpi_perf1 attach returned 6 acpi_perf1: invalid top level _PSS package device_attach: acpi_perf1 attach returned 6 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est1 attach returned 6 p4tcc1: on cpu1 cpu2: on acpi0 ACPI Warning: \\_PR_.CPU2._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf2: invalid top level _PSS package device_attach: acpi_perf2 attach returned 6 acpi_perf2: invalid top level _PSS package device_attach: acpi_perf2 attach returned 6 est2: on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est2 attach returned 6 p4tcc2: on cpu2 cpu3: on acpi0 ACPI Warning: \\_PR_.CPU3._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf3: invalid top level _PSS package device_attach: acpi_perf3 attach returned 6 acpi_perf3: invalid top level _PSS package device_attach: acpi_perf3 attach returned 6 est3: on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est3 attach returned 6 p4tcc3: on cpu3 cpu4: on acpi0 ACPI Warning: \\_PR_.CPU4._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf4: invalid top level _PSS package device_attach: acpi_perf4 attach returned 6 acpi_perf4: invalid top level _PSS package device_attach: acpi_perf4 attach returned 6 est4: on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est4 attach returned 6 p4tcc4: on cpu4 cpu5: on acpi0 ACPI Warning: \\_PR_.CPU5._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf5: invalid top level _PSS package device_attach: acpi_perf5 attach returned 6 acpi_perf5: invalid top level _PSS package device_attach: acpi_perf5 attach returned 6 est5: on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est5 attach returned 6 p4tcc5: on cpu5 cpu6: on acpi0 ACPI Warning: \\_PR_.CPU6._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf6: invalid top level _PSS package device_attach: acpi_perf6 attach returned 6 acpi_perf6: invalid top level _PSS package device_attach: acpi_perf6 attach returned 6 est6: on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est6 attach returned 6 p4tcc6: on cpu6 cpu7: on acpi0 ACPI Warning: \\_PR_.CPU7._PSS: Return type mismatch - found Integer, expected Package 20090521 nspredef-1051 acpi_perf7: invalid top level _PSS package device_attach: acpi_perf7 attach returned 6 acpi_perf7: invalid top level _PSS package device_attach: acpi_perf7 attach returned 6 est7: on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 11 device_attach: est7 attach returned 6 p4tcc7: on cpu7 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd: unable to get the current command byte value. atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to get the current command byte value. ppc0: cannot reserve I/O port range Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 acd0: DVDR at ata0-master SATA150 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 ad1: 1907729MB at ata0-slave SATA300 ad2: 1907729MB at ata1-master SATA300 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered mfi0: 6725 (330438117s/0x0020/info) - Time established as 06/21/10 12:21:57; (101 seconds since power on) mfid0: on mfi0 mfid0: 15251152MB (31234359296 sectors) RAID volume '' is optimal lapic4: Forcing LINT1 to edge trigger SMP: AP CPU #2 Launched! lapic16: Forcing LINT1 to edge trigger SMP: AP CPU #4 Launched! lapic6: Forcing LINT1 to edge trigger SMP: AP CPU #3 Launched! lapic20: Forcing LINT1 to edge trigger SMP: AP CPU #6 Launched! lapic2: Forcing LINT1 to edge trigger SMP: AP CPU #1 Launched! lapic18: Forcing LINT1 to edge trigger SMP: AP CPU #5 Launched! lapic22: Forcing LINT1 to edge trigger SMP: AP CPU #7 Launched! Root mount waiting for: usbus7 usbus3 uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered Root mount waiting for: usbus7 uhub_reattach_port:370: port 1 reset failed, error=USB_ERR_TIMEOUT uhub_reattach_port:456: device problem (USB_ERR_TIMEOUT), disabling port 1 Trying to mount root from ufs:/dev/ad1s1a WARNING: reducing size to maximum of 67108864 blocks per swap unit ugen2.2: at usbus2 ukbd0: on usbus2 kbd2 at ukbd0 ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 ugen4.2: at usbus4 ukbd1: on usbus4 kbd3 at ukbd1 ums1: on usbus4 ums1: 5 buttons and [XYZ] coordinates ID=1 igb1: link state changed to UP