From owner-freebsd-questions@FreeBSD.ORG Sat Jul 2 17:42:09 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC51A106566B for ; Sat, 2 Jul 2011 17:42:09 +0000 (UTC) (envelope-from freebsd@penx.com) Received: from Elmer.dco.penx.com (elmer.dco.penx.com [174.46.214.165]) by mx1.freebsd.org (Postfix) with ESMTP id A417B8FC0C for ; Sat, 2 Jul 2011 17:42:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by Elmer.dco.penx.com (8.14.5/8.14.4) with ESMTP id p62Hg6FA023761 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 2 Jul 2011 11:42:08 -0600 (MDT) (envelope-from freebsd@penx.com) Date: Sat, 2 Jul 2011 11:42:06 -0600 (MDT) From: Dennis Glatting X-X-Sender: dennisg@Elmer.dco.penx.com To: freebsd-questions@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: Trying to understand the mechanics of boot X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2011 17:42:09 -0000 I have a system with a large number of disks whose dmesg output is the last data component of this message, preceded by my zpool configuration which is preceded by my fstab. I boot off of ad4, which is the MARVELL Raid configured for RAID1 across two 1TB disks. The problem I have is if I add more disks to the Areca adapter, which is configured JBOD, or the ICH10, which is also configured JBOD, the BIOS sees the RAID1 disk, selects it, but when it starts to boot the BIOS appears to be the entity that prints the message: Reboot and Select proper boot device or insert boot media in selected boot device and press a key My motherboard is an ASUS RAMPAGE III Extreme but I have tried a different MB, in particular a Gigabyte board, with the same result. I do not see any hint of FreeBSD in the boot process and therefore do not know if the BIOS accesses the MBR of my RAID1 array or whetehr the problem is the BIOS or FreeBSD boot sequence. I do understand disk devices will move as one adds or removes disks, having had to edit fstab from CD boot many times previously, but I don't think that is the problem I am having, not to say it isn't a problem later in the boot sequence. I'm stumped how to go forward with debugging this problem. It appears to be some limitation is reached in the BIOSes. I am hopping someone with much more detailed knowledge of low level boot/disk sequences can point me in a direction. iirc> cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/ad4s1b none swap sw 0 0 /dev/ad4s1a / ufs rw 1 1 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 iirc> zpool status pool: disk-1 state: ONLINE scan: scrub repaired 0 in 1h8m with 0 errors on Sat Jul 2 04:10:41 2011 config: NAME STATE READ WRITE CKSUM disk-1 ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 errors: No known data errors pool: disk-2 state: ONLINE scan: scrub repaired 0 in 0h3m with 0 errors on Sat Jul 2 03:05:38 2011 config: NAME STATE READ WRITE CKSUM disk-2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ad34 ONLINE 0 0 0 ad36 ONLINE 0 0 0 cache ad20 ONLINE 0 0 0 errors: No known data errors iirc> dmesg Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-STABLE #25: Sat Jun 25 17:34:51 PDT 2011 root@iirc:/usr/src/sys/amd64/compile/IIRC amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM) i7 CPU 940 @ 2.93GHz (2960.51-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106a4 Family = 6 Model = 1a Stepping = 4 Features=0xbfebfbff Features2=0x98e3bd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant real memory = 12884901888 (12288 MB) avail memory = 12371382272 (11798 MB) ACPI APIC Table: <031711 APIC1638> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: <031711 XSDT1638> on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of fee00000, 1000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bff00000 (3) failed Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 atapci0: port 0xac00-0xac07,0xa880-0xa883,0xa800-0xa807,0xa480-0xa483,0xa400-0xa40f mem 0xfb9ff800-0xfb9fffff irq 28 at device 0.0 on pci1 atapci0: [ITHREAD] atapci0: AHCI v1.20 controller with 8 6Gbps ports, PM not supported ata2: on atapci0 ata2: [ITHREAD] ata3: on atapci0 ata3: [ITHREAD] ata4: on atapci0 ata4: [ITHREAD] ata5: on atapci0 ata5: [ITHREAD] ata6: on atapci0 ata6: [ITHREAD] ata7: on atapci0 ata7: [ITHREAD] ata8: on atapci0 ata8: [ITHREAD] ata9: on atapci0 ata9: [ITHREAD] pcib2: at device 2.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) pcib3: at device 3.0 on pci0 pci3: on pcib3 vgapci0: port 0xb000-0xb0ff mem 0xd0000000-0xdfffffff,0xfbbe0000-0xfbbeffff irq 24 at device 0.0 on pci3 pci3: at device 0.1 (no driver attached) pcib4: at device 5.0 on pci0 pci4: on pcib4 arcmsr0: port 0xc800-0xc8ff mem 0xfbcf0000-0xfbcfffff,0xfbc80000-0xfbcbffff irq 26 at device 0.0 on pci4 ARECA RAID ADAPTER0: Driver Version 1.20.00.21 2010-03-03 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.49 2010-12-10 arcmsr0: [ITHREAD] pcib5: at device 7.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) em0: port 0x9c00-0x9c1f mem 0xfb8e0000-0xfb8fffff,0xfb8df000-0xfb8dffff irq 20 at device 25.0 on pci0 em0: Using an MSI interrupt em0: [FILTER] em0: Ethernet address: f4:6d:04:01:c6:fc uhci0: port 0x9480-0x949f irq 16 at device 26.0 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x2f00 usbus0: on uhci0 uhci1: port 0x9800-0x981f irq 21 at device 26.1 on pci0 uhci1: [ITHREAD] uhci1: LegSup = 0x2f00 usbus1: on uhci1 uhci2: port 0x9880-0x989f irq 19 at device 26.2 on pci0 uhci2: [ITHREAD] uhci2: LegSup = 0x2f00 usbus2: on uhci2 ehci0: mem 0xfb8de000-0xfb8de3ff irq 18 at device 26.7 on pci0 ehci0: [ITHREAD] usbus3: EHCI version 1.0 usbus3: on ehci0 pcib6: irq 17 at device 28.0 on pci0 pci7: on pcib6 pcib7: irq 16 at device 0.0 on pci7 pci8: on pcib7 atapci1: port 0xec00-0xec0f mem 0xfbeffc00-0xfbeffc7f,0xfbef0000-0xfbef7fff irq 16 at device 0.0 on pci8 atapci1: [ITHREAD] ata10: on atapci1 ata10: [ITHREAD] ata11: on atapci1 ata11: [ITHREAD] ata12: on atapci1 ata12: [ITHREAD] ata13: on atapci1 ata13: [ITHREAD] pcib8: irq 17 at device 28.4 on pci0 pci6: on pcib8 atapci2: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f mem 0xfbdfe000-0xfbdfffff irq 16 at device 0.0 on pci6 atapci2: [ITHREAD] atapci3: on atapci2 atapci3: [ITHREAD] atapci3: AHCI v1.00 controller with 2 3Gbps ports, PM supported ata14: on atapci3 ata14: [ITHREAD] ata15: on atapci3 ata15: [ITHREAD] ata16: on atapci2 ata16: [ITHREAD] uhci3: port 0x9000-0x901f irq 23 at device 29.0 on pci0 uhci3: [ITHREAD] uhci3: LegSup = 0x2f00 usbus4: on uhci3 uhci4: port 0x9080-0x909f irq 19 at device 29.1 on pci0 uhci4: [ITHREAD] uhci4: LegSup = 0x2f00 usbus5: on uhci4 uhci5: port 0x9400-0x941f irq 18 at device 29.2 on pci0 uhci5: [ITHREAD] uhci5: LegSup = 0x2f00 usbus6: on uhci5 ehci1: mem 0xfb8dd000-0xfb8dd3ff irq 23 at device 29.7 on pci0 ehci1: [ITHREAD] usbus7: EHCI version 1.0 usbus7: on ehci1 pcib9: at device 30.0 on pci0 pci9: on pcib9 isab0: at device 31.0 on pci0 isa0: on isab0 atapci4: port 0x8880-0x8887,0x8800-0x8803,0x8480-0x8487,0x8400-0x8403,0x8080-0x809f mem 0xfb8db000-0xfb8db7ff irq 20 at device 31.2 on pci0 atapci4: [ITHREAD] atapci4: AHCI called from vendor specific driver atapci4: AHCI v1.20 controller with 6 3Gbps ports, PM supported ata17: on atapci4 ata17: [ITHREAD] ata18: on atapci4 ata18: [ITHREAD] ata19: on atapci4 ata19: [ITHREAD] ata20: on atapci4 ata20: [ITHREAD] ata21: on atapci4 ata21: [ITHREAD] ata22: on atapci4 ata22: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 qpi0: on motherboard pcib10: pcibus 255 on qpi0 pci255: on pcib10 orm0: at iomem 0xd0000-0xd4fff,0xd5000-0xd7fff,0xd9800-0xdc7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ppc0: cannot reserve I/O port range coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x32c offMax=0x56c usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered ad4: 1906688MB at ata2-master UDMA100 SATA 1.5Gb/s ad20: 57241MB at ata10-master UDMA100 SATA 3Gb/s ad22: 57241MB at ata11-master UDMA100 SATA 3Gb/s acd0: DVDR at ata15-master UDMA100 SATA 1.5Gb/s ad34: 1907729MB at ata17-master UDMA100 SATA 3Gb/s ad36: 1907729MB at ata18-master UDMA100 SATA 3Gb/s ad38: 1907729MB at ata19-master UDMA100 SATA 3Gb/s ad40: 1907729MB at ata20-master UDMA100 SATA 3Gb/s uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 sks=0x40 0x00 0x01 (probe18:ata15:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe18:ata15:0:0:0): CAM status: SCSI Status Error (probe18:ata15:0:0:0): SCSI status: Check Condition (probe18:ata15:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) da0 at arcmsr0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da0: Command Queueing enabled da0: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) da1 at arcmsr0 bus 0 scbus0 target 0 lun 1 da1: Fixed Direct Access SCSI-5 device da1: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da1: Command Queueing enabled da1: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) da2 at arcmsr0 bus 0 scbus0 target 0 lun 2 da2: Fixed Direct Access SCSI-5 device da2: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da2: Command Queueing enabled da2: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) da3 at arcmsr0 bus 0 scbus0 target 0 lun 3 da3: Fixed Direct Access SCSI-5 device da3: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da3: Command Queueing enabled da3: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) da4 at arcmsr0 bus 0 scbus0 target 0 lun 4 da4: Fixed Direct Access SCSI-5 device da4: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da4: Command Queueing enabled da4: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) da5 at arcmsr0 bus 0 scbus0 target 0 lun 5 da5: Fixed Direct Access SCSI-5 device da5: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da5: Command Queueing enabled da5: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) pass6 at arcmsr0 bus 0 scbus0 target 16 lun 0 pass6: Fixed Processor SCSI-0 device SMP: AP CPU #1 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #6 Launched! cd0 at ata15 bus 0 scbus5 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device SMP: AP CPU #2 Launched! cd0: 100.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed Trying to mount root from ufs:/dev/ad4s1a ugen6.2: at usbus6 ums0: on usbus6 ums0: 3 buttons and [XYZ] coordinates ID=0 ugen5.2: at usbus5 ZFS filesystem version 5 ZFS storage pool version 28 ugen6.3: at usbus6 ukbd0: on usbus6 kbd2 at ukbd0 uhid0: on usbus6 vboxnet0: Ethernet address: 0a:00:27:00:00:00 em0: link state changed to UP drm0: on vgapci0 info: [drm] MSI enabled 1 message(s) info: [drm] Initialized radeon 1.31.0 20080613