From owner-freebsd-sparc64@freebsd.org Fri May 4 08:11:16 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEAECFCB8FA for ; Fri, 4 May 2018 08:11:15 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from mail05.SiriusCloud.jp (mail05.SiriusCloud.jp [219.118.72.5]) by mx1.freebsd.org (Postfix) with ESMTP id 1C4E176182 for ; Fri, 4 May 2018 08:11:14 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from W701ds (124x37x209x41.ap124.ftth.ucom.ne.jp [124.37.209.41]) (Authenticated sender: iwama@t3.rim.or.jp) by access05.SiriusCloud.jp (Postfix) with ESMTPA id 40ckxf6Ldrz1XLgNp for ; Fri, 4 May 2018 17:01:58 +0900 (JST) Authentication-Results: access05.SiriusCloud.jp; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral From: To: Subject: Fire V445 support status Date: Fri, 4 May 2018 17:01:56 +0900 Message-ID: <000001d3e37e$2c1d52a0$8457f7e0$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdPjeY+2NCYiRSDRS/GXtaULC6zdGw== Content-Language: ja X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 08:11:16 -0000 I have tried installing 11.1-RELEASE after understanding that Sun Fire V445 is not on the support list as shown in the following page. https://www.freebsd.org/releases/11.1R/hardware.html#proc-sparc64 Sure enough, I was hit by an event hanging up during installation. I tried installing on other versions, but these were similar: 11.0-RELEASE 10.4-RELEASE 10.3-RELEASE 10.2-RELEASE I tried 12.0-CURRENT, but because even kernel did not start, I tried 3 or 4 snapshots, but I decided not to use this branch. Also, I tried it on NetBSD and OpenBSD and found the following results: NetBSD-7.1.2: failed OpenBSD-6.3: success Since OpenBSD seemed to be processing with a single processor at the time of installation, likewise in FreeBSD-11.1-RELEASE, "set kern.smp.disabled=1" was set with boot loader, but it hung up. (After rebooting after installation, OpenBSD is running with multi-processors.) Does anyone know how to use FreeBSD with Fire V445? Or is it obvious that I need the kernel code to support Fire V445? The following is a log when the installer hangs up with FreeBSD-11.1-RELEASE: === {3} ok boot cdrom SC Alert: Host System has Reset Probing system devices Probing memory Probing I/O buses screen not found. keyboard not found. Keyboard not present. Using ttya for input and output. Probing system devices Probing memory Probing I/O buses Sun Fire V445, No Keyboard Copyright 2007 Sun Microsystems, Inc. All rights reserved. OpenBoot 4.22.33, 8192 MB memory installed, Serial #XXXXXXXX. Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. Initializing 1MB of memory at addr 323ff20000 - Initializing 1MB of memory at addr 323fefe000 - Initializing 1MB of memory at addr 323fefa000 - Initializing 1MB of memory at addr 323fef0000 - Initializing 1MB of memory at addr 323fed0000 - Initializing 1MB of memory at addr 323fde0000 - Initializing 14MB of memory at addr 323f000000 - Initializing 16MB of memory at addr 323e008000 - Initializing 992MB of memory at addr 3200000000 /- Initializing 1024MB of memory at addr 3000000000 /- Initializing 1024MB of memory at addr 2200000000 /- Initializing 1024MB of memory at addr 2000000000 /- Initializing 1024MB of memory at addr 1200000000 /- Initializing 1024MB of memory at addr 1000000000 /- Initializing 1024MB of memory at addr 200000000 /- Initializing 1024MB of memory at addr 0 /- Rebooting with command: boot cdrom Boot device: /pci@1f,700000/pci@0/pci@1/pci@0/ide@1f/cdrom@0,0:f File and args: >> FreeBSD/sparc64 boot block Boot path: /pci@1f,700000/pci@0/pci@1/pci@0/ide@1f/cdrom@0,0:f Boot loader: /boot/loader Consoles: Open Firmware console FreeBSD/sparc64 bootstrap loader, Revision 1.0 (Fri Jul 21 08:44:40 UTC 2017 root@releng2.nyi.freebsd.org) bootpath="/pci@1f,700000/pci@0/pci@1/pci@0/ide@1f/cdrom@0,0:a" |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/Loading /boot/defaults/loader.conf -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/kernel data=0xc47740+0xf4078 /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|syms=[0x8+0xd5a38/-\|/-\|/-\|/-\|/-\|/-\|/-\+0x8+0xc746a|/-\|/-\ |/-\|/-\|/-\|/-\|] /-\|/-\|/-\| Hit [Enter] to boot immediately, or any other key for command prompt. Booting [/boot/kernel/kernel] in 9 seconds... Booting [/boot/kernel/kernel] in 8 seconds... Type '?' for a list of commands, 'help' for more detailed help. OK boot -v Booting... /-\|/jumping to kernel entry at 0xc00b0000. Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 08:46:54 UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/sparc64.sparc64/usr/src/sys/GENERIC sparc64 gcc version 4.2.1 20070831 patched [FreeBSD] Preloaded elf kernel "/boot/kernel/kernel" at 0xc0edc000. real memory = 8589934592 (8192 MB) avail memory = 8393572352 (8004 MB) machine: SUNW,Sun-Fire-V445 cpu0: Sun Microsystems UltraSparc-IIIi Processor (1592.00 MHz CPU) mask=0x34 maxtl=5 maxwin=7 initalizing intr_countp cpu1: Sun Microsystems UltraSparc-IIIi Processor (1592.00 MHz CPU) mask=0x34 maxtl=5 maxwin=7 INTR: Adding CPU 1 as a target cpu2: Sun Microsystems UltraSparc-IIIi Processor (1592.00 MHz CPU) mask=0x34 maxtl=5 maxwin=7 INTR: Adding CPU 2 as a target cpu3: Sun Microsystems UltraSparc-IIIi Processor (1592.00 MHz CPU) mask=0x34 maxtl=5 maxwin=7 INTR: Adding CPU 3 as a target FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs ULE: setup cpu 0 ULE: setup cpu 1 ULE: setup cpu 2 ULE: setup cpu 3 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 random: entropy device external interface wlan: <802.11 Link Layer> firmware: 'isp_1000' version 1: 20142 bytes loaded at 0xc09a5218 firmware: 'isp_1040' version 1: 22944 bytes loaded at 0xc09aa0c6 firmware: 'isp_1080' version 1: 31350 bytes loaded at 0xc09afa66 firmware: 'isp_12160' version 1: 28050 bytes loaded at 0xc09b74dc firmware: 'isp_2100' version 1: 76770 bytes loaded at 0xc09be26e firmware: 'isp_2200' version 1: 84566 bytes loaded at 0xc09d0e50 firmware: 'isp_2300' version 1: 125252 bytes loaded at 0xc09e58a6 firmware: 'isp_2322' version 1: 120814 bytes loaded at 0xc0a041ea firmware: 'isp_2400' version 1: 204384 bytes loaded at 0xc0a25550 firmware: 'isp_2500' version 1: 223908 bytes loaded at 0xc0a64f80 kbd0 at kbdmux0 nfslock: pseudo-device crypto: mem: null: openfirm: nexus0: pcib0: mem 0x4000f600000-0x4000f6effff,0x4000f410000-0x4000f473fff irq 1983,1982 on nexus0 pcib0: Fire, module-revision 3, IGN 0x1e Timecounter "pcib0" frequency 199000000 Hz quality 100 pcib0: DVMA map: 0xc0000000 to 0xffffffff 131072 entries pcib0: PROM IOTSB size: 0 (1024 entries) pcib0: bus range 2 to 9; PCI bus 2 pci0: on pcib0 pci0: domain=0, physical bus=2 found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=0, bus=2, slot=0, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0x100000, size 17, enabled pcib1: mem 0x100000-0x11ffff at device 0.0 on pci0 pcib1: domain 0 pcib1: secondary bus 3 pcib1: subordinate bus 9 pcib1: I/O decode 0x1000-0x1fff pcib1: memory decode 0x200000-0xbfffff pci1: on pcib1 pci1: domain=0, physical bus=3 found-> vendor=0x10b5, dev=0x8532, revid=0x03 domain=0, bus=3, slot=1, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0x0a domain=0, bus=3, slot=2, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=0, bus=3, slot=8, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=0, bus=3, slot=9, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit pcib2: at device 1.0 on pci1 pcib1: allocated memory range (0x200000-0x7fffff) for rid 20 of pcib2 pcib2: domain 0 pcib2: secondary bus 4 pcib2: subordinate bus 6 pcib2: memory decode 0x200000-0x7fffff pci2: on pcib2 pci2: domain=0, physical bus=4 found-> vendor=0x1166, dev=0x0103, revid=0xb5 domain=0, bus=4, slot=0, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 pcib3: at device 0.0 on pci2 pcib2: allocated memory range (0x200000-0x7fffff) for rid 20 of pcib3 pcib3: domain 0 pcib3: secondary bus 5 pcib3: subordinate bus 6 pcib3: memory decode 0x200000-0x7fffff pci3: on pcib3 pci3: domain=0, physical bus=5 found-> vendor=0x14e4, dev=0x1668, revid=0xa3 domain=0, bus=5, slot=4, func=0 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0xf8 (7440 ns), mingnt=0x40 (16000 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0x400000, size 16, memory disabled pcib3: allocated memory range (0x400000-0x40ffff) for rid 10 of pci0:5:4:0 map[18]: type Memory, range 64, base 0x210000, size 16, enabled pcib3: allocated memory range (0x210000-0x21ffff) for rid 18 of pci0:5:4:0 found-> vendor=0x14e4, dev=0x1668, revid=0xa3 domain=0, bus=5, slot=4, func=1 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0xf8 (7440 ns), mingnt=0x40 (16000 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0x600000, size 16, memory disabled pcib3: allocated memory range (0x600000-0x60ffff) for rid 10 of pci0:5:4:1 map[18]: type Memory, range 64, base 0x220000, size 16, enabled pcib3: allocated memory range (0x220000-0x22ffff) for rid 18 of pci0:5:4:1 found-> vendor=0x1166, dev=0x0104, revid=0xb4 domain=0, bus=5, slot=8, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0230, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) pcib4: at device 8.0 on pci3 pcib1: attempting to grow I/O port window for (0x1000-0xffff,0xf000) back candidate range: 0x1000-0xffff pcib1: grew I/O port window to 0x1000-0xffff pcib1: allocated I/O port range (0x1000-0xffff) for rid 1c of pcib2 pcib2: allocated initial I/O port window of 0x1000-0xffff pcib2: allocated I/O port range (0x1000-0xffff) for rid 1c of pcib3 pcib3: allocated initial I/O port window of 0x1000-0xffff pcib3: allocated I/O port range (0x1000-0xffff) for rid 1c of pcib4 pcib4: domain 0 pcib4: secondary bus 6 pcib4: subordinate bus 6 pcib4: I/O decode 0x1000-0xffff pci4: on pcib4 pci4: domain=0, physical bus=6 random: harvesting attach, 8 bytes (4 bits) from pci4 random: harvesting attach, 8 bytes (4 bits) from pcib4 random: harvesting attach, 8 bytes (4 bits) from pci3 random: harvesting attach, 8 bytes (4 bits) from pcib3 random: harvesting attach, 8 bytes (4 bits) from pci2 random: harvesting attach, 8 bytes (4 bits) from pcib2 pcib5: at device 2.0 on pci1 pcib1: attempting to grow I/O port window for (0x1000-0x1fff,0x1000) pcib5: failed to allocate initial I/O port window: 0x1000-0x1fff pcib1: allocated memory range (0x800000-0xbfffff) for rid 20 of pcib5 pcib5: domain 0 pcib5: secondary bus 7 pcib5: subordinate bus 7 pcib5: memory decode 0x800000-0xbfffff pci5: on pcib5 pci5: domain=0, physical bus=7 found-> vendor=0x1000, dev=0x0058, revid=0x02 domain=0, bus=7, slot=0, func=0 class=01-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit MSI-X supports 1 message in map 0x14 map[10]: type I/O Port, range 32, base 0x1000, size 8, port disabled pcib1: attempting to grow I/O port window for (0x1000-0x1fff,0x1000) pcib5: failed to allocate initial I/O port window (0x1000-0x10ff,0x100) pci5: pci0:7:0:0 bar 0x10 failed to allocate map[14]: type Memory, range 64, base 0x800000, size 14, enabled pcib5: allocated memory range (0x800000-0x803fff) for rid 14 of pci0:7:0:0 map[1c]: type Memory, range 64, base 0x810000, size 16, enabled pcib5: allocated memory range (0x810000-0x81ffff) for rid 1c of pci0:7:0:0 random: harvesting attach, 8 bytes (4 bits) from pci5 random: harvesting attach, 8 bytes (4 bits) from pcib5 pcib6: at device 8.0 on pci1 pcib1: attempting to grow I/O port window for (0x2000-0xffff,0xe000) pcib6: failed to allocate initial I/O port window: 0x2000-0xffff pcib6: domain 0 pcib6: secondary bus 8 pcib6: subordinate bus 8 pci6: on pcib6 pci6: domain=0, physical bus=8 random: harvesting attach, 8 bytes (4 bits) from pci6 random: harvesting attach, 8 bytes (4 bits) from pcib6 pcib7: at device 9.0 on pci1 pcib1: attempting to grow I/O port window for (0x2000-0xffff,0xe000) pcib7: failed to allocate initial I/O port window: 0x2000-0xffff pcib7: domain 0 pcib7: secondary bus 9 pcib7: subordinate bus 9 pci7: on pcib7 pci7: domain=0, physical bus=9 random: harvesting attach, 8 bytes (4 bits) from pci7 random: harvesting attach, 8 bytes (4 bits) from pcib7 random: harvesting attach, 8 bytes (4 bits) from pci1 random: harvesting attach, 8 bytes (4 bits) from pcib1 random: harvesting attach, 8 bytes (4 bits) from pci0 random: harvesting attach, 8 bytes (4 bits) from pcib0 pcib8: mem 0x4000ff00000-0x4000ffeffff,0x4000fc10000-0x4000fc73fff irq 2047,2046 on nexus0 pcib8: Fire, module-revision 3, IGN 0x1f pcib8: DVMA map: 0xc0000000 to 0xffffffff 131072 entries pcib8: PROM IOTSB size: 0 (1024 entries) pcib8: bus range 2 to 10; PCI bus 2 pci8: on pcib8 pci8: domain=8, physical bus=2 found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=8, bus=2, slot=0, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0x100000, size 17, enabled pcib9: mem 0x100000-0x11ffff at device 0.0 on pci8 pcib9: domain 8 pcib9: secondary bus 3 pcib9: subordinate bus 10 pcib9: I/O decode 0x0-0x1fff pcib9: memory decode 0x200000-0x45fffff pci9: on pcib9 pci9: domain=8, physical bus=3 found-> vendor=0x10b5, dev=0x8532, revid=0x03 domain=8, bus=3, slot=1, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0x0b domain=8, bus=3, slot=2, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=8, bus=3, slot=8, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit found-> vendor=0x10b5, dev=0x8532, revid=0xba domain=8, bus=3, slot=9, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 1 supports D0 D3 current D0 MSI supports 1 message, 64 bit pcib10: at device 1.0 on pci9 pcib9: allocated I/O port range (0-0x1fff) for rid 1c of pcib10 pcib9: allocated memory range (0x200000-0x3ffffff) for rid 20 of pcib10 pcib10: domain 8 pcib10: secondary bus 4 pcib10: subordinate bus 5 pcib10: I/O decode 0x0-0x1fff pcib10: memory decode 0x200000-0x3ffffff pci10: on pcib10 pci10: domain=8, physical bus=4 found-> vendor=0x10b9, dev=0x5249, revid=0x00 domain=8, bus=4, slot=0, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x2010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 pcib11: at device 0.0 on pci10 pcib10: allocated I/O port range (0x1000-0x1fff) for rid 1c of pcib11 pcib10: allocated memory range (0x200000-0x3ffffff) for rid 20 of pcib11 pcib11: domain 8 pcib11: secondary bus 5 pcib11: subordinate bus 5 pcib11: I/O decode 0x1000-0x1fff pcib11: memory decode 0x200000-0x3ffffff pcib11: special decode subtractive pci11: on pcib11 pci11: domain=8, physical bus=5 found-> vendor=0x10b9, dev=0x5237, revid=0x03 domain=8, bus=5, slot=28, func=0 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x50 (20000 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 1 message map[10]: type Memory, range 32, base 0x1000000, size 12, memory disabled pcib11: allocated memory range (0x1000000-0x1000fff) for rid 10 of pci8:5:28:0 found-> vendor=0x10b9, dev=0x5237, revid=0x03 domain=8, bus=5, slot=28, func=1 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x50 (20000 ns) intpin=b, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 1 message map[10]: type Memory, range 32, base 0x2000000, size 12, memory disabled pcib11: allocated memory range (0x2000000-0x2000fff) for rid 10 of pci8:5:28:1 found-> vendor=0x10b9, dev=0x5237, revid=0x03 domain=8, bus=5, slot=28, func=2 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x50 (20000 ns) intpin=c, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 1 message map[10]: type Memory, range 32, base 0x3000000, size 12, memory disabled pcib11: allocated memory range (0x3000000-0x3000fff) for rid 10 of pci8:5:28:2 found-> vendor=0x10b9, dev=0x5239, revid=0x01 domain=8, bus=5, slot=28, func=3 class=0c-03-20, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0x80 (3840 ns), mingnt=0x10 (4000 ns), maxlat=0x20 (8000 ns) intpin=d, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type Memory, range 32, base 0x200000, size 8, memory disabled pcib11: allocated memory range (0x200000-0x2000ff) for rid 10 of pci8:5:28:3 found-> vendor=0x10b9, dev=0x1575, revid=0x00 domain=8, bus=5, slot=30, func=0 class=06-01-00, hdrtype=0x00, mfdev=0 cmdreg=0x000f, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x08 (240 ns), mingnt=0x01 (250 ns), maxlat=0x18 (6000 ns) found-> vendor=0x10b9, dev=0x5229, revid=0xc8 domain=8, bus=5, slot=31, func=0 class=01-01-ff, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type I/O Port, range 32, base 0x1040, size 3, port disabled pcib11: allocated I/O port range (0x1040-0x1047) for rid 10 of pci8:5:31:0 map[14]: type I/O Port, range 32, base 0x1080, size 2, enabled pcib11: allocated I/O port range (0x1080-0x1083) for rid 14 of pci8:5:31:0 map[18]: type I/O Port, range 32, base 0x10c0, size 3, enabled pcib11: allocated I/O port range (0x10c0-0x10c7) for rid 18 of pci8:5:31:0 map[1c]: type I/O Port, range 32, base 0x1100, size 2, enabled pcib11: allocated I/O port range (0x1100-0x1103) for rid 1c of pci8:5:31:0 map[20]: type I/O Port, range 32, base 0x1000, size 4, enabled pcib11: allocated I/O port range (0x1000-0x100f) for rid 20 of pci8:5:31:0 random: harvesting attach, 8 bytes (4 bits) from pci11 random: harvesting attach, 8 bytes (4 bits) from pcib11 random: harvesting attach, 8 bytes (4 bits) from pci10 random: harvesting attach, 8 bytes (4 bits) from pcib10 pcib12: at device 2.0 on pci9 pcib9: allocated memory range (0x4000000-0x45fffff) for rid 20 of pcib12 pcib12: domain 8 pcib12: secondary bus 6 pcib12: subordinate bus 8 pcib12: memory decode 0x4000000-0x45fffff pci12: on pcib12 pci12: domain=8, physical bus=6 found-> vendor=0x1166, dev=0x0103, revid=0xb5 domain=8, bus=6, slot=0, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 pcib13: at device 0.0 on pci12 pcib12: allocated memory range (0x4000000-0x45fffff) for rid 20 of pcib13 pcib13: domain 8 pcib13: secondary bus 7 pcib13: subordinate bus 8 pcib13: memory decode 0x4000000-0x45fffff pci13: on pcib13 pci13: domain=8, physical bus=7 found-> vendor=0x14e4, dev=0x1668, revid=0xa3 domain=8, bus=7, slot=4, func=0 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0xf8 (7440 ns), mingnt=0x40 (16000 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0x4200000, size 16, memory disabled pcib13: allocated memory range (0x4200000-0x420ffff) for rid 10 of pci8:7:4:0 map[18]: type Memory, range 64, base 0x4010000, size 16, enabled pcib13: allocated memory range (0x4010000-0x401ffff) for rid 18 of pci8:7:4:0 found-> vendor=0x14e4, dev=0x1668, revid=0xa3 domain=8, bus=7, slot=4, func=1 class=02-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x02b0, cachelnsz=16 (dwords) lattimer=0xf8 (7440 ns), mingnt=0x40 (16000 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0x4400000, size 16, memory disabled pcib13: allocated memory range (0x4400000-0x440ffff) for rid 10 of pci8:7:4:1 map[18]: type Memory, range 64, base 0x4020000, size 16, enabled pcib13: allocated memory range (0x4020000-0x402ffff) for rid 18 of pci8:7:4:1 found-> vendor=0x1166, dev=0x0104, revid=0xb4 domain=8, bus=7, slot=8, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0230, cachelnsz=16 (dwords) lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) pcib14: at device 8.0 on pci13 pcib9: attempting to grow I/O port window for (0x2000-0xffff,0xe000) back candidate range: 0x2000-0xffff pcib9: grew I/O port window to 0-0xffff pcib9: allocated I/O port range (0x2000-0xffff) for rid 1c of pcib12 pcib12: allocated initial I/O port window of 0x2000-0xffff pcib12: allocated I/O port range (0x2000-0xffff) for rid 1c of pcib13 pcib13: allocated initial I/O port window of 0x2000-0xffff pcib13: allocated I/O port range (0x2000-0xffff) for rid 1c of pcib14 pcib14: domain 8 pcib14: secondary bus 8 pcib14: subordinate bus 8 pcib14: I/O decode 0x2000-0xffff pci14: on pcib14 pci14: domain=8, physical bus=8 random: harvesting attach, 8 bytes (4 bits) from pci14 random: harvesting attach, 8 bytes (4 bits) from pcib14 random: harvesting attach, 8 bytes (4 bits) from pci13 random: harvesting attach, 8 bytes (4 bits) from pcib13 random: harvesting attach, 8 bytes (4 bits) from pci12 random: harvesting attach, 8 bytes (4 bits) from pcib12 pcib15: at device 8.0 on pci9 pcib9: attempting to grow I/O port window for (0x2000-0xffff,0xe000) pcib15: failed to allocate initial I/O port window: 0x2000-0xffff pcib15: domain 8 pcib15: secondary bus 9 pcib15: subordinate bus 9 pci15: on pcib15 pci15: domain=8, physical bus=9 random: harvesting attach, 8 bytes (4 bits) from pci15 random: harvesting attach, 8 bytes (4 bits) from pcib15 pcib16: at device 9.0 on pci9 pcib9: attempting to grow I/O port window for (0x2000-0xffff,0xe000) pcib16: failed to allocate initial I/O port window: 0x2000-0xffff pcib16: domain 8 pcib16: secondary bus 10 pcib16: subordinate bus 10 pci16: on pcib16 pci16: domain=8, physical bus=10 random: harvesting attach, 8 bytes (4 bits) from pci16 random: harvesting attach, 8 bytes (4 bits) from pcib16 random: harvesting attach, 8 bytes (4 bits) from pci9 random: harvesting attach, 8 bytes (4 bits) from pcib9 random: harvesting attach, 8 bytes (4 bits) from pci8 random: harvesting attach, 8 bytes (4 bits) from pcib8 random: harvesting attach, 8 bytes (4 bits) from nexus0 ebus0: mem 0x4000fc64000-0x4000fc6401f on nexus0 random: harvesting attach, 8 bytes (4 bits) from ebus0 nexus0: mem 0x40000000000-0x40000000007 type memory-controller (no driver attached) nexus0: mem 0x40000800000-0x40000800007 type memory-controller (no driver attached) nexus0: mem 0x40001000000-0x40001000007 type memory-controller (no driver attached) nexus0: mem 0x40001800000-0x40001800007 type memory-controller (no driver attached) bge0: mem 0x400000-0x40ffff,0x210000-0x21ffff at device 4.0 on pci3 bge0: attempting to allocate 1 MSI vectors (8 supported) bge0: using IRQ 0 for MSI bge0: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: OUI 0x001018, model 0x0034, rev. 0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow random: harvesting attach, 8 bytes (4 bits) from brgphy0 random: harvesting attach, 8 bytes (4 bits) from miibus0 bge0: bpf attached bge0: Ethernet address: 00:14:4f:2b:b1:84 random: harvesting attach, 8 bytes (4 bits) from bge0 bge1: mem 0x600000-0x60ffff,0x220000-0x22ffff at device 4.1 on pci3 bge1: attempting to allocate 1 MSI vectors (8 supported) bge1: using IRQ 1 for MSI bge1: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: OUI 0x001018, model 0x0034, rev. 0 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow random: harvesting attach, 8 bytes (4 bits) from brgphy1 random: harvesting attach, 8 bytes (4 bits) from miibus1 bge1: bpf attached bge1: Ethernet address: 00:14:4f:2b:b1:85 random: harvesting attach, 8 bytes (4 bits) from bge1 mpt0: mem 0x800000-0x803fff,0x810000-0x81ffff at device 0.0 on pci5 pcib1: attempting to grow I/O port window for (0-0xffffffff,0x1000) front candidate range: 0-0xfff back candidate range: 0x10000-0x10fff pcib1: grew I/O port window to 0-0xffff pcib1: allocated I/O port range (0-0xfff) for rid 1c of pcib5 pcib5: allocated initial I/O port window of 0-0xfff pcib5: allocated I/O port range (0-0xff) for rid 10 of mpt0 mpt0: Lazy allocation of 0x100 bytes rid 0x10 type 4 at 0 mpt0: attempting to allocate 1 MSI-X vectors (1 supported) mpt0: using IRQ 255 for MSI-X mpt0: MPI Version=1.5.13.0 mpt0: chain depth limited to 96 (from 2040) mpt0: Maximum Segment Count: 864, Maximum CAM Segment Count: 17 mpt0: MsgLength=20 IOCNumber = 0 mpt0: IOCFACTS: GlobalCredits=511 BlockSize=8 bytes Request Frame Size 128 bytes Max Chain Depth 96 mpt0: IOCFACTS: Num Ports 1, FWImageSize 0, Flags=0 mpt0: No Handlers For Any Event Notify Frames. Event 0xa (ACK not required). mpt0: No Handlers For Any Event Notify Frames. Event 0x16 (ACK not required). mpt0: No Handlers For Any Event Notify Frames. Event 0x12 (ACK not required). mpt0: No Handlers For Any Event Notify Frames. Event 0xf (ACK required). mpt0: No Handlers For Any Event Notify Frames. Event 0x16 (ACK not required). mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 ) mpt0: 0 Active Volumes (2 Max) mpt0: 0 Hidden Drive Members (10 Max) mpt0: No Handlers For Any Event Notify Frames. Event 0xa (ACK not required). random: harvesting attach, 8 bytes (4 bits) from mpt0 ohci0: mem 0x1000000-0x1000fff at device 28.0 on pci11 usbus0 on ohci0 ohci0: usbpf: Attached random: harvesting attach, 8 bytes (4 bits) from usbus0 random: harvesting attach, 8 bytes (4 bits) from ohci0 ohci1: mem 0x2000000-0x2000fff at device 28.1 on pci11 usbus1 on ohci1 ohci1: usbpf: Attached random: harvesting attach, 8 bytes (4 bits) from usbus1 random: harvesting attach, 8 bytes (4 bits) from ohci1 ohci2: mem 0x3000000-0x3000fff at device 28.2 on pci11 usbus2 on ohci2 ohci2: usbpf: Attached random: harvesting attach, 8 bytes (4 bits) from usbus2 random: harvesting attach, 8 bytes (4 bits) from ohci2 ehci0: mem 0x200000-0x2000ff at device 28.3 on pci11 usbus3: EHCI version 1.0 usbus3 on ehci0 ehci0: usbpf: Attached random: harvesting attach, 8 bytes (4 bits) from usbus3 random: harvesting attach, 8 bytes (4 bits) from ehci0 isab0: at device 30.0 on pci11 isa0: on isab0 random: harvesting attach, 8 bytes (4 bits) from isa0 random: harvesting attach, 8 bytes (4 bits) from isab0 atapci0: port 0x1040-0x1047,0x1080-0x1083,0x10c0-0x10c7,0x1100-0x1103,0x1000-0x100f at device 31.0 on pci11 ata2: at channel 0 on atapci0 random: harvesting attach, 8 bytes (4 bits) from ata2 ata3: at channel 1 on atapci0 random: harvesting attach, 8 bytes (4 bits) from ata3 random: harvesting attach, 8 bytes (4 bits) from atapci0 bge2: mem 0x4200000-0x420ffff,0x4010000-0x401ffff at device 4.0 on pci13 bge2: attempting to allocate 1 MSI vectors (8 supported) bge2: using IRQ 0 for MSI bge2: couldn't map interrupt device_attach: bge2 attach returned 6 bge2: mem 0x4400000-0x440ffff,0x4020000-0x402ffff at device 4.1 on pci13 bge2: attempting to allocate 1 MSI vectors (8 supported) bge2: using IRQ 1 for MSI bge2: couldn't map interrupt device_attach: bge2 attach returned 6 ebus0: addr 0-0x1fffff (no driver attached) ebus0: addr 0xc00008 (no driver attached) ebus0: addr 0xc0a100-0xc0a12f irq 1990 (no driver attached) ebus0: addr 0xc28000-0xc28007 irq 1993 (no driver attached) uart0: <16750 or compatible> addr 0xc2c000-0xc2c007 irq 1991 on ebus0uart0: console (9600,n,8,1) uart0: fast interrupt uart0: PPS capture mode: DCDinvalid random: harvesting attach, 8 bytes (4 bits) from uart0 uart1: <16550 or compatible> addr 0x300fffff8-0x300ffffff irq 1994 on ebus0 uart1: fast interrupt uart1: PPS capture mode: DCDinvalid random: harvesting attach, 8 bytes (4 bits) from uart1 nexus0: mem 0x4000fd20000-0x4000fd2003f irq 1985 type i2c (no driver attached) cryptosoft0: on nexus0 crypto: assign cryptosoft0 driver id 0, flags 100663296 crypto: cryptosoft0 registers alg 1 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 2 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 3 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 4 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 5 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 16 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 6 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 7 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 18 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 19 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 20 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 8 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 15 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 9 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 10 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 13 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 14 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 11 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 22 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 23 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 25 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 24 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 26 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 27 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 28 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 21 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 17 flags 0 maxoplen 0 random: harvesting attach, 8 bytes (4 bits) from cryptosoft0 syscons0: no video adapter found. nexus0: type unknown (no driver attached) rtc0: at port 0x70-0x73 on isa0 pcib10: allocated I/O port range (0x70-0x73) for rid 0 of rtc0 rtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) rtc0: current time: 1525039418.000000000 random: harvesting attach, 8 bytes (4 bits) from rtc0 procfs registered Timecounter "tick" frequency 1592000000 Hz quality 10 Timecounter "stick" frequency 10000000 Hz quality 10 Event timer "tick" frequency 1592000000 Hz quality 1000 Timecounters tick every 1.000 msec lo0: bpf attached vlan: initialized, using hash tables with chaining tcp_init: net.inet.tcp.tcbhashsize auto tuned to 32768 IPsec: Initialized Security Association Processing. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ata2: reset tp1 mask=03 ostat0=50 ostat1=00 uhub0: 3 ports with 3 removable, self powered random: harvesting attach, 8 bytes (4 bits) from uhub0 uhub1: 3 ports with 3 removable, self powered random: harvesting attach, 8 bytes (4 bits) from uhub1 ata2: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb ata2: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 uhub2: 3 ports with 3 removable, self powered random: harvesting attach, 8 bytes (4 bits) from uhub2 ata2: reset tp2 stat0=00 stat1=00 devices=0x10000 ata3: reset tp1 mask=03 ostat0=00 ostat1=00 ata3: stat0=0x00 err=0x00 lsb=0x00 msb=0x00 ata3: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 ata3: reset tp2 stat0=00 stat1=00 devices=0x0 pass0 at mpt0 bus 0 scbus0 target 0 lun 0 GEOM: new disk da0 pass0: Fixed Direct Access SPC-2 SCSI device pass0: Serial Number 000736S0DRHP pass0: 300.000MB/s transfers pass0: Command Queueing enabled pass1 at ata2 bus 0 scbus2 target 0 lun 0 da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SPC-2 SCSI device da0: Serial Number 000736S0DRHP da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 70007MB (143374738 512 byte sectors) da0: Delete methods: pass1: Removable CD-ROM SCSI device pass1: 66.700MB/s transfers (UDMA4, ATAPI 12bytes, PIO 65534bytes) cd0 at ata2 bus 0 scbus2 target 0 lun 0 cd0: Removable CD-ROM SCSI device SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! cd0: 66.700MB/s transfers (UDMA4, ATAPI 12bytes, PIO 65534bytes) cd0: 549MB (281122 2048 byte sectors) GEOM: new disk cd0 Trying to mount root from cd9660:/dev/iso9660/11_1_RELEASE_SPARC64_CD [ro]... GEOM: da0: geometry does not match VTOC8 label (label: 24h,423s GEOM: 255h,63s). GEOM: da0: adding VTOC8 information. uhub3: 8 ports with 8 removable, self powered random: harvesting attach, 8 bytes (4 bits) from uhub3 ugen3.2: at usbus3 uhub4 on uhub3 uhub4: on usbus3 uhub4: MTT enabled (cd0:ata2:0:0:0): READ(10). CDB: 28 00 00 04 4a 21 00 00 01 00 (cd0:ata2:0:0:0): CAM status: SCSI Status Error (cd0:ata2:0:0:0): SCSI status: Check Condition (cd0:ata2:0:0:0): SCSI sense: ILLEGAL REQUEST asc:64,0 (Illegal mode for this track) (cd0:ata2:0:0:0): Info: 0x44a21 (cd0:ata2:0:0:0): Command Specific Info: 0x1d0000 (cd0:ata2:0:0:0): Error 6, Unretryable error uhub4: (cd0:4 ports with 4 removable, self powered ata2:0:random: harvesting attach, 8 bytes (4 bits) from uhub4 0:0): cddone: got error 0x6 back (cd0:ata2:0:0:0): READ(10). CDB: 28 00 00 04 4a 21 00 00 01 00 (cd0:ata2:0:0:0): CAM status: SCSI Status Error (cd0:ata2:0:0:0): SCSI status: Check Condition (cd0:ata2:0:0:0): SCSI sense: ILLEGAL REQUEST asc:64,0 (Illegal mode for this track) (cd0:ata2:0:0:0): Info: 0x44a21 (cd0:ata2:0:0:0): Command Specific Info: 0x1d0000 (cd0:ata2:0:0:0): Error 6, Unretryable error (cd0:ata2:0:0:0): cddone: got error 0x6 back GEOM: diskid/DISK-000736S0DRHP: geometry does not match VTOC8 label (label: 24h,423s GEOM: 255h,63s). GEOM: diskid/DISK-000736S0DRHP: adding VTOC8 information. (cd0:ata2:0:0:0): READ(10). CDB: 28 00 00 04 4a 21 00 00 01 00 (cd0:ata2:0:0:0): CAM status: SCSI Status Error (cd0:ata2:0:0:0): SCSI status: Check Condition (cd0:ata2:0:0:0): SCSI sense: ILLEGAL REQUEST asc:64,0 (Illegal mode for this track) (cd0:ata2:0:0:0): Info: 0x44a21 (cd0:ata2:0:0:0): Command Specific Info: 0x1d0000 (cd0:ata2:0:0:0): Error 6, Unretryable error (cd0:ata2:0:0:0): cddone: got error 0x6 back random: unblocking device. cd9660: RockRidge Extension start_init: trying /sbin/init Starting file sy === --- Yoshihiko Iwama From owner-freebsd-sparc64@freebsd.org Fri May 4 09:15:39 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54D06FA6013 for ; Fri, 4 May 2018 09:15:39 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from mail02.SiriusCloud.jp (mail02.SiriusCloud.jp [219.118.72.2]) by mx1.freebsd.org (Postfix) with ESMTP id AE41E83E35 for ; Fri, 4 May 2018 09:15:38 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from W701ds (124x37x209x41.ap124.ftth.ucom.ne.jp [124.37.209.41]) (Authenticated sender: iwama@t3.rim.or.jp) by access02.SiriusCloud.jp (Postfix) with ESMTPA id 40cmQG2hRKz1XLHBV for ; Fri, 4 May 2018 18:08:22 +0900 (JST) Authentication-Results: access02.SiriusCloud.jp; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral From: To: Subject: Root on ZFS RAIDZ1 Date: Fri, 4 May 2018 18:08:19 +0900 Message-ID: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdPjfsxbCcuxjMtlSmy2jON28+qtug== Content-Language: ja X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 09:15:39 -0000 I want Sun Fire V245 to use FreeBSD-11.1-RELEASE with Root on ZFS RAIDZ1. The installation procedure of Root on ZFS is as follow page: https://wiki.freebsd.org/RootOnZFS/VTOC8ZFSBoot/9.x-RELEASE I finished the installation without any problem, but it was some problems at reboot. 1) It do not boot at all 2) boot takes a very long time, more than 3 HOURS 3) It takes some time for boot, about 10 minutes I tried RAIDZ1 with 9.3-RELEASE, RAIDZ2, RAIDZ10 and mirror with 11.1-RELEASE, the results were as follows, da0-3 are the name of boot disk. (Fire V245 has four disks): RAIDZ1(zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a da1a da2a da3a) da0 da1 da2 da3 11.1-RELEASE NG NG SLOW SLOW 9.3-RELEASE NG NG SLOW SLOW RAIDZ2(zpool create -o altroot=/mnt -O canmount=off zroot raidz2 da0a da1a da2a da3a) da0 da1 da2 da3 11.1-RELEASE SLOW SLOW SLOW SLOW RAIDZ10(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a da1a mirror da2a da3a) da0 da1 da2 da3 11.1-RELEASE slow slow slow slow mirror(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a da1a da2a da3a) da0 da1 da2 da3 11.1-RELEASE OK OK OK OK OK: OK(boot at normal speed), NG: 1), SLOW: 2), slow: 3) From the above results and the logs below, there is no problem with mirror, but it is a big problem in RAIDZ1 or RAIDZ2. Has anyone been running Root on RAIDZ1 on the sparc64 platform? Install steps(only the part entered manually): === # zpool labelclear -f /dev/da0a # zpool labelclear -f /dev/da1a # zpool labelclear -f /dev/da2a # zpool labelclear -f /dev/da3a # # sysctl kern.disks kern.disks: cd0 da3 da2 da1 da0 # gpart destroy -F da0 da0 destroyed # gpart destroy -F da1 da1 destroyed # gpart destroy -F da2 da2 destroyed # gpart destroy -F da3 da3 destroyed # gpart create -s VTOC8 da0 da0 created # gpart create -s VTOC8 da1 da1 created # gpart create -s VTOC8 da2 da2 created # gpart create -s VTOC8 da3 da3 created # # gpart add -s 270G -t freebsd-zfs da0 da0a added # gpart add -s 270G -t freebsd-zfs da1 da1a added # gpart add -s 270G -t freebsd-zfs da2 da2a added # gpart add -s 270G -t freebsd-zfs da3 da3a added # gpart add -t freebsd-swap da0 da0b added # gpart add -t freebsd-swap da1 da1b added # gpart add -t freebsd-swap da2 da2b added # gpart add -t freebsd-swap da3 da3b added # ***The following lines differ depending on the RAID method # zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a da1a da2a da3a # # zpool export zroot # gpart bootcode -p /boot/zfsboot da0 # gpart bootcode -p /boot/zfsboot da1 # gpart bootcode -p /boot/zfsboot da2 # gpart bootcode -p /boot/zfsboot da3 # sysctl kern.geom.debugflags=0x10 kern.geom.debugflags: 0 -> 16 # dd if=/boot/zfsloader of=/dev/da0a bs=512 oseek=1024 conv=notrunc,sync 466+1 records in 467+0 records out 239104 bytes transferred in 2.845066 secs (84042 bytes/sec) # dd if=/boot/zfsloader of=/dev/da1a bs=512 oseek=1024 conv=notrunc,sync 466+1 records in 467+0 records out 239104 bytes transferred in 2.832603 secs (84411 bytes/sec) # dd if=/boot/zfsloader of=/dev/da2a bs=512 oseek=1024 conv=notrunc,sync 466+1 records in 467+0 records out 239104 bytes transferred in 2.815799 secs (84915 bytes/sec) # dd if=/boot/zfsloader of=/dev/da3a bs=512 oseek=1024 conv=notrunc,sync 466+1 records in 467+0 records out 239104 bytes transferred in 2.828102 secs (84546 bytes/sec) # zpool import -o altroot=/mnt zroot # # zfs set checksum=fletcher4 zroot # zfs set atime=off zroot # # zfs create -o mountpoint=/ zroot/ROOT # # zpool set bootfs=zroot/ROOT zroot # # cat << EOF > /tmp/bsdinstall_etc/fstab > # Device Mountpoint FStype Options Dump Pass# > /dev/da0b none swap sw 0 0 > /dev/da1b none swap sw 0 0 > /dev/da2b none swap sw 0 0 > /dev/da3b none swap sw 0 0 > ^D# # # exit # echo 'zfs_enable="YES"' >> /etc/rc.conf # # cat /boot/loader.conf zfs_load="YES" ***Except for 9.3-RELEASE, since the line of "zfs_load="YES"" is originally in ***/boot/loader.conf, addition is unnecessary # # zpool set cachefile=/boot/zfs/zpool.cache zroot # exit ***In 11.1-RELEASE, the last "zpool set cachefile=/boot/zfs/zpool.cache zroot" failed, shown as "internal error: failed to initialize ZFS library" === NOT boot log: === {1} ok boot disk0 SC Alert: Host System has Reset Probing system devices Probing memory Probing I/O buses screen not found. keyboard not found. Keyboard not present. Using ttya for input and output. Probing system devices Probing memory Probing I/O buses Sun Fire V245, No Keyboard Copyright 2006 Sun Microsystems, Inc. All rights reserved. OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. Initializing 1MB of memory at addr 137ff0a000 Initializing 1MB of memory at addr 137ff06000 Initializing 1MB of memory at addr 137ff00000 Initializing 1MB of memory at addr 137fee0000 Initializing 1MB of memory at addr 137fe90000 Initializing 1MB of memory at addr 137fdea000 Initializing 14MB of memory at addr 137f000000 Initializing 2032MB of memory at addr 1300000000 Initializing 2048MB of memory at addr 1200000000 Initializing 2048MB of memory at addr 1100000000 Initializing 2048MB of memory at addr 1000000000 Initializing 2048MB of memory at addr 300000000 Initializing 2048MB of memory at addr 200000000 Initializing 2048MB of memory at addr 100000000 Initializing 2048MB of memory at addr 0 Rebooting with command: boot disk0 Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 File and args: >> FreeBSD/sparc64 ZFS boot block Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a Consoles: Open Firmware console ZFS: i/o error - all block copies unavailable ZFS: can't read MOS object directory ZFS: can't find root filesystem FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) bootpath="" can't load 'kernel' Type '?' for a list of commands, 'help' for more detailed help. OK reboot Rebooting... Program terminated {1} ok === VERY SLOW boot log: === {1} ok boot disk3 SC Alert: Host System has Reset Probing system devices Probing memory Probing I/O buses screen not found. keyboard not found. Keyboard not present. Using ttya for input and output. Probing system devices Probing memory Probing I/O buses Sun Fire V245, No Keyboard Copyright 2006 Sun Microsystems, Inc. All rights reserved. OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. Initializing 1MB of memory at addr 137ff0a000 Initializing 1MB of memory at addr 137ff06000 Initializing 1MB of memory at addr 137ff00000 Initializing 1MB of memory at addr 137fee0000 Initializing 1MB of memory at addr 137fe90000 Initializing 1MB of memory at addr 137fdea000 Initializing 14MB of memory at addr 137f000000 Initializing 2032MB of memory at addr 1300000000 Initializing 2048MB of memory at addr 1200000000 Initializing 2048MB of memory at addr 1100000000 Initializing 2048MB of memory at addr 1000000000 Initializing 2048MB of memory at addr 300000000 Initializing 2048MB of memory at addr 200000000 Initializing 2048MB of memory at addr 100000000 Initializing 2048MB of memory at addr 0 Rebooting with command: boot disk3 Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0 File and args: >> FreeBSD/sparc64 ZFS boot block Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0:a Consoles: Open Firmware console ***Slow from here(\|/-\|/- animation)*** FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) bootpath="zfs:zroot/ROOT:" Loading /boot/defaults/loader.conf /boot/kernel/kernel data=0xc47740+0xf4078 syms=[0x8+0xd5a38+0x8+0xc746a] /boot/entropy size=0x1000 /boot/kernel/zfs.ko text=0x25cc60 data=0xb9d0+0x90050 |syms=[0x8+0x1bd38/-+0x8+0x16db2] loading required module 'opensolaris' /boot/kernel/opensolaris.ko text=0x3730 data=0x2c8+0x1a30 syms=[0x8+0xe10+0x8+0x968] ***It is slow until the above is displayed.*** Hit [Enter] to boot immediately, or any other key for command prompt. Booting [/boot/kernel/kernel] in 9 seconds... Booting [/boot/kernel/kernel] in 8 seconds... Booting [/boot/kernel/kernel] in 7 seconds... Booting [/boot/kernel/kernel] in 6 seconds... Booting [/boot/kernel/kernel] in 5 seconds... Booting [/boot/kernel/kernel] in 4 seconds... Booting [/boot/kernel/kernel] in 3 seconds... Booting [/boot/kernel/kernel] in 2 seconds... Booting [/boot/kernel/kernel] in 1 second... Booting [/boot/kernel/kernel]... -\|/-\|/-\|/-\|/-\|jumping to kernel entry at 0xc00b0000. === --- Yoshihiko Iwama From owner-freebsd-sparc64@freebsd.org Fri May 4 13:19:49 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0F3DFAD71C for ; Fri, 4 May 2018 13:19:49 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [192.108.105.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.soaustin.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 496A77947F for ; Fri, 4 May 2018 13:19:48 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from lonesome.com (bones.soaustin.net [192.108.105.22]) by mail.soaustin.net (Postfix) with ESMTPSA id 14186D20; Fri, 4 May 2018 08:19:42 -0500 (CDT) Date: Fri, 4 May 2018 08:19:41 -0500 From: Mark Linimon To: iwama@t3.rim.or.jp Cc: freebsd-sparc64@freebsd.org Subject: Re: Fire V445 support status Message-ID: <20180504131940.GA12497@lonesome.com> References: <000001d3e37e$2c1d52a0$8457f7e0$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000001d3e37e$2c1d52a0$8457f7e0$@t3.rim.or.jp> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 13:19:49 -0000 All that I can offer is that I have run 10.3 and 11.0 on V215s and V245s. I had assumed that they were similar. Although a few developers do work on sparc64, unfortunately there is little support for it in general in the FreeBSD developer community. I used to do package builds but have not during this year so far. I am afraid we are in a shrinking minority. mcl From owner-freebsd-sparc64@freebsd.org Fri May 4 13:30:39 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC8EAFADE11 for ; Fri, 4 May 2018 13:30:38 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8516E79F12 for ; Fri, 4 May 2018 13:30:38 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id w44DUb82001782; Fri, 4 May 2018 09:30:37 -0400 (EDT) (envelope-from lidl@pix.net) Subject: Re: Root on ZFS RAIDZ1 To: freebsd-sparc64@freebsd.org References: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> From: Kurt Lidl Message-ID: Date: Fri, 4 May 2018 09:30:37 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 13:30:39 -0000 I have been using various sparc64 machines with a ZFS only installation for quite a while, and found that it works reliably for me. My earlier machines were Netra-T1 (all retired now), and currently I have both a V240 and three V120 machines using this configuration. Here are my notes that I use during the partitioning phase of the install. (ie, start a normal install as discussed in the web page you mention, and during the 'Shell' method of partitioning, I run these commands): You may or may not want to execute the commented out sysctl, which will force the ZFS installation to use 4096 byte blocks. Given that all the disks I've ever used on sparc64 are really 512byte block devices, I'm not sure if this is a great idea or not. I typically do run with 4096 byte blocks. ----------------------------------------------------------------- kldload zfs # force 4K minimum zfs block size #sysctl vfs.zfs.min_auto_ashift=12 # appropriate for a 36GB disk w/4+GB of swap zfssize=30000M # appropriate for a 72GB disk w/4+GB of swap zfssize=65500M # appropriate for a 72GB disk w/8+GB of swap zfssize=61500M disks="da0 da1" for d in $disks do gpart destroy -F $d gpart create -s VTOC8 $d gpart add -t freebsd-zfs -s $zfssize $d gpart add -t freebsd-swap $d gpart bootcode -p /boot/zfsboot $d dd if=/boot/zfsloader of=/dev/${d}a bs=512 oseek=1024 \ conv=notrunc,sync done # note, we use -R /mnt to set a different root mountpoint P=zroot mnt=/mnt zpool create -f -o cachefile=none -O checksum=fletcher4 \ -R $mnt -m none $P mirror /dev/da0a /dev/da1a zfs create $P/ROOT zfs create $P/ROOT/default zpool set bootfs=$P/ROOT/default $P # because of "-R /mnt" earlier, this is "/mnt" effectively for this session zfs set mountpoint=/ $P/ROOT/default zfs create $P/vartmp zfs create $P/usrports zfs create $P/usrsrc zfs create $P/usrobj zfs create $P/home zfs set mountpoint=/var/tmp $P/vartmp zfs set mountpoint=/home $P/home zfs set mountpoint=/usr/ports $P/usrports zfs set mountpoint=/usr/src $P/usrsrc zfs set mountpoint=/usr/obj $P/usrobj zfs mount -a gmirror label -b prefer gswap0 da0b da1b eeprom boot-device="disk0 disk1" exit ----------------------------------------------------------------- At the end of the installation process, you will need to fix up the /etc/rc.conf and /boot/loader.conf too: # Add to /etc/rc.conf: clear_tmp_enable="YES" ntpdate_enable="YES" rwhod_enable="YES" rwhod_flags="-m" zfs_enable="YES" # Add to /boot/loader.conf: zfs_load="YES" geom_mirror_load="YES" vfs.zfs.prefetch_disable=1 # Add to /etc/fstab: # Device Mountpoint FStype Options Dmp Pass# /dev/mirror/gswap none swap sw 0 0 #md /tmp mfs rw,-s64m 0 0 tmpfs /tmp tmpfs rw,size=64m,mode=1777 0 0 # turn off ttyv? entries in /etc/ttys # turn off ttyu2 entry in /etc/ttys I hope this helps you get the system working well. Best wishes, -Kurt On 5/4/18 5:08 AM, iwama@t3.rim.or.jp wrote: > I want Sun Fire V245 to use FreeBSD-11.1-RELEASE with Root on ZFS RAIDZ1. > The installation procedure of Root on ZFS is as follow page: > > https://wiki.freebsd.org/RootOnZFS/VTOC8ZFSBoot/9.x-RELEASE > > I finished the installation without any problem, but it was some problems > at reboot. > > 1) It do not boot at all > 2) boot takes a very long time, more than 3 HOURS > 3) It takes some time for boot, about 10 minutes > > I tried RAIDZ1 with 9.3-RELEASE, RAIDZ2, RAIDZ10 and mirror with > 11.1-RELEASE, > the results were as follows, da0-3 are the name of boot disk. (Fire V245 has > four disks): > > RAIDZ1(zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a da1a > da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE NG NG SLOW SLOW > 9.3-RELEASE NG NG SLOW SLOW > > RAIDZ2(zpool create -o altroot=/mnt -O canmount=off zroot raidz2 da0a da1a > da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE SLOW SLOW SLOW SLOW > > RAIDZ10(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a da1a > mirror da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE slow slow slow slow > > mirror(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a da1a > da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE OK OK OK OK > > OK: OK(boot at normal speed), NG: 1), SLOW: 2), slow: 3) > > > From the above results and the logs below, there is no problem with > mirror, but it is a big > problem in RAIDZ1 or RAIDZ2. > Has anyone been running Root on RAIDZ1 on the sparc64 platform? > > > Install steps(only the part entered manually): > === > # zpool labelclear -f /dev/da0a > # zpool labelclear -f /dev/da1a > # zpool labelclear -f /dev/da2a > # zpool labelclear -f /dev/da3a > # > # sysctl kern.disks > kern.disks: cd0 da3 da2 da1 da0 > # gpart destroy -F da0 > da0 destroyed > # gpart destroy -F da1 > da1 destroyed > # gpart destroy -F da2 > da2 destroyed > # gpart destroy -F da3 > da3 destroyed > # gpart create -s VTOC8 da0 > da0 created > # gpart create -s VTOC8 da1 > da1 created > # gpart create -s VTOC8 da2 > da2 created > # gpart create -s VTOC8 da3 > da3 created > # > # gpart add -s 270G -t freebsd-zfs da0 > da0a added > # gpart add -s 270G -t freebsd-zfs da1 > da1a added > # gpart add -s 270G -t freebsd-zfs da2 > da2a added > # gpart add -s 270G -t freebsd-zfs da3 > da3a added > # gpart add -t freebsd-swap da0 > da0b added > # gpart add -t freebsd-swap da1 > da1b added > # gpart add -t freebsd-swap da2 > da2b added > # gpart add -t freebsd-swap da3 > da3b added > # > ***The following lines differ depending on the RAID method > # zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a da1a da2a > da3a > # > # zpool export zroot > # gpart bootcode -p /boot/zfsboot da0 > # gpart bootcode -p /boot/zfsboot da1 > # gpart bootcode -p /boot/zfsboot da2 > # gpart bootcode -p /boot/zfsboot da3 > # sysctl kern.geom.debugflags=0x10 > kern.geom.debugflags: 0 -> 16 > # dd if=/boot/zfsloader of=/dev/da0a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.845066 secs (84042 bytes/sec) > # dd if=/boot/zfsloader of=/dev/da1a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.832603 secs (84411 bytes/sec) > # dd if=/boot/zfsloader of=/dev/da2a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.815799 secs (84915 bytes/sec) > # dd if=/boot/zfsloader of=/dev/da3a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.828102 secs (84546 bytes/sec) > # zpool import -o altroot=/mnt zroot > # > # zfs set checksum=fletcher4 zroot > # zfs set atime=off zroot > # > # zfs create -o mountpoint=/ zroot/ROOT > > # > # zpool set bootfs=zroot/ROOT zroot > # > # cat << EOF > /tmp/bsdinstall_etc/fstab >> # Device Mountpoint FStype Options Dump Pass# >> /dev/da0b none swap sw 0 0 >> /dev/da1b none swap sw 0 0 >> /dev/da2b none swap sw 0 0 >> /dev/da3b none swap sw 0 0 >> ^D# > # > # exit > > > # echo 'zfs_enable="YES"' >> /etc/rc.conf > # > # cat /boot/loader.conf > zfs_load="YES" > ***Except for 9.3-RELEASE, since the line of "zfs_load="YES"" is originally > in > ***/boot/loader.conf, addition is unnecessary > # > # zpool set cachefile=/boot/zfs/zpool.cache zroot > # exit > > ***In 11.1-RELEASE, the last "zpool set cachefile=/boot/zfs/zpool.cache > zroot" failed, > shown as "internal error: failed to initialize ZFS library" > === > > > NOT boot log: > === > {1} ok boot disk0 > > SC Alert: Host System has Reset > Probing system devices > Probing memory > Probing I/O buses > screen not found. > keyboard not found. > Keyboard not present. Using ttya for input and output. > Probing system devices > Probing memory > Probing I/O buses > > > Sun Fire V245, No Keyboard > Copyright 2006 Sun Microsystems, Inc. All rights reserved. > OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. > Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. > > > > Initializing 1MB of memory at addr 137ff0a000 > Initializing 1MB of memory at addr 137ff06000 > Initializing 1MB of memory at addr 137ff00000 > Initializing 1MB of memory at addr 137fee0000 > Initializing 1MB of memory at addr 137fe90000 > Initializing 1MB of memory at addr 137fdea000 > Initializing 14MB of memory at addr 137f000000 > Initializing 2032MB of memory at addr 1300000000 > Initializing 2048MB of memory at addr 1200000000 > Initializing 2048MB of memory at addr 1100000000 > Initializing 2048MB of memory at addr 1000000000 > Initializing 2048MB of memory at addr 300000000 > Initializing 2048MB of memory at addr 200000000 > Initializing 2048MB of memory at addr 100000000 > Initializing 2048MB of memory at addr 0 > Rebooting with command: boot disk0 > Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 File > and args: > >>> FreeBSD/sparc64 ZFS boot block > Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a > Consoles: Open Firmware console > ZFS: i/o error - all block copies unavailable > ZFS: can't read MOS object directory > ZFS: can't find root filesystem > > FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 > (Fri Jul 21 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) > bootpath="" > > can't load 'kernel' > > Type '?' for a list of commands, 'help' for more detailed help. > OK reboot > Rebooting... > Program terminated > {1} ok > === > > > VERY SLOW boot log: > === > {1} ok boot disk3 > > SC Alert: Host System has Reset > Probing system devices > Probing memory > Probing I/O buses > screen not found. > keyboard not found. > Keyboard not present. Using ttya for input and output. > Probing system devices > Probing memory > Probing I/O buses > > > Sun Fire V245, No Keyboard > Copyright 2006 Sun Microsystems, Inc. All rights reserved. > OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. > Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. > > > > Initializing 1MB of memory at addr 137ff0a000 > Initializing 1MB of memory at addr 137ff06000 > Initializing 1MB of memory at addr 137ff00000 > Initializing 1MB of memory at addr 137fee0000 > Initializing 1MB of memory at addr 137fe90000 > Initializing 1MB of memory at addr 137fdea000 > Initializing 14MB of memory at addr 137f000000 > Initializing 2032MB of memory at addr 1300000000 > Initializing 2048MB of memory at addr 1200000000 > Initializing 2048MB of memory at addr 1100000000 > Initializing 2048MB of memory at addr 1000000000 > Initializing 2048MB of memory at addr 300000000 > Initializing 2048MB of memory at addr 200000000 > Initializing 2048MB of memory at addr 100000000 > Initializing 2048MB of memory at addr 0 > Rebooting with command: boot disk3 > Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0 File > and args: > >>> FreeBSD/sparc64 ZFS boot block > Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0:a > Consoles: Open Firmware console > > > ***Slow from here(\|/-\|/- animation)*** > FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 > (Fri Jul 21 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) > bootpath="zfs:zroot/ROOT:" > Loading /boot/defaults/loader.conf > /boot/kernel/kernel data=0xc47740+0xf4078 syms=[0x8+0xd5a38+0x8+0xc746a] > /boot/entropy size=0x1000 > /boot/kernel/zfs.ko text=0x25cc60 data=0xb9d0+0x90050 > |syms=[0x8+0x1bd38/-+0x8+0x16db2] > loading required module 'opensolaris' > /boot/kernel/opensolaris.ko text=0x3730 data=0x2c8+0x1a30 > syms=[0x8+0xe10+0x8+0x968] > ***It is slow until the above is displayed.*** > > > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel] in 9 seconds... Booting [/boot/kernel/kernel] > in 8 seconds... Booting [/boot/kernel/kernel] in 7 seconds... Booting > [/boot/kernel/kernel] in 6 seconds... Booting [/boot/kernel/kernel] in 5 > seconds... Booting [/boot/kernel/kernel] in 4 seconds... Booting > [/boot/kernel/kernel] in 3 seconds... Booting [/boot/kernel/kernel] in 2 > seconds... Booting [/boot/kernel/kernel] in 1 second... Booting > [/boot/kernel/kernel]... > -\|/-\|/-\|/-\|/-\|jumping to kernel entry at 0xc00b0000. > === > > --- > Yoshihiko Iwama > > > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" > From owner-freebsd-sparc64@freebsd.org Fri May 4 20:59:45 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D356FB7DBA for ; Fri, 4 May 2018 20:59:45 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from mail02.SiriusCloud.jp (mail02.SiriusCloud.jp [219.118.72.2]) by mx1.freebsd.org (Postfix) with ESMTP id E4CAA85F59 for ; Fri, 4 May 2018 20:59:44 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from W701ds (124x37x209x41.ap124.ftth.ucom.ne.jp [124.37.209.41]) (Authenticated sender: iwama@t3.rim.or.jp) by access02.SiriusCloud.jp (Postfix) with ESMTPA id 40d4C16Q5Vz1XLkrY; Sat, 5 May 2018 05:59:41 +0900 (JST) Authentication-Results: access02.SiriusCloud.jp; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral From: To: "'Mark Linimon'" Cc: References: <000001d3e37e$2c1d52a0$8457f7e0$@t3.rim.or.jp> <20180504131940.GA12497@lonesome.com> In-Reply-To: <20180504131940.GA12497@lonesome.com> Subject: RE: Fire V445 support status Date: Sat, 5 May 2018 05:59:41 +0900 Message-ID: <000001d3e3ea$d2ca92e0$785fb8a0$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQHrsIC50tBYgj99Jbex75vLCfmMJwHWzEhfo+G5z0A= Content-Language: ja X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 20:59:45 -0000 Hi mcl, Thank you for your reply. Actually, I also thought that V445 would work without problems as FreeBSD supports V125 and V245 as well as you. I have other choices but I believe it is best to use ZFS on FreeBSD to make effective use of the eight disks of V445. This problem is more troublesome as I think it is much more difficult than when I did send PR about Ultra 30(218478). I will try a little more hints for solution. --- Yoshihiko Iwama -----Original Message----- From: Mark Linimon Sent: Friday, May 4, 2018 10:20 PM To: iwama@t3.rim.or.jp Cc: freebsd-sparc64@freebsd.org Subject: Re: Fire V445 support status All that I can offer is that I have run 10.3 and 11.0 on V215s and V245s. I had assumed that they were similar. Although a few developers do work on sparc64, unfortunately there is little support for it in general in the FreeBSD developer community. I used to do package builds but have not during this year so far. I am afraid we are in a shrinking minority. mcl From owner-freebsd-sparc64@freebsd.org Fri May 4 21:09:28 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C90AFB80E3 for ; Fri, 4 May 2018 21:09:28 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from mail02.SiriusCloud.jp (mail02.SiriusCloud.jp [219.118.72.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8F4966A568 for ; Fri, 4 May 2018 21:09:26 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from W701ds (124x37x209x41.ap124.ftth.ucom.ne.jp [124.37.209.41]) (Authenticated sender: iwama@t3.rim.or.jp) by access02.SiriusCloud.jp (Postfix) with ESMTPA id 40d4QF1cWRz1XLmwX; Sat, 5 May 2018 06:09:25 +0900 (JST) Authentication-Results: access02.SiriusCloud.jp; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral From: To: "'Kurt Lidl'" , References: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> In-Reply-To: Subject: RE: Root on ZFS RAIDZ1 Date: Sat, 5 May 2018 06:09:23 +0900 Message-ID: <000101d3e3ec$2e98cff0$8bca6fd0$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQKfNQR6SVDgjQyf0qsvUAjYH8cwqwHv48o7onnsnjA= Content-Language: ja X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 21:09:28 -0000 Hi Kurt, Thank you for your reply. Thank you for providing your achievements and procedures. I will try it in your procedure. Please let me ask a question. In your procedure, the RAID method is mirror, but is there a record of RAIDZ of 4 disks with V240? Regards, --- Yoshihiko Iwama -----Original Message----- From: owner-freebsd-sparc64@freebsd.org On Behalf Of Kurt Lidl Sent: Friday, May 4, 2018 10:31 PM To: freebsd-sparc64@freebsd.org Subject: Re: Root on ZFS RAIDZ1 I have been using various sparc64 machines with a ZFS only installation for quite a while, and found that it works reliably for me. My earlier machines were Netra-T1 (all retired now), and currently I have both a V240 and three V120 machines using this configuration. Here are my notes that I use during the partitioning phase of the install. (ie, start a normal install as discussed in the web page you mention, and during the 'Shell' method of partitioning, I run these commands): You may or may not want to execute the commented out sysctl, which will force the ZFS installation to use 4096 byte blocks. Given that all the disks I've ever used on sparc64 are really 512byte block devices, I'm not sure if this is a great idea or not. I typically do run with 4096 byte blocks. ----------------------------------------------------------------- kldload zfs # force 4K minimum zfs block size #sysctl vfs.zfs.min_auto_ashift=12 # appropriate for a 36GB disk w/4+GB of swap zfssize=30000M # appropriate for a 72GB disk w/4+GB of swap zfssize=65500M # appropriate for a 72GB disk w/8+GB of swap zfssize=61500M disks="da0 da1" for d in $disks do gpart destroy -F $d gpart create -s VTOC8 $d gpart add -t freebsd-zfs -s $zfssize $d gpart add -t freebsd-swap $d gpart bootcode -p /boot/zfsboot $d dd if=/boot/zfsloader of=/dev/${d}a bs=512 oseek=1024 \ conv=notrunc,sync done # note, we use -R /mnt to set a different root mountpoint P=zroot mnt=/mnt zpool create -f -o cachefile=none -O checksum=fletcher4 \ -R $mnt -m none $P mirror /dev/da0a /dev/da1a zfs create $P/ROOT zfs create $P/ROOT/default zpool set bootfs=$P/ROOT/default $P # because of "-R /mnt" earlier, this is "/mnt" effectively for this session zfs set mountpoint=/ $P/ROOT/default zfs create $P/vartmp zfs create $P/usrports zfs create $P/usrsrc zfs create $P/usrobj zfs create $P/home zfs set mountpoint=/var/tmp $P/vartmp zfs set mountpoint=/home $P/home zfs set mountpoint=/usr/ports $P/usrports zfs set mountpoint=/usr/src $P/usrsrc zfs set mountpoint=/usr/obj $P/usrobj zfs mount -a gmirror label -b prefer gswap0 da0b da1b eeprom boot-device="disk0 disk1" exit ----------------------------------------------------------------- At the end of the installation process, you will need to fix up the /etc/rc.conf and /boot/loader.conf too: # Add to /etc/rc.conf: clear_tmp_enable="YES" ntpdate_enable="YES" rwhod_enable="YES" rwhod_flags="-m" zfs_enable="YES" # Add to /boot/loader.conf: zfs_load="YES" geom_mirror_load="YES" vfs.zfs.prefetch_disable=1 # Add to /etc/fstab: # Device Mountpoint FStype Options Dmp Pass# /dev/mirror/gswap none swap sw 0 0 #md /tmp mfs rw,-s64m 0 0 tmpfs /tmp tmpfs rw,size=64m,mode=1777 0 0 # turn off ttyv? entries in /etc/ttys # turn off ttyu2 entry in /etc/ttys I hope this helps you get the system working well. Best wishes, -Kurt On 5/4/18 5:08 AM, iwama@t3.rim.or.jp wrote: > I want Sun Fire V245 to use FreeBSD-11.1-RELEASE with Root on ZFS RAIDZ1. > The installation procedure of Root on ZFS is as follow page: > > https://wiki.freebsd.org/RootOnZFS/VTOC8ZFSBoot/9.x-RELEASE > > I finished the installation without any problem, but it was some > problems at reboot. > > 1) It do not boot at all > 2) boot takes a very long time, more than 3 HOURS > 3) It takes some time for boot, about 10 minutes > > I tried RAIDZ1 with 9.3-RELEASE, RAIDZ2, RAIDZ10 and mirror with > 11.1-RELEASE, the results were as follows, da0-3 are the name of boot > disk. (Fire V245 has four disks): > > RAIDZ1(zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a > da1a da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE NG NG SLOW SLOW > 9.3-RELEASE NG NG SLOW SLOW > > RAIDZ2(zpool create -o altroot=/mnt -O canmount=off zroot raidz2 da0a > da1a da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE SLOW SLOW SLOW SLOW > > RAIDZ10(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a > da1a mirror da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE slow slow slow slow > > mirror(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a > da1a da2a da3a) > da0 da1 da2 da3 > 11.1-RELEASE OK OK OK OK > > OK: OK(boot at normal speed), NG: 1), SLOW: 2), slow: 3) > > > From the above results and the logs below, there is no problem with > mirror, but it is a big problem in RAIDZ1 or RAIDZ2. > Has anyone been running Root on RAIDZ1 on the sparc64 platform? > > > Install steps(only the part entered manually): > === > # zpool labelclear -f /dev/da0a > # zpool labelclear -f /dev/da1a > # zpool labelclear -f /dev/da2a > # zpool labelclear -f /dev/da3a > # > # sysctl kern.disks > kern.disks: cd0 da3 da2 da1 da0 > # gpart destroy -F da0 > da0 destroyed > # gpart destroy -F da1 > da1 destroyed > # gpart destroy -F da2 > da2 destroyed > # gpart destroy -F da3 > da3 destroyed > # gpart create -s VTOC8 da0 > da0 created > # gpart create -s VTOC8 da1 > da1 created > # gpart create -s VTOC8 da2 > da2 created > # gpart create -s VTOC8 da3 > da3 created > # > # gpart add -s 270G -t freebsd-zfs da0 da0a added # gpart add -s 270G > -t freebsd-zfs da1 da1a added # gpart add -s 270G -t freebsd-zfs da2 > da2a added # gpart add -s 270G -t freebsd-zfs da3 da3a added # gpart > add -t freebsd-swap da0 da0b added # gpart add -t freebsd-swap da1 > da1b added # gpart add -t freebsd-swap da2 da2b added # gpart add -t > freebsd-swap da3 da3b added # ***The following lines differ depending > on the RAID method # zpool create -o altroot=/mnt -O canmount=off > zroot raidz1 da0a da1a da2a da3a # # zpool export zroot # gpart > bootcode -p /boot/zfsboot da0 # gpart bootcode -p /boot/zfsboot da1 # > gpart bootcode -p /boot/zfsboot da2 # gpart bootcode -p /boot/zfsboot > da3 # sysctl kern.geom.debugflags=0x10 > kern.geom.debugflags: 0 -> 16 > # dd if=/boot/zfsloader of=/dev/da0a bs=512 oseek=1024 > conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.845066 secs (84042 bytes/sec) # dd > if=/boot/zfsloader of=/dev/da1a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.832603 secs (84411 bytes/sec) # dd > if=/boot/zfsloader of=/dev/da2a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.815799 secs (84915 bytes/sec) # dd > if=/boot/zfsloader of=/dev/da3a bs=512 oseek=1024 conv=notrunc,sync > 466+1 records in > 467+0 records out > 239104 bytes transferred in 2.828102 secs (84546 bytes/sec) # zpool > import -o altroot=/mnt zroot # > # zfs set checksum=fletcher4 zroot > # zfs set atime=off zroot > # > # zfs create -o mountpoint=/ zroot/ROOT > > # > # zpool set bootfs=zroot/ROOT zroot > # > # cat << EOF > /tmp/bsdinstall_etc/fstab >> # Device Mountpoint FStype Options Dump Pass# >> /dev/da0b none swap sw 0 0 >> /dev/da1b none swap sw 0 0 >> /dev/da2b none swap sw 0 0 >> /dev/da3b none swap sw 0 0 >> ^D# > # > # exit > > > # echo 'zfs_enable="YES"' >> /etc/rc.conf # # cat /boot/loader.conf > zfs_load="YES" > ***Except for 9.3-RELEASE, since the line of "zfs_load="YES"" is > originally in ***/boot/loader.conf, addition is unnecessary # > # zpool set cachefile=/boot/zfs/zpool.cache zroot > # exit > > ***In 11.1-RELEASE, the last "zpool set > cachefile=/boot/zfs/zpool.cache zroot" failed, > shown as "internal error: failed to initialize ZFS library" > === > > > NOT boot log: > === > {1} ok boot disk0 > > SC Alert: Host System has Reset > Probing system devices > Probing memory > Probing I/O buses > screen not found. > keyboard not found. > Keyboard not present. Using ttya for input and output. > Probing system devices > Probing memory > Probing I/O buses > > > Sun Fire V245, No Keyboard > Copyright 2006 Sun Microsystems, Inc. All rights reserved. > OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. > Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. > > > > Initializing 1MB of memory at addr 137ff0a000 > Initializing 1MB of memory at addr 137ff06000 > Initializing 1MB of memory at addr 137ff00000 > Initializing 1MB of memory at addr 137fee0000 > Initializing 1MB of memory at addr 137fe90000 > Initializing 1MB of memory at addr 137fdea000 > Initializing 14MB of memory at addr 137f000000 > Initializing 2032MB of memory at addr 1300000000 > Initializing 2048MB of memory at addr 1200000000 > Initializing 2048MB of memory at addr 1100000000 > Initializing 2048MB of memory at addr 1000000000 > Initializing 2048MB of memory at addr 300000000 > Initializing 2048MB of memory at addr 200000000 > Initializing 2048MB of memory at addr 100000000 > Initializing 2048MB of memory at addr 0 > Rebooting with command: boot disk0 > Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 > File and args: > >>> FreeBSD/sparc64 ZFS boot block > Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a > Consoles: Open Firmware console > ZFS: i/o error - all block copies unavailable > ZFS: can't read MOS object directory > ZFS: can't find root filesystem > > FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 > 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) bootpath="" > > can't load 'kernel' > > Type '?' for a list of commands, 'help' for more detailed help. > OK reboot > Rebooting... > Program terminated > {1} ok > === > > > VERY SLOW boot log: > === > {1} ok boot disk3 > > SC Alert: Host System has Reset > Probing system devices > Probing memory > Probing I/O buses > screen not found. > keyboard not found. > Keyboard not present. Using ttya for input and output. > Probing system devices > Probing memory > Probing I/O buses > > > Sun Fire V245, No Keyboard > Copyright 2006 Sun Microsystems, Inc. All rights reserved. > OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. > Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. > > > > Initializing 1MB of memory at addr 137ff0a000 > Initializing 1MB of memory at addr 137ff06000 > Initializing 1MB of memory at addr 137ff00000 > Initializing 1MB of memory at addr 137fee0000 > Initializing 1MB of memory at addr 137fe90000 > Initializing 1MB of memory at addr 137fdea000 > Initializing 14MB of memory at addr 137f000000 > Initializing 2032MB of memory at addr 1300000000 > Initializing 2048MB of memory at addr 1200000000 > Initializing 2048MB of memory at addr 1100000000 > Initializing 2048MB of memory at addr 1000000000 > Initializing 2048MB of memory at addr 300000000 > Initializing 2048MB of memory at addr 200000000 > Initializing 2048MB of memory at addr 100000000 > Initializing 2048MB of memory at addr 0 > Rebooting with command: boot disk3 > Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0 > File and args: > >>> FreeBSD/sparc64 ZFS boot block > Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0:a > Consoles: Open Firmware console > > > ***Slow from here(\|/-\|/- animation)*** > FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 > 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) > bootpath="zfs:zroot/ROOT:" > Loading /boot/defaults/loader.conf > /boot/kernel/kernel data=0xc47740+0xf4078 > syms=[0x8+0xd5a38+0x8+0xc746a] /boot/entropy size=0x1000 > /boot/kernel/zfs.ko text=0x25cc60 data=0xb9d0+0x90050 > |syms=[0x8+0x1bd38/-+0x8+0x16db2] > loading required module 'opensolaris' > /boot/kernel/opensolaris.ko text=0x3730 data=0x2c8+0x1a30 > syms=[0x8+0xe10+0x8+0x968] ***It is slow until the above is > displayed.*** > > > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel] in 9 seconds... Booting > [/boot/kernel/kernel] in 8 seconds... Booting [/boot/kernel/kernel] in > 7 seconds... Booting [/boot/kernel/kernel] in 6 seconds... Booting > [/boot/kernel/kernel] in 5 seconds... Booting [/boot/kernel/kernel] in > 4 seconds... Booting [/boot/kernel/kernel] in 3 seconds... Booting > [/boot/kernel/kernel] in 2 seconds... Booting [/boot/kernel/kernel] in > 1 second... Booting [/boot/kernel/kernel]... > -\|/-\|/-\|/-\|/-\|jumping to kernel entry at 0xc00b0000. > === > > --- > Yoshihiko Iwama > > > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" > _______________________________________________ freebsd-sparc64@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" From owner-freebsd-sparc64@freebsd.org Fri May 4 21:18:12 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46B90FB8908 for ; Fri, 4 May 2018 21:18:10 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 85DF66B732 for ; Fri, 4 May 2018 21:18:10 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id w44LI9hm009287; Fri, 4 May 2018 17:18:09 -0400 (EDT) (envelope-from lidl@pix.net) Subject: Re: Root on ZFS RAIDZ1 To: iwama@t3.rim.or.jp, freebsd-sparc64@freebsd.org References: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> <000101d3e3ec$2e98cff0$8bca6fd0$@t3.rim.or.jp> From: Kurt Lidl Message-ID: <7b0229eb-1deb-888a-db7c-b973c39251dc@pix.net> Date: Fri, 4 May 2018 17:18:09 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <000101d3e3ec$2e98cff0$8bca6fd0$@t3.rim.or.jp> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 21:18:12 -0000 I have my V240 configured to have one set of mirrored drives on the first two disks, and a second set of mirrored disks on the last two disks. This allows me to have two completely independent OS installs on the machine, and easily switch between them, depending on which disk I boot the system from. I use different ZFS pool names on the different mirrors, so I can, if needed, import the ZFS pool from the other set of disks. Handy for fixing up certain classes of experimental errors on the machine. For a while, I ran with one set of mirrored disks, and just had non-mirrored zpools set up on the other two disks, one mounted as /usr/src and the other mounted as /usr/obj. That experiment didn't show any huge benefit, so I just went to my current setup, with two sets of independent mirrored disks. I never tried a RAIDZ of all four disks on the sparc64. -Kurt On 5/4/18 5:09 PM, iwama@t3.rim.or.jp wrote: > Hi Kurt, > > Thank you for your reply. > > Thank you for providing your achievements and procedures. I will try it in > your procedure. > > Please let me ask a question. > In your procedure, the RAID method is mirror, but is there a record of > RAIDZ of 4 disks > with V240? > > Regards, > --- > Yoshihiko Iwama > > -----Original Message----- > From: owner-freebsd-sparc64@freebsd.org > On Behalf Of Kurt Lidl > Sent: Friday, May 4, 2018 10:31 PM > To: freebsd-sparc64@freebsd.org > Subject: Re: Root on ZFS RAIDZ1 > > I have been using various sparc64 machines with a ZFS only installation for > quite a while, and found that it works reliably for me. My earlier machines > were Netra-T1 (all retired now), and currently I have both a V240 and three > V120 machines using this configuration. > > Here are my notes that I use during the partitioning phase of the install. > (ie, start a normal install as discussed in the web page you mention, and > during the 'Shell' method of partitioning, I run these commands): > > You may or may not want to execute the commented out sysctl, which will > force the ZFS installation to use 4096 byte blocks. > Given that all the disks I've ever used on sparc64 are really 512byte block > devices, I'm not sure if this is a great idea or not. I typically do run > with 4096 byte blocks. > > ----------------------------------------------------------------- > > kldload zfs > > # force 4K minimum zfs block size > #sysctl vfs.zfs.min_auto_ashift=12 > > # appropriate for a 36GB disk w/4+GB of swap zfssize=30000M # appropriate > for a 72GB disk w/4+GB of swap zfssize=65500M # appropriate for a 72GB disk > w/8+GB of swap zfssize=61500M > > disks="da0 da1" > for d in $disks > do > gpart destroy -F $d > gpart create -s VTOC8 $d > gpart add -t freebsd-zfs -s $zfssize $d > gpart add -t freebsd-swap $d > gpart bootcode -p /boot/zfsboot $d > dd if=/boot/zfsloader of=/dev/${d}a bs=512 oseek=1024 \ > conv=notrunc,sync > done > > # note, we use -R /mnt to set a different root mountpoint P=zroot mnt=/mnt > zpool create -f -o cachefile=none -O checksum=fletcher4 \ > -R $mnt -m none $P mirror /dev/da0a /dev/da1a > > zfs create $P/ROOT > zfs create $P/ROOT/default > zpool set bootfs=$P/ROOT/default $P > > # because of "-R /mnt" earlier, this is "/mnt" effectively for this session > zfs set mountpoint=/ $P/ROOT/default > > zfs create $P/vartmp > zfs create $P/usrports > zfs create $P/usrsrc > zfs create $P/usrobj > zfs create $P/home > zfs set mountpoint=/var/tmp $P/vartmp > zfs set mountpoint=/home $P/home > zfs set mountpoint=/usr/ports $P/usrports zfs set mountpoint=/usr/src > $P/usrsrc zfs set mountpoint=/usr/obj $P/usrobj zfs mount -a > > gmirror label -b prefer gswap0 da0b da1b > > eeprom boot-device="disk0 disk1" > > exit > > ----------------------------------------------------------------- > > At the end of the installation process, you will need to fix up the > /etc/rc.conf and /boot/loader.conf too: > > # Add to /etc/rc.conf: > clear_tmp_enable="YES" > ntpdate_enable="YES" > rwhod_enable="YES" > rwhod_flags="-m" > zfs_enable="YES" > > # Add to /boot/loader.conf: > zfs_load="YES" > geom_mirror_load="YES" > vfs.zfs.prefetch_disable=1 > > # Add to /etc/fstab: > # Device Mountpoint FStype Options Dmp Pass# > /dev/mirror/gswap none swap sw 0 0 > #md /tmp mfs rw,-s64m 0 0 > tmpfs /tmp tmpfs rw,size=64m,mode=1777 0 0 > > # turn off ttyv? entries in /etc/ttys > # turn off ttyu2 entry in /etc/ttys > > I hope this helps you get the system working well. > > Best wishes, > > -Kurt > > On 5/4/18 5:08 AM, iwama@t3.rim.or.jp wrote: >> I want Sun Fire V245 to use FreeBSD-11.1-RELEASE with Root on ZFS > RAIDZ1. >> The installation procedure of Root on ZFS is as follow page: >> >> https://wiki.freebsd.org/RootOnZFS/VTOC8ZFSBoot/9.x-RELEASE >> >> I finished the installation without any problem, but it was some >> problems at reboot. >> >> 1) It do not boot at all >> 2) boot takes a very long time, more than 3 HOURS >> 3) It takes some time for boot, about 10 minutes >> >> I tried RAIDZ1 with 9.3-RELEASE, RAIDZ2, RAIDZ10 and mirror with >> 11.1-RELEASE, the results were as follows, da0-3 are the name of boot >> disk. (Fire V245 has four disks): >> >> RAIDZ1(zpool create -o altroot=/mnt -O canmount=off zroot raidz1 da0a >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE NG NG SLOW SLOW >> 9.3-RELEASE NG NG SLOW SLOW >> >> RAIDZ2(zpool create -o altroot=/mnt -O canmount=off zroot raidz2 da0a >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE SLOW SLOW SLOW SLOW >> >> RAIDZ10(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a >> da1a mirror da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE slow slow slow slow >> >> mirror(zpool create -o altroot=/mnt -O canmount=off zroot mirror da0a >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE OK OK OK OK >> >> OK: OK(boot at normal speed), NG: 1), SLOW: 2), slow: 3) >> >> >> From the above results and the logs below, there is no problem with >> mirror, but it is a big problem in RAIDZ1 or RAIDZ2. >> Has anyone been running Root on RAIDZ1 on the sparc64 platform? >> >> >> Install steps(only the part entered manually): >> === >> # zpool labelclear -f /dev/da0a >> # zpool labelclear -f /dev/da1a >> # zpool labelclear -f /dev/da2a >> # zpool labelclear -f /dev/da3a >> # >> # sysctl kern.disks >> kern.disks: cd0 da3 da2 da1 da0 >> # gpart destroy -F da0 >> da0 destroyed >> # gpart destroy -F da1 >> da1 destroyed >> # gpart destroy -F da2 >> da2 destroyed >> # gpart destroy -F da3 >> da3 destroyed >> # gpart create -s VTOC8 da0 >> da0 created >> # gpart create -s VTOC8 da1 >> da1 created >> # gpart create -s VTOC8 da2 >> da2 created >> # gpart create -s VTOC8 da3 >> da3 created >> # >> # gpart add -s 270G -t freebsd-zfs da0 da0a added # gpart add -s 270G >> -t freebsd-zfs da1 da1a added # gpart add -s 270G -t freebsd-zfs da2 >> da2a added # gpart add -s 270G -t freebsd-zfs da3 da3a added # gpart >> add -t freebsd-swap da0 da0b added # gpart add -t freebsd-swap da1 >> da1b added # gpart add -t freebsd-swap da2 da2b added # gpart add -t >> freebsd-swap da3 da3b added # ***The following lines differ depending >> on the RAID method # zpool create -o altroot=/mnt -O canmount=off >> zroot raidz1 da0a da1a da2a da3a # # zpool export zroot # gpart >> bootcode -p /boot/zfsboot da0 # gpart bootcode -p /boot/zfsboot da1 # >> gpart bootcode -p /boot/zfsboot da2 # gpart bootcode -p /boot/zfsboot >> da3 # sysctl kern.geom.debugflags=0x10 >> kern.geom.debugflags: 0 -> 16 >> # dd if=/boot/zfsloader of=/dev/da0a bs=512 oseek=1024 >> conv=notrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.845066 secs (84042 bytes/sec) # dd >> if=/boot/zfsloader of=/dev/da1a bs=512 oseek=1024 conv=notrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.832603 secs (84411 bytes/sec) # dd >> if=/boot/zfsloader of=/dev/da2a bs=512 oseek=1024 conv=notrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.815799 secs (84915 bytes/sec) # dd >> if=/boot/zfsloader of=/dev/da3a bs=512 oseek=1024 conv=notrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.828102 secs (84546 bytes/sec) # zpool >> import -o altroot=/mnt zroot # >> # zfs set checksum=fletcher4 zroot >> # zfs set atime=off zroot >> # >> # zfs create -o mountpoint=/ zroot/ROOT >> >> # >> # zpool set bootfs=zroot/ROOT zroot >> # >> # cat << EOF > /tmp/bsdinstall_etc/fstab >>> # Device Mountpoint FStype Options Dump Pass# >>> /dev/da0b none swap sw 0 0 >>> /dev/da1b none swap sw 0 0 >>> /dev/da2b none swap sw 0 0 >>> /dev/da3b none swap sw 0 0 >>> ^D# >> # >> # exit >> >> >> # echo 'zfs_enable="YES"' >> /etc/rc.conf # # cat /boot/loader.conf >> zfs_load="YES" >> ***Except for 9.3-RELEASE, since the line of "zfs_load="YES"" is >> originally in ***/boot/loader.conf, addition is unnecessary # >> # zpool set cachefile=/boot/zfs/zpool.cache zroot >> # exit >> >> ***In 11.1-RELEASE, the last "zpool set >> cachefile=/boot/zfs/zpool.cache zroot" failed, >> shown as "internal error: failed to initialize ZFS library" >> === >> >> >> NOT boot log: >> === >> {1} ok boot disk0 >> >> SC Alert: Host System has Reset >> Probing system devices >> Probing memory >> Probing I/O buses >> screen not found. >> keyboard not found. >> Keyboard not present. Using ttya for input and output. >> Probing system devices >> Probing memory >> Probing I/O buses >> >> >> Sun Fire V245, No Keyboard >> Copyright 2006 Sun Microsystems, Inc. All rights reserved. >> OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. >> Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. >> >> >> >> Initializing 1MB of memory at addr 137ff0a000 >> Initializing 1MB of memory at addr 137ff06000 >> Initializing 1MB of memory at addr 137ff00000 >> Initializing 1MB of memory at addr 137fee0000 >> Initializing 1MB of memory at addr 137fe90000 >> Initializing 1MB of memory at addr 137fdea000 >> Initializing 14MB of memory at addr 137f000000 >> Initializing 2032MB of memory at addr 1300000000 >> Initializing 2048MB of memory at addr 1200000000 >> Initializing 2048MB of memory at addr 1100000000 >> Initializing 2048MB of memory at addr 1000000000 >> Initializing 2048MB of memory at addr 300000000 >> Initializing 2048MB of memory at addr 200000000 >> Initializing 2048MB of memory at addr 100000000 >> Initializing 2048MB of memory at addr 0 >> Rebooting with command: boot disk0 >> Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 >> File and args: >> >>>> FreeBSD/sparc64 ZFS boot block >> Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a >> Consoles: Open Firmware console >> ZFS: i/o error - all block copies unavailable >> ZFS: can't read MOS object directory >> ZFS: can't find root filesystem >> >> FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 >> 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) bootpath="" >> >> can't load 'kernel' >> >> Type '?' for a list of commands, 'help' for more detailed help. >> OK reboot >> Rebooting... >> Program terminated >> {1} ok >> === >> >> >> VERY SLOW boot log: >> === >> {1} ok boot disk3 >> >> SC Alert: Host System has Reset >> Probing system devices >> Probing memory >> Probing I/O buses >> screen not found. >> keyboard not found. >> Keyboard not present. Using ttya for input and output. >> Probing system devices >> Probing memory >> Probing I/O buses >> >> >> Sun Fire V245, No Keyboard >> Copyright 2006 Sun Microsystems, Inc. All rights reserved. >> OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. >> Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. >> >> >> >> Initializing 1MB of memory at addr 137ff0a000 >> Initializing 1MB of memory at addr 137ff06000 >> Initializing 1MB of memory at addr 137ff00000 >> Initializing 1MB of memory at addr 137fee0000 >> Initializing 1MB of memory at addr 137fe90000 >> Initializing 1MB of memory at addr 137fdea000 >> Initializing 14MB of memory at addr 137f000000 >> Initializing 2032MB of memory at addr 1300000000 >> Initializing 2048MB of memory at addr 1200000000 >> Initializing 2048MB of memory at addr 1100000000 >> Initializing 2048MB of memory at addr 1000000000 >> Initializing 2048MB of memory at addr 300000000 >> Initializing 2048MB of memory at addr 200000000 >> Initializing 2048MB of memory at addr 100000000 >> Initializing 2048MB of memory at addr 0 >> Rebooting with command: boot disk3 >> Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0 >> File and args: >> >>>> FreeBSD/sparc64 ZFS boot block >> Boot path: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0:a >> Consoles: Open Firmware console >> >> >> ***Slow from here(\|/-\|/- animation)*** >> FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul 21 >> 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) >> bootpath="zfs:zroot/ROOT:" >> Loading /boot/defaults/loader.conf >> /boot/kernel/kernel data=0xc47740+0xf4078 >> syms=[0x8+0xd5a38+0x8+0xc746a] /boot/entropy size=0x1000 >> /boot/kernel/zfs.ko text=0x25cc60 data=0xb9d0+0x90050 >> |syms=[0x8+0x1bd38/-+0x8+0x16db2] >> loading required module 'opensolaris' >> /boot/kernel/opensolaris.ko text=0x3730 data=0x2c8+0x1a30 >> syms=[0x8+0xe10+0x8+0x968] ***It is slow until the above is >> displayed.*** >> >> >> Hit [Enter] to boot immediately, or any other key for command prompt. >> Booting [/boot/kernel/kernel] in 9 seconds... Booting >> [/boot/kernel/kernel] in 8 seconds... Booting [/boot/kernel/kernel] in >> 7 seconds... Booting [/boot/kernel/kernel] in 6 seconds... Booting >> [/boot/kernel/kernel] in 5 seconds... Booting [/boot/kernel/kernel] in >> 4 seconds... Booting [/boot/kernel/kernel] in 3 seconds... Booting >> [/boot/kernel/kernel] in 2 seconds... Booting [/boot/kernel/kernel] in >> 1 second... Booting [/boot/kernel/kernel]... >> -\|/-\|/-\|/-\|/-\|jumping to kernel entry at 0xc00b0000. >> === >> >> --- >> Yoshihiko Iwama >> >> >> _______________________________________________ >> freebsd-sparc64@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 >> To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" > From owner-freebsd-sparc64@freebsd.org Sat May 5 17:54:14 2018 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1A9BFB5442 for ; Sat, 5 May 2018 17:54:14 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from mail05.SiriusCloud.jp (mail05.SiriusCloud.jp [219.118.72.5]) by mx1.freebsd.org (Postfix) with ESMTP id 0F52C7F596 for ; Sat, 5 May 2018 17:54:13 +0000 (UTC) (envelope-from iwama@t3.rim.or.jp) Received: from W701ds (124x37x209x41.ap124.ftth.ucom.ne.jp [124.37.209.41]) (Authenticated sender: iwama@t3.rim.or.jp) by access05.SiriusCloud.jp (Postfix) with ESMTPA id 40dc2P5M32z1XLp5q; Sun, 6 May 2018 02:54:05 +0900 (JST) Authentication-Results: access05.SiriusCloud.jp; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral From: To: "'Kurt Lidl'" , References: <000501d3e387$727581b0$57608510$@t3.rim.or.jp> <000101d3e3ec$2e98cff0$8bca6fd0$@t3.rim.or.jp> <7b0229eb-1deb-888a-db7c-b973c39251dc@pix.net> In-Reply-To: <7b0229eb-1deb-888a-db7c-b973c39251dc@pix.net> Subject: RE: Root on ZFS RAIDZ1 Date: Sun, 6 May 2018 02:54:03 +0900 Message-ID: <00b201d3e49a$0e6e9120$2b4bb360$@t3.rim.or.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQKfNQR6SVDgjQyf0qsvUAjYH8cwqwHv48o7ApdkAXsBrTqWpaJYXFrg Content-Language: ja X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 May 2018 17:54:14 -0000 Hi Kurt, I understood your configuration. It is very reasonable. I am satisfied = with=20 the mirror configuration as well. I tried installing again with RAIDZ1 in the procedure you sent to me.=20 Unfortunately, the result was the same as before. That is, as follows: da0 da1 da2 da3 11.1-RELEASE NG NG SLOW SLOW NG: not boot, SLOW: boot at very slow On other, eg amd64 platform, RAIDZ1 will work fine, so is there a = problem=20 with the sparc64 boot code? --- Yoshihiko Iwama -----Original Message----- From: Kurt Lidl =20 Sent: Saturday, May 5, 2018 6:18 AM To: iwama@t3.rim.or.jp; freebsd-sparc64@freebsd.org Subject: Re: Root on ZFS RAIDZ1 I have my V240 configured to have one set of mirrored drives on the = first two disks, and a second set of mirrored disks on the last two = disks. This allows me to have two completely independent OS installs on the = machine, and easily switch between them, depending on which disk I boot = the system from. I use different ZFS pool names on the different = mirrors, so I can, if needed, import the ZFS pool from the other set of = disks. Handy for fixing up certain classes of experimental errors on = the machine. For a while, I ran with one set of mirrored disks, and just had = non-mirrored zpools set up on the other two disks, one mounted as = /usr/src and the other mounted as /usr/obj. That experiment didn't show = any huge benefit, so I just went to my current setup, with two sets of = independent mirrored disks. I never tried a RAIDZ of all four disks on the sparc64. -Kurt On 5/4/18 5:09 PM, iwama@t3.rim.or.jp wrote: > Hi Kurt, >=20 > Thank you for your reply. >=20 > Thank you for providing your achievements and procedures. I will=20 > try it in your procedure. >=20 > Please let me ask a question. > In your procedure, the RAID method is mirror, but is there a record = > of RAIDZ of 4 disks with V240? >=20 > Regards, > --- > Yoshihiko Iwama >=20 > -----Original Message----- > From: owner-freebsd-sparc64@freebsd.org=20 > > On Behalf Of Kurt Lidl > Sent: Friday, May 4, 2018 10:31 PM > To: freebsd-sparc64@freebsd.org > Subject: Re: Root on ZFS RAIDZ1 >=20 > I have been using various sparc64 machines with a ZFS only=20 > installation for quite a while, and found that it works reliably for=20 > me. My earlier machines were Netra-T1 (all retired now), and=20 > currently I have both a V240 and three > V120 machines using this configuration. >=20 > Here are my notes that I use during the partitioning phase of the = install. > (ie, start a normal install as discussed in the web page you mention,=20 > and during the 'Shell' method of partitioning, I run these commands): >=20 > You may or may not want to execute the commented out sysctl, which=20 > will force the ZFS installation to use 4096 byte blocks. > Given that all the disks I've ever used on sparc64 are really 512byte=20 > block devices, I'm not sure if this is a great idea or not. I=20 > typically do run with 4096 byte blocks. >=20 > ----------------------------------------------------------------- >=20 > kldload zfs >=20 > # force 4K minimum zfs block size > #sysctl vfs.zfs.min_auto_ashift=3D12 >=20 > # appropriate for a 36GB disk w/4+GB of swap zfssize=3D30000M #=20 > appropriate for a 72GB disk w/4+GB of swap zfssize=3D65500M #=20 > appropriate for a 72GB disk w/8+GB of swap zfssize=3D61500M >=20 > disks=3D"da0 da1" > for d in $disks > do > gpart destroy -F $d > gpart create -s VTOC8 $d > gpart add -t freebsd-zfs -s $zfssize $d > gpart add -t freebsd-swap $d > gpart bootcode -p /boot/zfsboot $d > dd if=3D/boot/zfsloader of=3D/dev/${d}a bs=3D512 = oseek=3D1024 \ > conv=3Dnotrunc,sync > done >=20 > # note, we use -R /mnt to set a different root mountpoint P=3Dzroot=20 > mnt=3D/mnt zpool create -f -o cachefile=3Dnone -O checksum=3Dfletcher4 = \ > -R $mnt -m none $P mirror /dev/da0a /dev/da1a >=20 > zfs create $P/ROOT > zfs create $P/ROOT/default > zpool set bootfs=3D$P/ROOT/default $P >=20 > # because of "-R /mnt" earlier, this is "/mnt" effectively for this=20 > session zfs set mountpoint=3D/ $P/ROOT/default >=20 > zfs create $P/vartmp > zfs create $P/usrports > zfs create $P/usrsrc > zfs create $P/usrobj > zfs create $P/home > zfs set mountpoint=3D/var/tmp $P/vartmp > zfs set mountpoint=3D/home $P/home > zfs set mountpoint=3D/usr/ports $P/usrports zfs set = mountpoint=3D/usr/src=20 > $P/usrsrc zfs set mountpoint=3D/usr/obj $P/usrobj zfs mount -a >=20 > gmirror label -b prefer gswap0 da0b da1b >=20 > eeprom boot-device=3D"disk0 disk1" >=20 > exit >=20 > ----------------------------------------------------------------- >=20 > At the end of the installation process, you will need to fix up the=20 > /etc/rc.conf and /boot/loader.conf too: >=20 > # Add to /etc/rc.conf: > clear_tmp_enable=3D"YES" > ntpdate_enable=3D"YES" > rwhod_enable=3D"YES" > rwhod_flags=3D"-m" > zfs_enable=3D"YES" >=20 > # Add to /boot/loader.conf: > zfs_load=3D"YES" > geom_mirror_load=3D"YES" > vfs.zfs.prefetch_disable=3D1 >=20 > # Add to /etc/fstab: > # Device Mountpoint FStype Options Dmp Pass# > /dev/mirror/gswap none swap sw 0 = 0 > #md /tmp mfs rw,-s64m 0 = 0 > tmpfs /tmp tmpfs = rw,size=3D64m,mode=3D1777 0 0 >=20 > # turn off ttyv? entries in /etc/ttys > # turn off ttyu2 entry in /etc/ttys >=20 > I hope this helps you get the system working well. >=20 > Best wishes, >=20 > -Kurt >=20 > On 5/4/18 5:08 AM, iwama@t3.rim.or.jp wrote: >> I want Sun Fire V245 to use FreeBSD-11.1-RELEASE with Root on ZFS > RAIDZ1. >> The installation procedure of Root on ZFS is as follow page: >> >> https://wiki.freebsd.org/RootOnZFS/VTOC8ZFSBoot/9.x-RELEASE >> >> I finished the installation without any problem, but it was some=20 >> problems at reboot. >> >> 1) It do not boot at all >> 2) boot takes a very long time, more than 3 HOURS >> 3) It takes some time for boot, about 10 minutes >> >> I tried RAIDZ1 with 9.3-RELEASE, RAIDZ2, RAIDZ10 and mirror with=20 >> 11.1-RELEASE, the results were as follows, da0-3 are the name of boot = >> disk. (Fire V245 has four disks): >> >> RAIDZ1(zpool create -o altroot=3D/mnt -O canmount=3Doff zroot raidz1 = da0a=20 >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE NG NG SLOW SLOW >> 9.3-RELEASE NG NG SLOW SLOW >> >> RAIDZ2(zpool create -o altroot=3D/mnt -O canmount=3Doff zroot raidz2 = da0a=20 >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE SLOW SLOW SLOW SLOW >> >> RAIDZ10(zpool create -o altroot=3D/mnt -O canmount=3Doff zroot mirror = >> da0a da1a mirror da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE slow slow slow slow >> >> mirror(zpool create -o altroot=3D/mnt -O canmount=3Doff zroot mirror = da0a=20 >> da1a da2a da3a) >> da0 da1 da2 da3 >> 11.1-RELEASE OK OK OK OK >> >> OK: OK(boot at normal speed), NG: 1), SLOW: 2), slow: 3) >> >> >> From the above results and the logs below, there is no problem=20 >> with mirror, but it is a big problem in RAIDZ1 or RAIDZ2. >> Has anyone been running Root on RAIDZ1 on the sparc64 platform? >> >> >> Install steps(only the part entered manually): >> =3D=3D=3D >> # zpool labelclear -f /dev/da0a >> # zpool labelclear -f /dev/da1a >> # zpool labelclear -f /dev/da2a >> # zpool labelclear -f /dev/da3a >> # >> # sysctl kern.disks >> kern.disks: cd0 da3 da2 da1 da0 >> # gpart destroy -F da0 >> da0 destroyed >> # gpart destroy -F da1 >> da1 destroyed >> # gpart destroy -F da2 >> da2 destroyed >> # gpart destroy -F da3 >> da3 destroyed >> # gpart create -s VTOC8 da0 >> da0 created >> # gpart create -s VTOC8 da1 >> da1 created >> # gpart create -s VTOC8 da2 >> da2 created >> # gpart create -s VTOC8 da3 >> da3 created >> # >> # gpart add -s 270G -t freebsd-zfs da0 da0a added # gpart add -s 270G = >> -t freebsd-zfs da1 da1a added # gpart add -s 270G -t freebsd-zfs da2=20 >> da2a added # gpart add -s 270G -t freebsd-zfs da3 da3a added # gpart=20 >> add -t freebsd-swap da0 da0b added # gpart add -t freebsd-swap da1=20 >> da1b added # gpart add -t freebsd-swap da2 da2b added # gpart add -t=20 >> freebsd-swap da3 da3b added # ***The following lines differ depending = >> on the RAID method # zpool create -o altroot=3D/mnt -O canmount=3Doff = >> zroot raidz1 da0a da1a da2a da3a # # zpool export zroot # gpart=20 >> bootcode -p /boot/zfsboot da0 # gpart bootcode -p /boot/zfsboot da1 # = >> gpart bootcode -p /boot/zfsboot da2 # gpart bootcode -p /boot/zfsboot >> da3 # sysctl kern.geom.debugflags=3D0x10 >> kern.geom.debugflags: 0 -> 16 >> # dd if=3D/boot/zfsloader of=3D/dev/da0a bs=3D512 oseek=3D1024=20 >> conv=3Dnotrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.845066 secs (84042 bytes/sec) # dd=20 >> if=3D/boot/zfsloader of=3D/dev/da1a bs=3D512 oseek=3D1024 = conv=3Dnotrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.832603 secs (84411 bytes/sec) # dd=20 >> if=3D/boot/zfsloader of=3D/dev/da2a bs=3D512 oseek=3D1024 = conv=3Dnotrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.815799 secs (84915 bytes/sec) # dd=20 >> if=3D/boot/zfsloader of=3D/dev/da3a bs=3D512 oseek=3D1024 = conv=3Dnotrunc,sync >> 466+1 records in >> 467+0 records out >> 239104 bytes transferred in 2.828102 secs (84546 bytes/sec) # zpool=20 >> import -o altroot=3D/mnt zroot # >> # zfs set checksum=3Dfletcher4 = zroot >> # zfs set atime=3Doff = zroot >> # >> # zfs create -o mountpoint=3D/ = zroot/ROOT >> >> # >> # zpool set bootfs=3Dzroot/ROOT zroot >> # >> # cat << EOF > /tmp/bsdinstall_etc/fstab >>> # Device Mountpoint FStype Options Dump Pass# >>> /dev/da0b none swap sw 0 0 >>> /dev/da1b none swap sw 0 0 >>> /dev/da2b none swap sw 0 0 >>> /dev/da3b none swap sw 0 0 >>> ^D# >> # >> # exit >> >> >> # echo 'zfs_enable=3D"YES"' >> /etc/rc.conf # # cat /boot/loader.conf = >> zfs_load=3D"YES" >> ***Except for 9.3-RELEASE, since the line of "zfs_load=3D"YES"" is=20 >> originally in ***/boot/loader.conf, addition is unnecessary # >> # zpool set cachefile=3D/boot/zfs/zpool.cache zroot >> # exit >> >> ***In 11.1-RELEASE, the last "zpool set=20 >> cachefile=3D/boot/zfs/zpool.cache zroot" failed, >> shown as "internal error: failed to initialize ZFS library" >> =3D=3D=3D >> >> >> NOT boot log: >> =3D=3D=3D >> {1} ok boot disk0 >> >> SC Alert: Host System has Reset >> Probing system devices >> Probing memory >> Probing I/O buses >> screen not found. >> keyboard not found. >> Keyboard not present. Using ttya for input and output. >> Probing system devices >> Probing memory >> Probing I/O buses >> >> >> Sun Fire V245, No Keyboard >> Copyright 2006 Sun Microsystems, Inc. All rights reserved. >> OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. >> Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. >> >> >> >> Initializing 1MB of memory at addr 137ff0a000 >> Initializing 1MB of memory at addr 137ff06000 >> Initializing 1MB of memory at addr 137ff00000 >> Initializing 1MB of memory at addr 137fee0000 >> Initializing 1MB of memory at addr 137fe90000 >> Initializing 1MB of memory at addr 137fdea000 >> Initializing 14MB of memory at addr 137f000000 >> Initializing 2032MB of memory at addr 1300000000 >> Initializing 2048MB of memory at addr 1200000000 >> Initializing 2048MB of memory at addr 1100000000 >> Initializing 2048MB of memory at addr 1000000000 >> Initializing 2048MB of memory at addr 300000000 >> Initializing 2048MB of memory at addr 200000000 >> Initializing 2048MB of memory at addr 100000000 >> Initializing 2048MB of memory at addr 0 >> Rebooting with command: boot disk0 >> Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 >> File and args: >> =20 >>>> FreeBSD/sparc64 ZFS boot block >> Boot path: = /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a >> Consoles: Open Firmware console >> ZFS: i/o error - all block copies unavailable >> ZFS: can't read MOS object directory >> ZFS: can't find root filesystem >> >> FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul=20 >> 21 >> 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org) bootpath=3D"" >> >> can't load 'kernel' >> >> Type '?' for a list of commands, 'help' for more detailed help. >> OK reboot >> Rebooting... >> Program terminated >> {1} ok >> =3D=3D=3D >> >> >> VERY SLOW boot log: >> =3D=3D=3D >> {1} ok boot disk3 >> >> SC Alert: Host System has Reset >> Probing system devices >> Probing memory >> Probing I/O buses >> screen not found. >> keyboard not found. >> Keyboard not present. Using ttya for input and output. >> Probing system devices >> Probing memory >> Probing I/O buses >> >> >> Sun Fire V245, No Keyboard >> Copyright 2006 Sun Microsystems, Inc. All rights reserved. >> OpenBoot 4.22.22, 16384 MB memory installed, Serial #XXXXXXXX. >> Ethernet address 0:14:4f:XX:XX:XX, Host ID: XXXXXXXX. >> >> >> >> Initializing 1MB of memory at addr 137ff0a000 >> Initializing 1MB of memory at addr 137ff06000 >> Initializing 1MB of memory at addr 137ff00000 >> Initializing 1MB of memory at addr 137fee0000 >> Initializing 1MB of memory at addr 137fe90000 >> Initializing 1MB of memory at addr 137fdea000 >> Initializing 14MB of memory at addr 137f000000 >> Initializing 2032MB of memory at addr 1300000000 >> Initializing 2048MB of memory at addr 1200000000 >> Initializing 2048MB of memory at addr 1100000000 >> Initializing 2048MB of memory at addr 1000000000 >> Initializing 2048MB of memory at addr 300000000 >> Initializing 2048MB of memory at addr 200000000 >> Initializing 2048MB of memory at addr 100000000 >> Initializing 2048MB of memory at addr 0 >> Rebooting with command: boot disk3 >> Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0 >> File and args: >> =20 >>>> FreeBSD/sparc64 ZFS boot block >> Boot path: = /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@3,0:a >> Consoles: Open Firmware console >> >> >> ***Slow from here(\|/-\|/- animation)*** >> FreeBSD/sparc64 ZFS enabled bootstrap loader, Revision 1.0 (Fri Jul=20 >> 21 >> 08:44:42 UTC 2017 root@releng2.nyi.freebsd.org)=20 >> bootpath=3D"zfs:zroot/ROOT:" >> Loading /boot/defaults/loader.conf >> /boot/kernel/kernel data=3D0xc47740+0xf4078=20 >> syms=3D[0x8+0xd5a38+0x8+0xc746a] /boot/entropy size=3D0x1000=20 >> /boot/kernel/zfs.ko text=3D0x25cc60 data=3D0xb9d0+0x90050 >> |syms=3D[0x8+0x1bd38/-+0x8+0x16db2] >> loading required module 'opensolaris' >> /boot/kernel/opensolaris.ko text=3D0x3730 data=3D0x2c8+0x1a30=20 >> syms=3D[0x8+0xe10+0x8+0x968] ***It is slow until the above is >> displayed.*** >> >> >> Hit [Enter] to boot immediately, or any other key for command prompt. >> Booting [/boot/kernel/kernel] in 9 seconds... Booting=20 >> [/boot/kernel/kernel] in 8 seconds... Booting [/boot/kernel/kernel]=20 >> in >> 7 seconds... Booting [/boot/kernel/kernel] in 6 seconds... Booting=20 >> [/boot/kernel/kernel] in 5 seconds... Booting [/boot/kernel/kernel]=20 >> in >> 4 seconds... Booting [/boot/kernel/kernel] in 3 seconds... Booting=20 >> [/boot/kernel/kernel] in 2 seconds... Booting [/boot/kernel/kernel]=20 >> in >> 1 second... Booting [/boot/kernel/kernel]... >> -\|/-\|/-\|/-\|/-\|jumping to kernel entry at 0xc00b0000. >> =3D=3D=3D >> >> --- >> Yoshihiko Iwama >> >> >> _______________________________________________ >> freebsd-sparc64@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 >> To unsubscribe, send any mail to = "freebsd-sparc64-unsubscribe@freebsd.org" >> >=20 > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to = "freebsd-sparc64-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to = "freebsd-sparc64-unsubscribe@freebsd.org" >=20