Date: Tue, 26 Feb 2013 19:30:09 +0100 From: "bsd@todoo.biz" <bsd@todoo.biz> To: Liste FreeBSD <freebsd-questions@freebsd.org> Subject: Hang on reboot with ZIL on SSD Message-ID: <6530A711-942C-4A16-94DD-572D67D7C9DC@todoo.biz>
next in thread | raw e-mail | index | archive | help
Hi,=20 I have a quite big server that I am tuning with FreeNAS running on It.=20= It is based on an Intel Server and uses an Adaptec Controler ASR-6805 = for a potential 12 disks pool (only 6 deployed for the moment).=20 I have two more SSD disks intended for the ZIL cache. Connected directly = on the mother board.=20 It comes equipped with 32Gb of memory ECC.=20 The system is installed on a specific dongle on the mother board (4Gb = SLC dongle).=20 The system is happy (=3D reboots without stopping at the real end of the = reboot) as long as there is no SSD involved for the ZIL. As soon as the = SSD are running, system freezes (or at least can't proceed with the = reboot).=20 It really freezes at the real end of the reboot after :=20 > Syncing disks, vnodes remaining=85*0 0 0 0 done=20 > All buffers synced.=20 > Uptime: 3d4h12min I have to manually Power-Cycle the unit for It to complete the reboot.=20= Here is the output of the dmesg :=20 > [root@freenas] ~# dmesg=20 > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, = 1994 > The Regents of the University of California. All rights = reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 8.3-RELEASE-p5 #2 r244158M: Wed Dec 12 10:04:42 PST 2012 > = root@build.ixsystems.com:/home/jpaetzel/8.3.0/os-base/amd64/usr/home/jpaet= zel/8.3.0/FreeBSD/src/sys/FREENAS.amd64 amd64 > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz (2394.25-MHz K8-class = CPU) > Origin =3D "GenuineIntel" Id =3D 0x206d7 Family =3D 6 Model =3D = 2d Stepping =3D 7 > = Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE= ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > = Features2=3D0x17bee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SS= SE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE= ,AVX> > AMD Features=3D0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> > AMD Features2=3D0x1<LAHF> > TSC: P-state invariant > real memory =3D 34359738368 (32768 MB) > avail memory =3D 33071357952 (31539 MB) > ACPI APIC Table: <INTEL S2600GZ> > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs > FreeBSD/SMP: 1 package(s) x 4 core(s) > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 2 > cpu2 (AP): APIC ID: 4 > cpu3 (AP): APIC ID: 6 > WARNING: VIMAGE (virtualized network stack) is a highly experimental = feature. > ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default = 16 (20101013/tbfadt-707) > ioapic0 <Version 2.0> irqs 0-23 on motherboard > ioapic1 <Version 2.0> irqs 24-47 on motherboard > kbd1 at kbdmux0 > hpt27xx: RocketRAID 27xx controller driver v1.0 (Dec 12 2012 10:04:31) > cryptosoft0: <software crypto> on motherboard > aesni0: <AES-CBC,AES-XTS> on motherboard > acpi0: <INTEL S2600GZ> on motherboard > acpi0: [ITHREAD] > acpi0: Power Button (fixed) > acpi0: reservation of 0, 9d000 (3) failed > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > cpu0: <ACPI CPU> on acpi0 > cpu1: <ACPI CPU> on acpi0 > cpu2: <ACPI CPU> on acpi0 > cpu3: <ACPI CPU> on acpi0 > pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 > pci0: <ACPI PCI bus> on pcib0 > pcib1: <ACPI PCI-PCI bridge> irq 47 at device 1.0 on pci0 > pci1: <ACPI PCI bus> on pcib1 > pcib2: <ACPI PCI-PCI bridge> irq 47 at device 1.1 on pci0 > pci2: <ACPI PCI bus> on pcib2 > igb0: <Intel(R) PRO/1000 Network Connection version - 2.3.1> port = 0x1060-0x107f mem 0xd2160000-0xd217ffff,0xd21b0000-0xd21b3fff irq 27 at = device 0.0 on pci2 > igb0: Using MSIX interrupts with 5 vectors > igb0: Ethernet address: 00:1e:67:54:9f:cd > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb1: <Intel(R) PRO/1000 Network Connection version - 2.3.1> port = 0x1040-0x105f mem 0xd2140000-0xd215ffff,0xd21a0000-0xd21a3fff irq 30 at = device 0.1 on pci2 > igb1: Using MSIX interrupts with 5 vectors > igb1: Ethernet address: 00:1e:67:54:9f:ce > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb2: <Intel(R) PRO/1000 Network Connection version - 2.3.1> port = 0x1020-0x103f mem 0xd2120000-0xd213ffff,0xd2190000-0xd2193fff irq 28 at = device 0.2 on pci2 > igb2: Using MSIX interrupts with 5 vectors > igb2: Ethernet address: 00:1e:67:54:9f:cf > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb3: <Intel(R) PRO/1000 Network Connection version - 2.3.1> port = 0x1000-0x101f mem 0xd2100000-0xd211ffff,0xd2180000-0xd2183fff irq 29 at = device 0.3 on pci2 > igb3: Using MSIX interrupts with 5 vectors > igb3: Ethernet address: 00:1e:67:54:9f:d0 > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > pcib3: <ACPI PCI-PCI bridge> irq 47 at device 2.0 on pci0 > pci4: <ACPI PCI bus> on pcib3 > pcib4: <ACPI PCI-PCI bridge> irq 47 at device 2.2 on pci0 > pci5: <ACPI PCI bus> on pcib4 > pcib5: <ACPI PCI-PCI bridge> irq 16 at device 3.0 on pci0 > pci6: <ACPI PCI bus> on pcib5 > aacu0: <Adaptec RAID Controller> mem = 0xd1c00000-0xd1ffffff,0xd2050000-0xd20507ff,0xd2040000-0xd20400ff irq 40 = at device 0.0 on pci6 > aacu0: Enable Raw I/O > aacu0: Enable 64-bit array > aacu0: New comm. interface type1 enabled > aacu0: [ITHREAD] > aacu0: Adaptec 6805, aac driver 3.1.2-30035 > aacp0: <Container Bus> on aacu0 > aacp1: <SCSI Passthrough Bus> on aacu0 > aacp2: <SCSI Passthrough Bus> on aacu0 > aacp3: <SCSI Passthrough Bus> on aacu0 > pcib6: <ACPI PCI-PCI bridge> irq 16 at device 3.2 on pci0 > pci7: <ACPI PCI bus> on pcib6 > pci0: <base peripheral> at device 4.0 (no driver attached) > pci0: <base peripheral> at device 4.1 (no driver attached) > pci0: <base peripheral> at device 4.2 (no driver attached) > pci0: <base peripheral> at device 4.3 (no driver attached) > pci0: <base peripheral> at device 4.4 (no driver attached) > pci0: <base peripheral> at device 4.5 (no driver attached) > pci0: <base peripheral> at device 4.6 (no driver attached) > pci0: <base peripheral> at device 4.7 (no driver attached) > pci0: <base peripheral> at device 5.0 (no driver attached) > pci0: <base peripheral> at device 5.2 (no driver attached) > pcib7: <ACPI PCI-PCI bridge> irq 16 at device 17.0 on pci0 > pci8: <ACPI PCI bus> on pcib7 > pci0: <simple comms> at device 22.0 (no driver attached) > pci0: <simple comms> at device 22.1 (no driver attached) > ehci0: <EHCI (generic) USB 2.0 controller> mem 0xd2320000-0xd23203ff = irq 22 at device 26.0 on pci0 > ehci0: [ITHREAD] > usbus0: EHCI version 1.0 > usbus0: <EHCI (generic) USB 2.0 controller> on ehci0 > pcib8: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0 > pci9: <ACPI PCI bus> on pcib8 > pcib9: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0 > pci10: <ACPI PCI bus> on pcib9 > vgapci0: <VGA-compatible display> mem = 0xd0000000-0xd0ffffff,0xd1810000-0xd1813fff,0xd1000000-0xd17fffff irq 19 = at device 0.0 on pci10 > ehci1: <EHCI (generic) USB 2.0 controller> mem 0xd2310000-0xd23103ff = irq 20 at device 29.0 on pci0 > ehci1: [ITHREAD] > usbus1: EHCI version 1.0 > usbus1: <EHCI (generic) USB 2.0 controller> on ehci1 > pcib10: <ACPI PCI-PCI bridge> at device 30.0 on pci0 > pci11: <ACPI PCI bus> on pcib10 > isab0: <PCI-ISA bridge> at device 31.0 on pci0 > isa0: <ISA bus> on isab0 > ahci0: <Intel Patsburg AHCI SATA controller> port = 0x2070-0x2077,0x2060-0x2063,0x2050-0x2057,0x2040-0x2043,0x2020-0x203f = mem 0xd2300000-0xd23007ff irq 21 at device 31.2 on pci0 > ahci0: [ITHREAD] > ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported > ahcich0: <AHCI channel> at channel 0 on ahci0 > ahcich0: [ITHREAD] > ahcich1: <AHCI channel> at channel 1 on ahci0 > ahcich1: [ITHREAD] > pci0: <serial bus, SMBus> at device 31.3 (no driver attached) > pcib11: <ACPI Host-PCI bridge> on acpi0 > pci255: <ACPI PCI bus> on pcib11 > pci255: <base peripheral> at device 8.0 (no driver attached) > pci255: <base peripheral> at device 9.0 (no driver attached) > pci255: <base peripheral> at device 10.0 (no driver attached) > pci255: <base peripheral> at device 10.1 (no driver attached) > pci255: <base peripheral> at device 10.2 (no driver attached) > pci255: <base peripheral> at device 10.3 (no driver attached) > pci255: <base peripheral> at device 11.0 (no driver attached) > pci255: <base peripheral> at device 11.3 (no driver attached) > pci255: <base peripheral> at device 12.0 (no driver attached) > pci255: <base peripheral> at device 12.1 (no driver attached) > pci255: <base peripheral> at device 12.6 (no driver attached) > pci255: <base peripheral> at device 12.7 (no driver attached) > pci255: <base peripheral> at device 13.0 (no driver attached) > pci255: <base peripheral> at device 13.1 (no driver attached) > pci255: <base peripheral> at device 13.6 (no driver attached) > pci255: <base peripheral> at device 14.0 (no driver attached) > pci255: <dasp> at device 14.1 (no driver attached) > pci255: <base peripheral> at device 15.0 (no driver attached) > pci255: <base peripheral> at device 15.1 (no driver attached) > pci255: <base peripheral> at device 15.2 (no driver attached) > pci255: <base peripheral> at device 15.3 (no driver attached) > pci255: <base peripheral> at device 15.4 (no driver attached) > pci255: <base peripheral> at device 15.5 (no driver attached) > pci255: <base peripheral> at device 15.6 (no driver attached) > pci255: <base peripheral> at device 16.0 (no driver attached) > pci255: <base peripheral> at device 16.1 (no driver attached) > pci255: <base peripheral> at device 16.2 (no driver attached) > pci255: <base peripheral> at device 16.3 (no driver attached) > pci255: <base peripheral> at device 16.4 (no driver attached) > pci255: <base peripheral> at device 16.5 (no driver attached) > pci255: <base peripheral> at device 16.6 (no driver attached) > pci255: <base peripheral> at device 16.7 (no driver attached) > pci255: <base peripheral> at device 17.0 (no driver attached) > pci255: <base peripheral> at device 19.0 (no driver attached) > pci255: <dasp> at device 19.1 (no driver attached) > pci255: <dasp> at device 19.4 (no driver attached) > pci255: <dasp> at device 19.5 (no driver attached) > pci255: <base peripheral> at device 19.6 (no driver attached) > acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff = on acpi0 > Timecounter "HPET" frequency 14318180 Hz quality 900 > atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 > atrtc0: Warning: Couldn't map I/O. > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on = acpi0 > uart0: [FILTER] > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > uart1: [FILTER] > orm0: <ISA Option ROMs> at iomem = 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff,0xcb000-0x= cbfff,0xcc000-0xd27ff on isa0 > sc0: <System console> at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=3D0x300> > vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 > atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 > atkbd0: <AT Keyboard> irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > atkbd0: [ITHREAD] > ppc0: cannot reserve I/O port range > coretemp0: <CPU On-Die Thermal Sensors> on cpu0 > est0: <Enhanced SpeedStep Frequency Control> on cpu0 > p4tcc0: <CPU Frequency Thermal Control> on cpu0 > coretemp1: <CPU On-Die Thermal Sensors> on cpu1 > est1: <Enhanced SpeedStep Frequency Control> on cpu1 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 1ee700001800 > device_attach: est1 attach returned 6 > p4tcc1: <CPU Frequency Thermal Control> on cpu1 > coretemp2: <CPU On-Die Thermal Sensors> on cpu2 > est2: <Enhanced SpeedStep Frequency Control> on cpu2 > p4tcc2: <CPU Frequency Thermal Control> on cpu2 > coretemp3: <CPU On-Die Thermal Sensors> on cpu3 > est3: <Enhanced SpeedStep Frequency Control> on cpu3 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 1f1000001800 > device_attach: est3 attach returned 6 > p4tcc3: <CPU Frequency Thermal Control> on cpu3 > fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8 > Timecounters tick every 1.000 msec > hpt27xx: no controller detected. > usbus0: 480Mbps High Speed USB v2.0 > usbus1: 480Mbps High Speed USB v2.0 > ugen0.1: <Intel> at usbus0 > uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on = usbus0 > ugen1.1: <Intel> at usbus1 > uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on = usbus1 > uhub0: 2 ports with 2 removable, self powered > uhub1: 2 ports with 2 removable, self powered > ugen0.2: <vendor 0x8087> at usbus0 > uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr = 2> on usbus0 > ugen1.2: <vendor 0x8087> at usbus1 > uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr = 2> on usbus1 > uhub2: 6 ports with 6 removable, self powered > uhub3: 8 ports with 8 removable, self powered > ugen1.3: <InnoDisk> at usbus1 > umass0: <InnoDisk USB EDC, class 0/0, rev 2.00/9.10, addr 3> on usbus1 > ugen1.4: <vendor 0x04d9> at usbus1 > ukbd0: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/3.10, addr 4> = on usbus1 > kbd2 at ukbd0 > uhid0: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/3.10, addr 4> = on usbus1 > ugen1.5: <American Megatrends Inc.> at usbus1 > ukbd1: <Keyboard Interface> on usbus1 > kbd3 at ukbd1 > ums0: <Mouse Interface> on usbus1 > ums0: 3 buttons and [Z] coordinates ID=3D0 > ada0 at ahcich0 bus 0 scbus4 target 0 lun 0 > ada0: <INTEL SSDSA2BZ100G3 6PB10362> ATA-8 SATA 2.x device > ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 95396MB (195371568 512 byte sectors: 16H 63S/T 16383C) > ada1 at ahcich1 bus 0 scbus5 target 0 lun 0 > ada1: <INTEL SSDSA2BZ100G3 6PB10362> ATA-8 SATA 2.x device > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 95396MB (195371568 512 byte sectors: 16H 63S/T 16383C) > da6 at umass-sim0 bus 0 scbus6 target 0 lun 0 > da6: <InnoDisk USB EDC 0910> Fixed Direct Access SCSI-0 device=20 > da6: 40.000MB/s transfers > da6: 3920MB (8028160 512 byte sectors: 255H 63S/T 499C) > SMP: AP CPU #1 Launched! > SMP: AP CPU #2 Launched! > SMP: AP CPU #3 Launched! > da0 at aacp1 bus 0 scbus1 target 0 lun 0 > da0: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da0: 300.000MB/s transfers > da0: Command Queueing enabled > da0: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da1 at aacp1 bus 0 scbus1 target 1 lun 0 > da1: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da1: 300.000MB/s transfers > da1: Command Queueing enabled > da1: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da2 at aacp1 bus 0 scbus1 target 4 lun 0 > da2: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da2: 300.000MB/s transfers > da2: Command Queueing enabled > da2: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da3 at aacp1 bus 0 scbus1 target 5 lun 0 > da3: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da3: 300.000MB/s transfers > da3: Command Queueing enabled > da3: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da4 at aacp1 bus 0 scbus1 target 6 lun 0 > da4: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da4: 300.000MB/s transfers > da4: Command Queueing enabled > da4: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da5 at aacp1 bus 0 scbus1 target 7 lun 0 > da5: <WDC WD30EFRX-68AX9N0 80.0> Fixed Direct Access SCSI-5 device=20 > da5: 300.000MB/s transfers > da5: Command Queueing enabled > da5: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > GEOM: da6s1: geometry does not match label (16h,63s !=3D 255h,63s). > Trying to mount root from ufs:/dev/ufs/FreeNASs1a > ZFS filesystem version 5 > ZFS storage pool version 28 Here is an example of the pool with only one disk used for ZIL (not the = target) - But I tried this in order to see how It worked=85=20 > [root@freenas] ~# zpool status > pool: tank > state: ONLINE > scan: none requested > config: >=20 > NAME STATE READ = WRITE CKSUM > tank ONLINE 0 = 0 0 > raidz2-0 ONLINE 0 = 0 0 > gptid/053bf90c-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/08a7dcbd-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/0c0767d0-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/0f6ac358-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/12ce2b68-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/162f71c0-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > logs > gptid/17980c47-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 >=20 > errors: No known data errors Is there some tuning to be done on the Bios when you use disks as ZIL ?=20= Thanks for your feedback.=20 G.B.=20 ________________________________________________ =AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?= =BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7 =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF Your provider of OpenSource Appliances www.osnet.eu =09 ________________________________________________ =AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?= =BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7 =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF PGP ID --> 0x1BA3C2FD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6530A711-942C-4A16-94DD-572D67D7C9DC>