Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 1 Oct 2016 18:02:40 +0200
From:      Rainer Duffner <rainer@ultra-secure.de>
To:        FreeBSD Filesystems <freebsd-fs@freebsd.org>
Cc:        freebsd-stable <freebsd-stable@freebsd.org>
Subject:   zfs receive leads to a stuck server
Message-ID:  <4707908B-4868-4AA6-ADD6-D24121EFAE38@ultra-secure.de>

next in thread | raw e-mail | index | archive | help
Hi,

I posted this before, but I didn=E2=80=99t really get an answer and =
I=E2=80=99m still looking for ways to debug this.

I have to servers (HP DL 380 Gen8, 192 GB RAM, 6C CPU).
I=E2=80=99ve outfitted them with HP=E2=80=99s H22x cards (really OEMed =
9207-8x, three altogether) that I recently cross-flashed to LSI=E2=80=99s =
latest firmware.
They have:
 - 14 600GB SAS disks in their internal cages (2x ZFs mirror to boot =
from and 2x RAIDZ2)
 - another 12 1200 GB SAS disks in an external HP 2700 enclosure (2x =
RAID-Z2)
 - all the RAID Z2s together form a single zpool

These are NFS-servers (master + warm standby) and MySQL slaves.
The client uses the NFS-server and previously used both MySQL-Slaves for =
SELECT queries only (via a middleware that seems to be able to sort this =
out).

When this system was built, we had twelve single RAID0 logical volumes =
on HP P4x0 RAID-controllers (with 2 GB cache), merged into two RAID-Z2 =
vdevs for storage (and a RAID1 boot-drive).
However, the customer outgrew that solution and because there are no =
native HP utilities for managing the RAID-cards on FreeBSD, each time a =
drive dies on such a setup, you have to reboot the server to make sure =
ZFS =E2=80=9Esees=E2=80=9C the new drive (go into adapter BIOS, destroy =
RAID, create RAID=E2=80=A6).

So, I switched to LSI-type JBOD HBAs.
I had to re-install both servers from scratch, obviously, and also =
upgraded to 10.3 (from 10.1).


I have zfsnap (sysutils/zfsnap) create hourly, daily and weekly =
snapshots that I transfer via zxfer from server1 to server2.

Previously, this only took a couple of seconds. But since I switched to =
LSI HBAs, it=E2=80=99s taking longer - even though the deltas are really =
small in most cases.
After 03:00 AM, when zfsnap deletes some snapshots on the source system, =
it even takes several minutes to zxfer the snapshots.
At that point, it=E2=80=99s really noticeable that the 2nd system =
complete freezes. The first system (that zfs sends) is absolutely =
unaffected and continues like normal.

I=E2=80=99ve setup a cron-job that outputs iostat and vmstat into =
second-timestamped files and a few seconds after the zxfer starts, it =
doesn=E2=80=99t generate any new files until shortly before it finishes.
With these hangs, the customer can=E2=80=99t use the 2nd MySQL-server =
and is thus unhappy.

I=E2=80=99ve upgraded to 11.0-RELEAE on the 2nd system - but it hasn=E2=80=
=99t helped (it might even be slower actually).

How can I debug this? What=E2=80=99s the best way forward?
I don=E2=80=99t have any ZFS errors (checksum etc.). So, I doubt it=E2=80=99=
s a cable.


Here=E2=80=99s various system information


------------------------------------------------------------------------
ZFS Subsystem Report				Sat Oct  1 17:50:48 2016
------------------------------------------------------------------------

System Information:

	Kernel Version:				1100122 (osreldate)
	Hardware Platform:			amd64
	Processor Architecture:			amd64

	ZFS Storage pool Version:		5000
	ZFS Filesystem Version:			5

FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016 root
 5:50PM  up 2 days, 19:50, 1 users, load averages: 0.11, 0.09, 0.08

------------------------------------------------------------------------

System Memory:

	0.26%	506.77	MiB Active,	11.52%	21.54	GiB Inact
	62.29%	116.49	GiB Wired,	0.00%	0 Cache
	25.93%	48.49	GiB Free,	0.00%	0 Gap

	Real Installed:				192.00	GiB
	Real Available:			99.96%	191.93	GiB
	Real Managed:			97.44%	187.01	GiB

	Logical Total:				192.00	GiB
	Logical Used:			63.53%	121.97	GiB
	Logical Free:			36.47%	70.03	GiB

Kernel Memory:					2.15	GiB
	Data:				98.41%	2.12	GiB
	Text:				1.59%	35.02	MiB

Kernel Memory Map:				187.01	GiB
	Size:				40.16%	75.10	GiB
	Free:				59.84%	111.92	GiB

------------------------------------------------------------------------

ARC Summary: (HEALTHY)
	Memory Throttle Count:			0

ARC Misc:
	Deleted:				2.16m
	Recycle Misses:				0
	Mutex Misses:				1.92k
	Evict Skips:				54.73k

ARC Size:				64.39%	41.21	GiB
	Target Size: (Adaptive)		100.00%	64.00	GiB
	Min Size (Hard Limit):		36.33%	23.25	GiB
	Max Size (High Water):		2:1	64.00	GiB

ARC Size Breakdown:
	Recently Used Cache Size:	89.29%	57.14	GiB
	Frequently Used Cache Size:	10.71%	6.86	GiB

ARC Hash Breakdown:
	Elements Max:				5.37m
	Elements Current:		92.43%	4.97m
	Collisions:				2.58m
	Chain Max:				5
	Chains:					333.76k

------------------------------------------------------------------------

ARC Efficiency:					265.65m
	Cache Hit Ratio:		61.83%	164.25m
	Cache Miss Ratio:		38.17%	101.40m
	Actual Hit Ratio:		55.18%	146.57m

	Data Demand Efficiency:		10.17%	7.66m
	Data Prefetch Efficiency:	0.00%	32

	CACHE HITS BY CACHE LIST:
	  Anonymously Used:		8.77%	14.41m
	  Most Recently Used:		29.22%	48.00m
	  Most Frequently Used:		60.02%	98.58m
	  Most Recently Used Ghost:	1.67%	2.74m
	  Most Frequently Used Ghost:	0.33%	536.90k

	CACHE HITS BY DATA TYPE:
	  Demand Data:			0.47%	778.95k
	  Prefetch Data:		0.00%	0
	  Demand Metadata:		88.30%	145.04m
	  Prefetch Metadata:		11.22%	18.43m

	CACHE MISSES BY DATA TYPE:
	  Demand Data:			6.79%	6.88m
	  Prefetch Data:		0.00%	32
	  Demand Metadata:		84.42%	85.60m
	  Prefetch Metadata:		8.79%	8.91m

------------------------------------------------------------------------

L2ARC is disabled

------------------------------------------------------------------------

File-Level Prefetch: (HEALTHY)

DMU Efficiency:					970.94m
	Hit Ratio:			0.22%	2.09m
	Miss Ratio:			99.78%	968.85m

	Colinear:				0
	  Hit Ratio:			100.00%	0
	  Miss Ratio:			100.00%	0

	Stride:					0
	  Hit Ratio:			100.00%	0
	  Miss Ratio:			100.00%	0

DMU Misc:
	Reclaim:				0
	  Successes:			100.00%	0
	  Failures:			100.00%	0

	Streams:				0
	  +Resets:			100.00%	0
	  -Resets:			100.00%	0
	  Bogus:				0

------------------------------------------------------------------------

VDEV cache is disabled

------------------------------------------------------------------------

ZFS Tunables (sysctl):
	kern.maxusers                           12619
	vm.kmem_size                            200803987456
	vm.kmem_size_scale                      1
	vm.kmem_size_min                        0
	vm.kmem_size_max                        1319413950874
	vfs.zfs.trim.max_interval               1
	vfs.zfs.trim.timeout                    30
	vfs.zfs.trim.txg_delay                  32
	vfs.zfs.trim.enabled                    1
	vfs.zfs.vol.unmap_enabled               1
	vfs.zfs.vol.recursive                   0
	vfs.zfs.vol.mode                        1
	vfs.zfs.version.zpl                     5
	vfs.zfs.version.spa                     5000
	vfs.zfs.version.acl                     1
	vfs.zfs.version.ioctl                   6
	vfs.zfs.debug                           1
	vfs.zfs.super_owner                     0
	vfs.zfs.sync_pass_rewrite               2
	vfs.zfs.sync_pass_dont_compress         5
	vfs.zfs.sync_pass_deferred_free         2
	vfs.zfs.zio.exclude_metadata            0
	vfs.zfs.zio.use_uma                     1
	vfs.zfs.cache_flush_disable             0
	vfs.zfs.zil_replay_disable              0
	vfs.zfs.min_auto_ashift                 12
	vfs.zfs.max_auto_ashift                 13
	vfs.zfs.vdev.trim_max_pending           10000
	vfs.zfs.vdev.bio_delete_disable         0
	vfs.zfs.vdev.bio_flush_disable          0
	vfs.zfs.vdev.write_gap_limit            4096
	vfs.zfs.vdev.read_gap_limit             32768
	vfs.zfs.vdev.aggregation_limit          131072
	vfs.zfs.vdev.trim_max_active            64
	vfs.zfs.vdev.trim_min_active            1
	vfs.zfs.vdev.scrub_max_active           2
	vfs.zfs.vdev.scrub_min_active           1
	vfs.zfs.vdev.async_write_max_active     10
	vfs.zfs.vdev.async_write_min_active     1
	vfs.zfs.vdev.async_read_max_active      3
	vfs.zfs.vdev.async_read_min_active      1
	vfs.zfs.vdev.sync_write_max_active      10
	vfs.zfs.vdev.sync_write_min_active      10
	vfs.zfs.vdev.sync_read_max_active       10
	vfs.zfs.vdev.sync_read_min_active       10
	vfs.zfs.vdev.max_active                 1000
	vfs.zfs.vdev.async_write_active_max_dirty_percent60
	vfs.zfs.vdev.async_write_active_min_dirty_percent30
	vfs.zfs.vdev.mirror.non_rotating_seek_inc1
	vfs.zfs.vdev.mirror.non_rotating_inc    0
	vfs.zfs.vdev.mirror.rotating_seek_offset1048576
	vfs.zfs.vdev.mirror.rotating_seek_inc   5
	vfs.zfs.vdev.mirror.rotating_inc        0
	vfs.zfs.vdev.trim_on_init               1
	vfs.zfs.vdev.cache.bshift               16
	vfs.zfs.vdev.cache.size                 0
	vfs.zfs.vdev.cache.max                  16384
	vfs.zfs.vdev.metaslabs_per_vdev         200
	vfs.zfs.txg.timeout                     5
	vfs.zfs.space_map_blksz                 4096
	vfs.zfs.spa_slop_shift                  5
	vfs.zfs.spa_asize_inflation             24
	vfs.zfs.deadman_enabled                 1
	vfs.zfs.deadman_checktime_ms            5000
	vfs.zfs.deadman_synctime_ms             1000000
	vfs.zfs.debug_flags                     0
	vfs.zfs.recover                         0
	vfs.zfs.spa_load_verify_data            1
	vfs.zfs.spa_load_verify_metadata        1
	vfs.zfs.spa_load_verify_maxinflight     10000
	vfs.zfs.ccw_retry_interval              300
	vfs.zfs.check_hostid                    1
	vfs.zfs.mg_fragmentation_threshold      85
	vfs.zfs.mg_noalloc_threshold            0
	vfs.zfs.condense_pct                    200
	vfs.zfs.metaslab.bias_enabled           1
	vfs.zfs.metaslab.lba_weighting_enabled  1
	vfs.zfs.metaslab.fragmentation_factor_enabled1
	vfs.zfs.metaslab.preload_enabled        1
	vfs.zfs.metaslab.preload_limit          3
	vfs.zfs.metaslab.unload_delay           8
	vfs.zfs.metaslab.load_pct               50
	vfs.zfs.metaslab.min_alloc_size         33554432
	vfs.zfs.metaslab.df_free_pct            4
	vfs.zfs.metaslab.df_alloc_threshold     131072
	vfs.zfs.metaslab.debug_unload           0
	vfs.zfs.metaslab.debug_load             0
	vfs.zfs.metaslab.fragmentation_threshold70
	vfs.zfs.metaslab.gang_bang              16777217
	vfs.zfs.free_bpobj_enabled              1
	vfs.zfs.free_max_blocks                 -1
	vfs.zfs.no_scrub_prefetch               0
	vfs.zfs.no_scrub_io                     0
	vfs.zfs.resilver_min_time_ms            3000
	vfs.zfs.free_min_time_ms                1000
	vfs.zfs.scan_min_time_ms                1000
	vfs.zfs.scan_idle                       50
	vfs.zfs.scrub_delay                     4
	vfs.zfs.resilver_delay                  2
	vfs.zfs.top_maxinflight                 32
	vfs.zfs.zfetch.array_rd_sz              1048576
	vfs.zfs.zfetch.max_distance             8388608
	vfs.zfs.zfetch.min_sec_reap             2
	vfs.zfs.zfetch.max_streams              8
	vfs.zfs.prefetch_disable                0
	vfs.zfs.delay_scale                     500000
	vfs.zfs.delay_min_dirty_percent         60
	vfs.zfs.dirty_data_sync                 67108864
	vfs.zfs.dirty_data_max_percent          10
	vfs.zfs.dirty_data_max_max              4294967296
	vfs.zfs.dirty_data_max                  4294967296
	vfs.zfs.max_recordsize                  1048576
	vfs.zfs.mdcomp_disable                  0
	vfs.zfs.nopwrite_enabled                1
	vfs.zfs.dedup.prefetch                  1
	vfs.zfs.l2c_only_size                   0
	vfs.zfs.mfu_ghost_data_lsize            57031680
	vfs.zfs.mfu_ghost_metadata_lsize        24999013888
	vfs.zfs.mfu_ghost_size                  25056045568
	vfs.zfs.mfu_data_lsize                  7619584
	vfs.zfs.mfu_metadata_lsize              2427032064
	vfs.zfs.mfu_size                        7355872768
	vfs.zfs.mru_ghost_data_lsize            0
	vfs.zfs.mru_ghost_metadata_lsize        27116232704
	vfs.zfs.mru_ghost_size                  27116232704
	vfs.zfs.mru_data_lsize                  12935902208
	vfs.zfs.mru_metadata_lsize              18539549696
	vfs.zfs.mru_size                        31494193152
	vfs.zfs.anon_data_lsize                 0
	vfs.zfs.anon_metadata_lsize             0
	vfs.zfs.anon_size                       233472
	vfs.zfs.l2arc_norw                      1
	vfs.zfs.l2arc_feed_again                1
	vfs.zfs.l2arc_noprefetch                1
	vfs.zfs.l2arc_feed_min_ms               200
	vfs.zfs.l2arc_feed_secs                 1
	vfs.zfs.l2arc_headroom                  2
	vfs.zfs.l2arc_write_boost               8388608
	vfs.zfs.l2arc_write_max                 8388608
	vfs.zfs.arc_meta_limit                  17179869184
	vfs.zfs.arc_free_target                 339900
	vfs.zfs.arc_shrink_shift                7
	vfs.zfs.arc_average_blocksize           8192
	vfs.zfs.arc_min                         24966280704
	vfs.zfs.arc_max                         68719476736

=
=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=
=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=
=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=
=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=
=80=94=E2=80=94=E2=80=94






Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights =
reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016
    root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on =
LLVM 3.8.0)
VT(vga): resolution 640x480
CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (2294.52-MHz K8-class CPU)
  Origin=3D"GenuineIntel"  Id=3D0x206d7  Family=3D0x6  Model=3D0x2d  =
Stepping=3D7
  =
Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE=
,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  =
Features2=3D0x1fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SS=
SE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE=
,OSXSAVE,AVX>
  AMD Features=3D0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=3D0x1<LAHF>
  XSAVE Features=3D0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  =3D 206158430208 (196608 MB)
avail memory =3D 200330240000 (191049 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <HP     ProLiant>
FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs
FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads
random: unblocking device.
ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, =
using default 16 (20160527/tbfadt-733)
ACPI BIOS Warning (bug): Invalid length for FADT/Pm2ControlBlock: 32, =
using default 8 (20160527/tbfadt-733)
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic0 <Version 2.0> irqs 0-23 on motherboard
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff8101c950, 0) error 19
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <HP ProLiant> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu1: <ACPI CPU> numa-domain 0 on acpi0
cpu2: <ACPI CPU> numa-domain 0 on acpi0
cpu3: <ACPI CPU> numa-domain 0 on acpi0
cpu4: <ACPI CPU> numa-domain 0 on acpi0
cpu5: <ACPI CPU> numa-domain 0 on acpi0
cpu6: <ACPI CPU> numa-domain 0 on acpi0
cpu7: <ACPI CPU> numa-domain 0 on acpi0
cpu8: <ACPI CPU> numa-domain 0 on acpi0
cpu9: <ACPI CPU> numa-domain 0 on acpi0
cpu10: <ACPI CPU> numa-domain 0 on acpi0
cpu11: <ACPI CPU> numa-domain 0 on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Event timer "HPET4" frequency 14318180 Hz quality 340
Event timer "HPET5" frequency 14318180 Hz quality 340
Event timer "HPET6" frequency 14318180 Hz quality 340
Event timer "HPET7" frequency 14318180 Hz quality 340
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0
pcib0: <ACPI Host-PCI bridge> numa-domain 0 on acpi0
pcib0: _OSC returned error 0x10
pci0: <ACPI PCI bus> numa-domain 0 on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 0 on pci0
pci1: <ACPI PCI bus> numa-domain 0 on pcib1
mps0: <Avago Technologies (LSI) SAS2308> port 0x6000-0x60ff mem =
0xfbef0000-0xfbefffff,0xfbe80000-0xfbebffff irq 26 at device 0.0 =
numa-domain 0 on pci1
mps0: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps0: IOCCapabilities: =
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>=

pcib2: <ACPI PCI-PCI bridge> at device 1.1 numa-domain 0 on pci0
pci2: <ACPI PCI bus> numa-domain 0 on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 0 on pci0
pci3: <ACPI PCI bus> numa-domain 0 on pcib3
bge0: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem =
0xfabf0000-0xfabfffff,0xfabe0000-0xfabeffff,0xfabd0000-0xfabdffff irq 32 =
at device 0.0 numa-domain 0 on pci3
bge0: APE FW version: NCSI v1.3.12.0
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus0: <MII bus> numa-domain 0 on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, =
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 2c:44:fd:91:c2:bc
bge1: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem =
0xfabc0000-0xfabcffff,0xfabb0000-0xfabbffff,0xfaba0000-0xfabaffff irq 36 =
at device 0.1 numa-domain 0 on pci3
bge1: APE FW version: NCSI v1.3.12.0
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus1: <MII bus> numa-domain 0 on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, =
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 2c:44:fd:91:c2:bd
bge2: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem =
0xfab90000-0xfab9ffff,0xfab80000-0xfab8ffff,0xfab70000-0xfab7ffff irq 32 =
at device 0.2 numa-domain 0 on pci3
bge2: APE FW version: NCSI v1.3.12.0
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus2: <MII bus> numa-domain 0 on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, =
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: Ethernet address: 2c:44:fd:91:c2:be
bge3: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem =
0xfab60000-0xfab6ffff,0xfab50000-0xfab5ffff,0xfab40000-0xfab4ffff irq 36 =
at device 0.3 numa-domain 0 on pci3
bge3: APE FW version: NCSI v1.3.12.0
bge3: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus3: <MII bus> numa-domain 0 on bge3
brgphy3: <BCM5719C 1000BASE-T media interface> PHY 4 on miibus3
brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, =
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge3: Using defaults for TSO: 65518/35/2048
bge3: Ethernet address: 2c:44:fd:91:c2:bf
pcib4: <ACPI PCI-PCI bridge> at device 2.1 numa-domain 0 on pci0
pci4: <ACPI PCI bus> numa-domain 0 on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 2.2 numa-domain 0 on pci0
pci5: <ACPI PCI bus> numa-domain 0 on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 2.3 numa-domain 0 on pci0
pci6: <ACPI PCI bus> numa-domain 0 on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 0 on pci0
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
mps1: <Avago Technologies (LSI) SAS2308> port 0x5000-0x50ff mem =
0xfbdf0000-0xfbdfffff,0xfbd80000-0xfbdbffff irq 40 at device 0.0 =
numa-domain 0 on pci7
mps1: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps1: IOCCapabilities: =
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>=

pcib8: <ACPI PCI-PCI bridge> at device 3.1 numa-domain 0 on pci0
pci8: <ACPI PCI bus> numa-domain 0 on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 0 on pci0
pci9: <ACPI PCI bus> numa-domain 0 on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 3.3 numa-domain 0 on pci0
pci10: <ACPI PCI bus> numa-domain 0 on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 17.0 numa-domain 0 on pci0
pci11: <ACPI PCI bus> numa-domain 0 on pcib11
ehci0: <Intel Patsburg USB 2.0 controller> mem 0xfac60000-0xfac603ff irq =
21 at device 26.0 numa-domain 0 on pci0
usbus0: EHCI version 1.0
usbus0 numa-domain 0 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 28.0 numa-domain 0 on pci0
pci12: <ACPI PCI bus> numa-domain 0 on pcib12
mps2: <Avago Technologies (LSI) SAS2308> port 0x7000-0x70ff mem =
0xfbff0000-0xfbffffff,0xfbf80000-0xfbfbffff irq 16 at device 0.0 =
numa-domain 0 on pci12
mps2: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps2: IOCCapabilities: =
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>=

pcib13: <ACPI PCI-PCI bridge> at device 28.7 numa-domain 0 on pci0
pci13: <ACPI PCI bus> numa-domain 0 on pcib13
vgapci0: <VGA-compatible display> mem =
0xf9000000-0xf9ffffff,0xfbce0000-0xfbce3fff,0xfb000000-0xfb7fffff irq 16 =
at device 0.1 numa-domain 0 on pci13
vgapci0: Boot video device
uhci0: <HP iLO Standard Virtual USB controller> port 0x3c00-0x3c1f irq =
16 at device 0.4 numa-domain 0 on pci13
usbus1 numa-domain 0 on uhci0
ehci1: <Intel Patsburg USB 2.0 controller> mem 0xfac50000-0xfac503ff irq =
20 at device 29.0 numa-domain 0 on pci0
usbus2: EHCI version 1.0
usbus2 numa-domain 0 on ehci1
pcib14: <PCI-PCI bridge> at device 30.0 numa-domain 0 on pci0
pci14: <PCI bus> numa-domain 0 on pcib14
isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci0
isa0: <ISA bus> numa-domain 0 on isab0
atapci0: <Intel Patsburg SATA600 controller> port =
0x4000-0x4007,0x4008-0x400b,0x4010-0x4017,0x4018-0x401b,0x4020-0x402f,0x40=
30-0x403f irq 17 at device 31.2 numa-domain 0 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
acpi_tz0: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart1: <Non-standard ns8250 class UART with FIFOs> port 0x2f8-0x2ff irq =
3 on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
ppc0: cannot reserve I/O port range
uart0: <Non-standard ns8250 class UART with FIFOs> at port 0x3f8 irq 4 =
flags 0x10 on isa0
uart0: console (115200,n,8,1)
est0: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est0 attach returned 6
est1: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est1 attach returned 6
est2: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est2 attach returned 6
est3: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est3 attach returned 6
est4: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est4 attach returned 6
est5: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est5 attach returned 6
est6: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est6 attach returned 6
est7: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est7 attach returned 6
est8: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu8
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est8 attach returned 6
est9: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu9
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est9 attach returned 6
est10: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu10
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est10 attach returned 6
est11: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu11
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est11 attach returned 6
usbus0: 480Mbps High Speed USB v2.0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
nvme cam probe device init
usbus1: 12Mbps Full Speed USB v1.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <HP> at usbus1
uhub1: <HP UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
usbus2: 480Mbps High Speed USB v2.0
ugen2.1: <Intel> at usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
ugen1.2: <BMC> at usbus1
ukbd0: <Virtual Keyboard > on usbus1
kbd2 at ukbd0
ugen0.2: <vendor 0x8087> at usbus0
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> =
on usbus0
ugen2.2: <vendor 0x8087> at usbus2
uhub4: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> =
on usbus2
uhub3: 6 ports with 6 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ugen2.3: <vendor 0x0424> at usbus2
uhub5: <vendor 0x0424 product 0x2660, class 9/0, rev 2.00/8.01, addr 3> =
on usbus2
uhub5: 2 ports with 1 removable, self powered
ses0 at mps2 bus 0 scbus2 target 10 lun 0
ses0: <HP D2700 SAS AJ941A 0147> Fixed Enclosure Services SPC-3 SCSI =
device
ses0: Serial Number 7CE539P12M     =20
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 ENC Device
(da0:mps0:0:8:0): UNMAPPED
(da1:da0 at mps0 bus 0 scbus0 target 8 lun 0
da0: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number S0M1F2B10000B418HVFX
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 572325MB (1172123568 512 byte sectors)
mps0:0:9:0): UNMAPPED
(da2:da1 at mps0 bus 0 scbus0 target 9 lun 0
da1: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number S0M1F7CG0000M419BS87
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 572325MB (1172123568 512 byte sectors)
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 1, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec197c9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 2, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ebde785
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 3, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec1ae59
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 4, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008eb2e04d
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 5, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ebdfe25
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 6, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ebfea71
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 7, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec3f06d
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 8, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ebf2ab5
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 9, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec3c2f9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 10, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ebedb35
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 11, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec012b9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 12, Not All Phys
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 5001438035a2617f addr 5000c5008ec58d09
mps0:0:10:0): UNMAPPED
(da3:da2 at mps0 bus 0 scbus0 target 10 lun 0
da2: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number S0M1EQSL0000B419CE37
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 572325MB (1172123568 512 byte sectors)
mps0:0:11:0): UNMAPPED
(da4:da3 at mps0 bus 0 scbus0 target 11 lun 0
da3: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number S0M1EQHL0000B419CG8Y
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 572325MB (1172123568 512 byte sectors)
mps0:0:12:0): UNMAPPED
(da5:da4 at mps0 bus 0 scbus0 target 12 lun 0
da4: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number S0M1ERQR0000M41919NZ
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 572325MB (1172123568 512 byte sectors)
mps0:0:13:0): UNMAPPED
(da6:da5 at mps0 bus 0 scbus0 target 13 lun 0
da5: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number S0M1EP8H0000M419667Y
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 572325MB (1172123568 512 byte sectors)
mps0:0:14:0): UNMAPPED
(da7:da6 at mps0 bus 0 scbus0 target 14 lun 0
da6: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number S0M1F9B00000B419CQF1
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 572325MB (1172123568 512 byte sectors)
mps0:0:15:0): UNMAPPED
(da8:da7 at mps0 bus 0 scbus0 target 15 lun 0
da7: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number S0M1ER7Y0000B418J2VA
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 572325MB (1172123568 512 byte sectors)
mps1:0:8:0): UNMAPPED
(da9:da8 at mps1 bus 0 scbus1 target 8 lun 0
da8: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number S0M1E2VZ0000B419CQAF
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 572325MB (1172123568 512 byte sectors)
mps1:0:9:0): UNMAPPED
(da10:da9 at mps1 bus 0 scbus1 target 9 lun 0
da9: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number S0M1ESN20000M418DKFL
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 572325MB (1172123568 512 byte sectors)
mps1:0:10:0): UNMAPPED
(da11:da10 at mps1 bus 0 scbus1 target 10 lun 0
da10: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number S0M1F8V00000B419CPWR
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 572325MB (1172123568 512 byte sectors)
mps1:0:11:0): UNMAPPED
(da12:da11 at mps1 bus 0 scbus1 target 11 lun 0
da11: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number S0M1ESLL0000B419DALG
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 572325MB (1172123568 512 byte sectors)
mps1:0:12:0): UNMAPPED
(da13:da12 at mps1 bus 0 scbus1 target 12 lun 0
da12: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number S0M1DXJR0000B419CJKG
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: 572325MB (1172123568 512 byte sectors)
mps1:0:13:0): UNMAPPED
(da14:da13 at mps1 bus 0 scbus1 target 13 lun 0
da13: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number S0M1ES7R0000B419CJZX
da13: 600.000MB/s transfers
da13: Command Queueing enabled
da13: 572325MB (1172123568 512 byte sectors)
mps1:0:14:0): UNMAPPED
(da15:da14 at mps1 bus 0 scbus1 target 14 lun 0
da14: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da14: Serial Number S0M19J9D0000B419CEEC
da14: 600.000MB/s transfers
da14: Command Queueing enabled
da14: 572325MB (1172123568 512 byte sectors)
mps1:0:15:0): UNMAPPED
(da16:da15 at mps1 bus 0 scbus1 target 15 lun 0
da15: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da15: Serial Number S0M1EQPR0000B419CGAC
da15: 600.000MB/s transfers
da15: Command Queueing enabled
da15: 572325MB (1172123568 512 byte sectors)
mps2:0:11:0): UNMAPPED
(da17:da16 at mps2 bus 0 scbus2 target 11 lun 0
da16: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da16: Serial Number S3L29R3L0000M608CP3Y
da16: 600.000MB/s transfers
da16: Command Queueing enabled
da16: 1144641MB (2344225968 512 byte sectors)
mps2:0:13:0): UNMAPPED
(da18:da17 at mps2 bus 0 scbus2 target 13 lun 0
da17: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da17: Serial Number S3L29XFQ0000M608CQD6
da17: 600.000MB/s transfers
da17: Command Queueing enabled
da17: 1144641MB (2344225968 512 byte sectors)
mps2:0:14:0): UNMAPPED
(da19:da18 at mps2 bus 0 scbus2 target 14 lun 0
da18: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da18: Serial Number S3L29QTK0000M608CN9K
da18: 600.000MB/s transfers
da18: Command Queueing enabled
da18: 1144641MB (2344225968 512 byte sectors)
mps2:0:15:0): UNMAPPED
(da20:da19 at mps2 bus 0 scbus2 target 15 lun 0
da19: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da19: Serial Number S3L29TAA0000M608CS65
da19: 600.000MB/s transfers
da19: Command Queueing enabled
da19: 1144641MB (2344225968 512 byte sectors)
mps2:0:16:0): UNMAPPED
(da21:da20 at mps2 bus 0 scbus2 target 16 lun 0
da20: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da20: Serial Number S3L28ZFA0000M607T1JN
da20: 600.000MB/s transfers
da20: Command Queueing enabled
da20: 1144641MB (2344225968 512 byte sectors)
mps2:0:17:0): UNMAPPED
(da22:da21 at mps2 bus 0 scbus2 target 17 lun 0
da21: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da21: Serial Number S3L29PG90000M6089MMG
da21: 600.000MB/s transfers
da21: Command Queueing enabled
da21: 1144641MB (2344225968 512 byte sectors)
mps2:0:18:0): UNMAPPED
(da23:da22 at mps2 bus 0 scbus2 target 18 lun 0
da22: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da22: Serial Number S3L29RHR0000M608STP7
da22: 600.000MB/s transfers
da22: Command Queueing enabled
da22: 1144641MB (2344225968 512 byte sectors)
mps2:0:19:0): UNMAPPED
(da24:da23 at mps2 bus 0 scbus2 target 19 lun 0
da23: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da23: Serial Number S3L29VQT0000M6069ZU8
da23: 600.000MB/s transfers
da23: Command Queueing enabled
da23: 1144641MB (2344225968 512 byte sectors)
mps2:0:20:0): UNMAPPED
(da25:da24 at mps2 bus 0 scbus2 target 20 lun 0
da24: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da24: Serial Number S3L2A7WM0000M608SSU7
da24: 600.000MB/s transfers
da24: Command Queueing enabled
da24: 1144641MB (2344225968 512 byte sectors)
mps2:0:21:0): UNMAPPED
(da26:da25 at mps2 bus 0 scbus2 target 21 lun 0
da25: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da25: Serial Number S3L29GXN0000M605A4C6
da25: 600.000MB/s transfers
da25: Command Queueing enabled
da25: 1144641MB (2344225968 512 byte sectors)
mps2:0:22:0): UNMAPPED
(da27:da26 at mps2 bus 0 scbus2 target 22 lun 0
da26: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da26: Serial Number S3L29TPT0000M608CSAA
da26: 600.000MB/s transfers
da26: Command Queueing enabled
da26: 1144641MB (2344225968 512 byte sectors)
mps2:0:23:0): UNMAPPED
da27 at mps2 bus 0 scbus2 target 23 lun 0
da27: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da27: Serial Number S3L2A4EJ0000M607J095
da27: 600.000MB/s transfers
da27: Command Queueing enabled
da27: 1144641MB (2344225968 512 byte sectors)
SMP: AP CPU #1 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #9 Launched!
SMP: AP CPU #5 Launched!
Timecounter "TSC-low" frequency 1147258217 Hz quality 1000
Trying to mount root from zfs:zroot/ROOT/default []...
GEOM_MIRROR: Device mirror/swap launched (2/2).
GEOM_ELI: Device mirror/swap.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: software
bge0: link state changed to DOWN
bge1: link state changed to DOWN
bge2: link state changed to DOWN
ums0: <Virtual Mouse> on usbus1
bge0: link state changed to UP
bge1: link state changed to UP
bge2: link state changed to UP
ugen1.2: <BMC> at usbus1 (disconnected)
ukbd0: at uhub1, port 1, addr 2 (disconnected)
ums0: at uhub1, port 1, addr 2 (disconnected)
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ESLL_C1S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ESLL_C1S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F8V0_C1S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F8V0_C1S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQPR_C1S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQPR_C1S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M19J9D_C1S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M19J9D_C1S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ES7R_C1S07.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ES7R_C1S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1DXJR_C1S08.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1DXJR_C1S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQHL_C2S01.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQHL_C2S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQSL_C2S02.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQSL_C2S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F7CG_C2S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F7CG_C2S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F2B1_C2S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F2B1_C2S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F9B0_C2S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F9B0_C2S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1ESLL_C1S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ESLL_C1S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ESLL_C1S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ESLL_C1S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1F8V0_C1S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F8V0_C1S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F8V0_C1S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F8V0_C1S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1EQPR_C1S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQPR_C1S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQPR_C1S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQPR_C1S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M19J9D_C1S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M19J9D_C1S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M19J9D_C1S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M19J9D_C1S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1ES7R_C1S07.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ES7R_C1S07...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ES7R_C1S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ES7R_C1S07.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1DXJR_C1S08.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1DXJR_C1S08...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1DXJR_C1S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1DXJR_C1S08.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1EQHL_C2S01.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQHL_C2S01...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQHL_C2S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQHL_C2S01.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1EQSL_C2S02.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQSL_C2S02...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1EQSL_C2S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQSL_C2S02.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1F7CG_C2S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F7CG_C2S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F7CG_C2S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F7CG_C2S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1F2B1_C2S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F2B1_C2S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F2B1_C2S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F2B1_C2S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ER7Y_C2S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S0M1F9B0_C2S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F9B0_C2S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S0M1F9B0_C2S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F9B0_C2S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29R3L_EC1_S01...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29XFQ_EC1_S02...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29QTK_EC1_S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L28ZFA_EC1_S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29PG9_EC1_S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29TAA_EC1_S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29RHR_EC1_S07...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29VQT_EC1_S08...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L2A7WM_EC1_S09...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29GXN_EC1_S10...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L29TPT_EC1_S11...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11.
vdev_geom_open_by_path:744[1]: Found provider by name =
/dev/gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12.
vdev_geom_read_config:429[1]: Reading config from =
gpt/S3L2A4EJ_EC1_S12...
vdev_geom_detach:297[1]: Detaching consumer. Provider =
gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12.
vdev_attach_ok:651[1]: guids match for provider =
/dev/gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12.



console=3D"comconsole,vidconsole"
comconsole_speed=3D"115200"
geom_mirror_load=3D"YES"
kern.geom.label.gptid.enable=3D"0"
kern.geom.label.disk_ident.enable=3D"0"
zfs_load=3D"YES"
kern.cam.scsi_delay=3D15000

# $FreeBSD: releng/11.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux =
$
#
#  This file is read when going to multi-user and its contents piped =
thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for =
details.
#

# Uncomment this to prevent users from seeing information about =
processes that
# are being run under another UID.
#security.bsd.see_other_uids=3D0

kern.ipc.shm_use_phys=3D1
kern.ipc.somaxconn=3D16384
# seems to be higher anyway
#kern.ipc.maxsockets=3D131072
#kern.maxfiles=3D131072
#kern.maxfilesperproc=3D104856
#kern.threads.max_threads_per_proc=3D4096
net.inet.tcp.fast_finwait2_recycle=3D1
net.inet.tcp.finwait2_timeout=3D15000
net.inet.tcp.msl=3D5000
machdep.panic_on_nmi=3D0
net.inet6.ip6.auto_flowlabel=3D0
security.bsd.see_other_gids=3D0
security.bsd.see_other_uids=3D0
# was 49152
net.inet.ip.portrange.hifirst=3D10000

security.bsd.unprivileged_proc_debug=3D0

net.inet.ip.redirect=3D0
net.inet6.ip6.redirect=3D0
net.inet.icmp.drop_redirect=3D1
net.inet6.icmp6.rediraccept=3D0

security.bsd.hardlink_check_uid=3D1
security.bsd.hardlink_check_gid=3D1

kern.coredump=3D0
kern.nodump_coredump=3D1

net.inet.ip.random_id=3D1
net.inet.ip.check_interface=3D1

net.inet.tcp.blackhole=3D1
net.inet.udp.blackhole=3D1

security.bsd.unprivileged_read_msgbuf=3D0


vfs.zfs.min_auto_ashift=3D12
vfs.zfs.arc_max=3D68719476736



I=E2=80=99ve set vfs.zfs.debug to 1.

Does anybody know what the attach/detach messages from GEOM mean?






Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4707908B-4868-4AA6-ADD6-D24121EFAE38>