Date: Mon, 25 Apr 2016 10:47:23 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com> To: Marcin Cieslak <saper@saper.info> Cc: <freebsd-virtualization@freebsd.org> Subject: Re: Booting r298488 as Xen Dom0 may break ZFS pool? Message-ID: <20160425084714.64ejioyqgquwu4gw@mac> In-Reply-To: <alpine.BSF.2.11.1604232216330.1768@z.fncre.vasb> References: <alpine.BSF.2.11.1604232216330.1768@z.fncre.vasb>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Apr 23, 2016 at 10:40:20PM +0000, Marcin Cieslak wrote: > On a freshly installed (via upgrade from 10.3 from source) > -CURRENT on this machine: > > FreeBSD 11.0-CURRENT #0 r298488: Sat Apr 23 11:10:01 UTC 2016 > root@o.saper.info:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) > WARNING: WITNESS option enabled, expect reduced performance. > VT(vga): resolution 640x480 > CPU: Intel(R) Xeon(R) CPU E31245 @ 3.30GHz (3300.09-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x206a7 Family=0x6 Model=0x2a Stepping=7 > Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > Features2=0x1fbae3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX> > AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > XSAVE Features=0x1<XSAVEOPT> > VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > TSC: P-state invariant, performance statistics > real memory = 17179869184 (16384 MB) > avail memory = 16475140096 (15711 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: <ALASKA A M I> > FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs > FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads > random: unblocking device. > ioapic0 <Version 2.0> irqs 0-23 on motherboard > > ... i am trying to boot the system as Dom0 under Xen (4.5.2_2 > installed via pkg install). > > The kernel boots in one of four cases, mostly > though I don't get a block cursor after Xen messages > and the machine sits and waits. > > > After trying Xen suddenly the system no longer > boots again: > > Booting from local disk... > PXE-M0F: Existing Intel Boot Agent. > > ZFS: i/o error - all block copies unavailable > ZFS: i/o error - all block copies unavailable > ZFS: i/o error - all block copies unavailable > > Can't find /boot/zfsloader > > FreeBSD/x86 boot > Default: zroot:/boot/kernel/kernel > boot: > ZFS: i/o error - all block copies unavailable I'm currently running a r298464 world+kernel with ZFS on root and have seen no issues so far. > (...) > > The zpool imports without problems when > booting from the rescue mfsbsd (10.3): > > pool: zroot > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > ada0p3 ONLINE 0 0 0 > ada1p3 ONLINE 0 0 0 > > errors: No known data errors > > gpart layout: > > => 34 5860533101 ada0 GPT (2.7T) > 34 1024 1 freebsd-boot (512K) > 1058 4194304 2 freebsd-swap (2.0G) > 4195362 5856337773 3 freebsd-zfs (2.7T) > > => 34 5860533101 ada1 GPT (2.7T) > 34 1024 1 freebsd-boot (512K) > 1058 4194304 2 freebsd-swap (2.0G) > 4195362 5856337773 3 freebsd-zfs (2.7T) > > I have managed to make zpool boot again by doing voodoo > similar to this one: > > [root@rescue ~]# zpool import -R /mnt zroot > [root@rescue ~]# mount -t devfs devfs /mnt/dev > [root@rescue ~]# chroot /mnt /bin/tcsh > > (... Running make install in /usr/src/sys/boot ...) > > root@rescue:/ # gpart bootcode -p /boot/gptzfsboot -i 1 ada0 > partcode written to ada0p1 > root@rescue:/ # gpart bootcode -p /boot/gptzfsboot -i 1 ada1 > partcode written to ada1p1 > root@rescue:/ # exit > [root@rescue ~]# umount /mnt/dev > [root@rescue ~]# zpool export zroot > [root@rescue ~]# reboot > > Why zpool metadata get corrupted? TBH, I have no idea. Can you also paste the log of the system (Xen + FreeBSD) when it fails to boot? If that's not possible, can you at least paste the output of `xl dmesg` when booted correctly under Xen? What operations did you perform when the system booted correctly using FreeBSD/Xen? Does the disk get corrupted even if the system fails to boot? AFAICT, it seems like it's only the bootcode that gets corrupted, is that right? Roger.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160425084714.64ejioyqgquwu4gw>