From owner-freebsd-fs@FreeBSD.ORG Sun May 18 10:25:55 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B64FF2D7 for ; Sun, 18 May 2014 10:25:55 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 2A6A12DE8 for ; Sun, 18 May 2014 10:25:54 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id B314C20E7088B; Sun, 18 May 2014 10:25:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 6F7BF20E70886; Sun, 18 May 2014 10:25:42 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Jeff Chan" , References: <1129127016.20140517185154@supranet.net> <1198503903.20140518030907@supranet.net> Subject: Re: ZFS snapshot restore not quite working; missing steps? Date: Sun, 18 May 2014 11:25:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 10:25:55 -0000 Not used legacy here have you tried: zfs set mountpoint=/ zroot Silly question given your using mfi have confirmed your controller / machine bios are set to boot from the relavent disk? Also have you confirmed zfs in /boot/loader.conf e.g. zfs_load="YES" vfs.root.mountfrom="zfs:zroot" Finally do be aware its not ideal to use a RAID controller for ZFS your better of with a HBA which does less "fancy" stuff when communicating with the disks allowing ZFS do see whats really going on. Regards Steve ----- Original Message ----- From: "Jeff Chan" To: Sent: Sunday, May 18, 2014 11:09 AM Subject: Re: ZFS snapshot restore not quite working; missing steps? > On Saturday, May 17, 2014, 6:51:54 PM, Jeff Chan wrote: >> We're trying to do a ZFS snapshot restore of a ZFS non-RAIDZ* FreeBSD >> 9.2-RELEASE to a new bare system configured with ZFS RAIDZ, and it's not >> quite working right. The restore itself seems to complete, but we're >> not able to successfully boot the resulting system. We can't recall >> the exact error we got, but think the ZFS cache file was not found. > >> Presumably we're missing some steps, don't have all magic for ZFS >> booting from a FreeBSD root partition installed/configured correctly, >> etc. What did we miss? > >> (We have a private network to serve the snapshot over NFS from a third >> server on 192.168.0.2.) > > >> (There may be some line wrap below.) > >> (Comments in (parens)) > >> (Some outputs in [brackets]) > > > > >> On the old system, call it foo: > > >> foo: [103]% zfs list >> NAME USED AVAIL REFER MOUNTPOINT >> zroot 169G 3.40T 545M legacy >> zroot/home 15.1G 3.40T 10.3G /home >> zroot/home/username 4.04G 3.40T 2.74G /home/username >> zroot/bar 128G 3.40T 97K /bar >> zroot/bar/prod 126G 3.40T 73.8G /bar/prod >> zroot/bar/test 2.18G 3.40T 2.18G /bar/test >> zroot/tmp 1.35G 3.40T 1.35G /tmp >> zroot/usr 9.60G 3.40T 9.52G /usr >> zroot/var 14.0G 3.40T 14.0G /var >> foo: [104]% zpool list >> NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT >> zroot 3.62T 170G 3.46T 4% 1.00x ONLINE - > > > >> On the new system: > > >> Boot from a FreeBSD 9.2-RELEASE installer USB flash drive >> into LiveCD, then: > >> (bring up NFS private LAN) >> ifconfig ix3 192.168.0.3 netmask 255.255.255.0 up >> [ix2: Could not setup receive structures >> [ix2: Could not setup receive structures > ->> >> sysctl kern.ipc.nmbclusters=131072 >> sysctl kern.ipc.nmbjumbo9=38400 > >> (mount remote NFS backup directory) >> mount -t nfs 192.168.0.2:/home/backup /mnt >> mkdir /var/mnt > >> sysctl kern.disks >> [kern.disks: da0 mfid2 mfid1 mfid0] > >> gpart create -s gpt mfid0 >> gpart create -s gpt mfid1 >> gpart create -s gpt mfid2 >> gpart add -s 222 -a 4k -t freebsd-boot -l boot0 mfid0 >> gpart add -s 222 -a 4k -t freebsd-boot -l boot1 mfid1 >> gpart add -s 222 -a 4k -t freebsd-boot -l boot2 mfid2 >> gpart add -s 8g -a 4k -t freebsd-swap -l swap0 mfid0 >> gpart add -s 8g -a 4k -t freebsd-swap -l swap1 mfid1 >> gpart add -s 8g -a 4k -t freebsd-swap -l swap2 mfid2 >> gpart add -a 4k -t freebsd-zfs -l disk0 mfid0 >> gpart add -a 4k -t freebsd-zfs -l disk1 mfid1 >> gpart add -a 4k -t freebsd-zfs -l disk2 mfid2 >> (Clear any old zfs data) >> dd if=/dev/zero of=/dev/mfid0p3 count=560 bs=512 >> dd if=/dev/zero of=/dev/mfid1p3 count=560 bs=512 >> dd if=/dev/zero of=/dev/mfid2p3 count=560 bs=512 >> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid0 >> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid1 >> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid2 >> zpool create -f -m none -o altroot=/var/mnt -o >> cachefile=/tmp/zpool.cache zroot raidz mfid0 mfid1 mfid2 > >> gunzip -c /mnt/foo-backup-full/zroot@20140516183425.gz | zfs receive -vdFu zroot >> zpool set bootfs=zroot zroot >> zpool get all zroot >> zfs get all zroot > >> zfs set mountpoint=/var/mnt zroot >> (add swap to fstab) >> vi /var/mnt/var/mnt/etc/fstab >> [/dev/mfid0p2 none swap sw 0 0 >> [/dev/mfid1p2 none swap sw 0 0 >> [/dev/mfid2p2 none swap sw 0 0 > >> vi /var/mnt/var/mnt/etc/rc.conf [ change adapter name & IP ] >> vi /var/mnt/var/mnt/etc/pf.conf [ change adapter name ] >> [vi /var/mnt/var/mnt/boot/loader.conf] (not performed) >> cd / >> zpool export zroot >> zpool import -o altroot=/var/mnt -o cachefile=/tmp/zpool.cache zroot >> [zfs set mountpoint=/ zroot] (not performed) >> cp /tmp/zpool.cache /var/mnt/boot/zfs >> zpool get all zroot >> zpool set cachefile=/boot/zfs/zpool.cache zroot >> zfs unmount -a >> zfs set mountpoint=legacy zroot >> zfs set mountpoint=/home zroot/home >> zfs set mountpoint=/bar zroot/bar >> zfs set mountpoint=/tmp zroot/tmp >> zfs set mountpoint=/usr zroot/usr >> zfs set mountpoint=/var zroot/var > >> [remove USB] >> reboot > > > > Additional information, gpart list on the new system: > > > root@:~ # gpart list > Geom name: da0 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 7821311 > first: 0 > entries: 8 > scheme: BSD > Providers: > 1. Name: da0a > Mediasize: 734167040 (700M) > Sectorsize: 512 > Mode: r1w0e1 > rawtype: 7 > length: 734167040 > offset: 0 > type: freebsd-ufs > index: 1 > end: 1433919 > start: 0 > Consumers: > 1. Name: da0 > Mediasize: 4004511744 (3.7G) > Sectorsize: 512 > Mode: r1w0e1 > > Geom name: mfid0 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 5859373022 > first: 34 > entries: 128 > scheme: GPT > Providers: > 1. Name: mfid0p1 > Mediasize: 110592 (108k) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 20480 > Mode: r0w0e0 > rawuuid: fe8923b4-ddd6-11e3-8d53-a0369f312ea0 > rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f > label: boot0 > length: 110592 > offset: 20480 > type: freebsd-boot > index: 1 > end: 255 > start: 40 > 2. Name: mfid0p2 > Mediasize: 8589934592 (8.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 0863e7f4-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b > label: swap0 > length: 8589934592 > offset: 131072 > type: freebsd-swap > index: 2 > end: 16777471 > start: 256 > 3. Name: mfid0p3 > Mediasize: 2991408918528 (2.7T) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 08a7dd63-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b > label: disk0 > length: 2991408918528 > offset: 8590065664 > type: freebsd-zfs > index: 3 > end: 5859373015 > start: 16777472 > Consumers: > 1. Name: mfid0 > Mediasize: 2999999004672 (2.7T) > Sectorsize: 512 > Mode: r0w0e0 > > Geom name: mfid1 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 5859373022 > first: 34 > entries: 128 > scheme: GPT > Providers: > 1. Name: mfid1p1 > Mediasize: 110592 (108k) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 20480 > Mode: r0w0e0 > rawuuid: 02b8c77a-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f > label: boot1 > length: 110592 > offset: 20480 > type: freebsd-boot > index: 1 > end: 255 > start: 40 > 2. Name: mfid1p2 > Mediasize: 8589934592 (8.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 086d2831-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b > label: swap1 > length: 8589934592 > offset: 131072 > type: freebsd-swap > index: 2 > end: 16777471 > start: 256 > 3. Name: mfid1p3 > Mediasize: 2991408918528 (2.7T) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 08b854b6-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b > label: disk1 > length: 2991408918528 > offset: 8590065664 > type: freebsd-zfs > index: 3 > end: 5859373015 > start: 16777472 > Consumers: > 1. Name: mfid1 > Mediasize: 2999999004672 (2.7T) > Sectorsize: 512 > Mode: r0w0e0 > > Geom name: mfid2 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 5859373022 > first: 34 > entries: 128 > scheme: GPT > Providers: > 1. Name: mfid2p1 > Mediasize: 110592 (108k) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 20480 > Mode: r0w0e0 > rawuuid: 03c0b756-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f > label: boot2 > length: 110592 > offset: 20480 > type: freebsd-boot > index: 1 > end: 255 > start: 40 > 2. Name: mfid2p2 > Mediasize: 8589934592 (8.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 087d8c31-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b > label: swap2 > length: 8589934592 > offset: 131072 > type: freebsd-swap > index: 2 > end: 16777471 > start: 256 > 3. Name: mfid2p3 > Mediasize: 2991408918528 (2.7T) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 131072 > Mode: r0w0e0 > rawuuid: 08c61aed-ddd7-11e3-8d53-a0369f312ea0 > rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b > label: disk2 > length: 2991408918528 > offset: 8590065664 > type: freebsd-zfs > index: 3 > end: 5859373015 > start: 16777472 > Consumers: > 1. Name: mfid2 > Mediasize: 2999999004672 (2.7T) > Sectorsize: 512 > Mode: r0w0e0 > > > > > -- > Jeff Chan > mailto:jeffc@supranet.net > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >