From owner-freebsd-questions@freebsd.org Fri Dec 6 05:31:17 2019 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC0041BB265 for ; Fri, 6 Dec 2019 05:31:17 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from mx.sdf.org (mx.sdf.org [205.166.94.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Th5b68Mtz42qd for ; Fri, 6 Dec 2019 05:31:15 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from sdf.org (IDENT:bennett@miku.sdf.org [205.166.94.6]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id xB65VD1V010997 (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits) verified NO); Fri, 6 Dec 2019 05:31:13 GMT Received: (from bennett@localhost) by sdf.org (8.15.2/8.12.8/Submit) id xB65VDvq004145; Thu, 5 Dec 2019 23:31:13 -0600 (CST) From: Scott Bennett Message-Id: <201912060531.xB65VDvq004145@sdf.org> Date: Thu, 05 Dec 2019 23:31:13 -0600 To: trond.endrestol@ximalas.info Subject: Re: Moving root on ZFS on GELI so GELI is no longer used Cc: FreeBSD Questions User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47Th5b68Mtz42qd X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of bennett@sdf.org has no SPF policy when checking 205.166.94.20) smtp.mailfrom=bennett@sdf.org X-Spamd-Result: default: False [-0.14 / 15.00]; ARC_NA(0.00)[]; FAKE_REPLY(1.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.877,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(-0.31)[ip: (-0.98), ipnet: 205.166.94.0/24(-0.49), asn: 14361(-0.03), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sdf.org]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.85)[-0.854,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[20.94.166.205.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14361, ipnet:205.166.94.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Dec 2019 05:31:18 -0000 On Thu, 5 Dec 2019 08:21:30 +0100 (CET) Trond Endrest?l wrote: >On Wed, 4 Dec 2019 22:31-0000, Kralj Karlo wrote: I realized too late in my editing of your message that you, Trond, had omitted the OP's email address. :-( > >> I selected the installer option for putting root on ZFS on GELI. >> I want to switch the system so that GELI is not used. (That is, >> the zpool should be on the disk directly.) >> >> I figure I will replicate the zpool (currently a mirror of two GELI >> devices) to the new disk and also do something so that the boot stuff Well, the existing mirror certainly makes this very simple and easy. See further below. >> is on the disk and the boot stuff does not point to the GELI disk. >> The part I don't know how to do is changing the boot stuff. >> >> Are there directions on how to do this? If there is nothing specific, >> could you suggest which manual pages I should read? > >man zfs, searching for zfs snapshot, zfs send, and zfs receive. > >My take usually is: > >1. Make sure boot partitions are present and populated on the new disk(s). > The same for any swap partitions. > The rest of Trond's procedure can be ignored. See further below. >2. Create a new pool. > >3. Create a recursive snapshot from the top of the existing rootpool, e.g. > zfs snap -r rootpool@transfer > >4. Send and receive a replicating stream, e.g. > zfs send -RLe rootpool@transfer | zfs recv -Fduv newpool > > The results of zfs send can be saved as a file and later > transferred using zfs recv, if you prefer this over a pipeline. > >5. Remove the recursive snapshots, e.g. > zfs destroy -Rv rootpool@transfer > zfs destroy -Rv newpool@transfer > >6. Set the bootfs property, if necessary, e.g. > zpool set bootfs=newpool/something newpool > >7. Reboot using newpool. > Because the boot pool is already a mirror, the above procedure is a big waste of time and introduces may extra places for errors to be made. The OP neglected to provide any details of his disk configureation, so I will use my own boot disks as an example that is consistent what he did write. The two drives at ada0 and ada1 are partitioned like this: hellas] 123 % gpart show -l ada0 ada1 => 40 1953525088 ada0 GPT (932G) 40 1024 1 gptboot0 (512K) 1064 984 - free - (492K) 2048 637534208 2 system0 (304G) 637536256 25163776 4 swap0 (12G) 662700032 1064 - free - (532K) 662701096 1048576000 6 local1 (500G) 1711277096 4194304 8 dbtor0 (2.0G) 1715471400 184549376 - free - (88G) 1900020776 52428800 13 crashdump (25G) 1952449576 1075552 - free - (525M) => 40 1953525088 ada1 GPT (932G) 40 1024 1 gptboot1 (512K) 1064 984 - free - (492K) 2048 637534208 2 system1 (304G) 637536256 25163776 4 swap1 (12G) 662700032 1064 - free - (532K) 662701096 1048576000 6 local0 (500G) 1711277096 4194304 8 dbtor1 (2.0G) 1715471400 184549376 - free - (88G) 1900020776 52428800 13 varcrash (25G) 1952449576 1075552 - free - (525M) In this case the only partitions that need concern us are ada{0,1}p{1,2,4}. adap{0,1} contain identical copies of the boot looader code. ada{0,1}p2 are geli-encrypted and contain the boot pool as follows. ada{0,1}p4 are geli- encrypted swapping/paging areas with throwaway keys. hellas] 124 % zpool status system pool: system state: ONLINE scan: scrub repaired 0 in 0 days 00:57:51 with 0 errors on Sat Nov 30 11:44:42 2019 config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0p2.eli ONLINE 0 0 0 (100% initialized, completed at Sun May 5 07:37:22 2019) ada1p2.eli ONLINE 0 0 0 (100% initialized, completed at Sun May 5 07:52:42 2019) errors: No known data error To do what the OP wants to do is very easy, to wit: zpool detach system ada1p2.eli geli stop ada1p2 geli clear -v ada1p2 zpool attach system ada0p2.eli ada1p2 zpool detach system ada0p2.eli geli stop ada0p2 geli clear -v ada0p2 zpool attach system ada1p2 ada0p2 The only issue left over at this point is whether the single sector of each part of the mirror that was freed up by eliminating the geli labels is to be added to the pool's space. If that is considered important and worth doing, then the following sequence should be done. zpool offline -t system ada0p2 zpool online -e system ada0p2 zpool offline -t system ada1p2 zpool online -e system ada1p2 However, because each geli label supposedly occupies only one sector of either 512 bytes or 4096 bytes, depending upon the physical characteristics of the devices, these last four commands may not be worth bothering with. This procedure requires no snapshots, no sends, and no receives. It also seems very likely to require less time to accomplish. It basically copies the pool from one device to the other, then copies it back, and optionally expands the pool by one sector of disk space. Meanwhile, the pool is usable, though slowly, during the entire procedure, and no reboot is necessary. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************