From owner-freebsd-geom@FreeBSD.ORG Wed Sep 3 19:15:35 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21F95EA8 for ; Wed, 3 Sep 2014 19:15:35 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C00D217B6 for ; Wed, 3 Sep 2014 19:15:34 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s83JDwgo055792 for ; Wed, 3 Sep 2014 14:13:58 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Wed Sep 3 14:13:58 2014 Message-ID: <54076871.5010405@denninger.net> Date: Wed, 03 Sep 2014 14:13:53 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Attempt to add multiple device attachment to "geli attach" Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms020807070100050304090902" X-Antivirus: avast! (VPS 140903-0, 09/03/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 19:15:35 -0000 This is a cryptographically signed message in MIME format. --------------ms020807070100050304090902 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable I'm aware of the potential issues here in terms of keying risks, but=20 there are plenty of reasons to support this capability with the largest=20 one being ZFS volumes that you wish to run encrypted. Take the following: label/pool0 label/pool1 label/pool2 label/pool3 (all relative to /dev, of course) These are all gpt partitions on different devices (typically full disks=20 less labels.) You "geli init" them and then attach them and build a=20 raidz2 pool on that. OK, now the system is rebooted. If you use the rc.conf file's option to = request geli passwords during the boot you had better not screw up three = times for only ONE of these volumes or the pool WILL come up degraded! =20 Needless to say that's not nice. It's even worse if it's a raidz pool,=20 you blow it, you reattach that disk and allow it to resilver *and take a = disk error on the remaining drives during the resilver* -- now you're=20 completely hosed. So, here's the idea -- if you use the same password and/or keyfile for=20 ALL of the volumes then either they ALL mount (if you get it right) or=20 NONE of them mount (if you get it wrong.) Now the pool won't import if=20 you get it wrong and you're safe from the risk of taking a forced=20 resilver and potential data loss. The geom subclass command has a simple "nargs" test (must be "1") in the = attach command; I replaced that with "nargs < 1" for the error case. =20 Now I can pass multiple devices to the kernel's geom handler and they=20 get passed to the kernel ctl handler. The following patch should, I believe, work -- but it doesn't. The=20 first disk attaches but the second one that was init'd with the same=20 passphrase fails. As near as I can tell the key components are not picked up off the=20 metadata until the geom driver gets ahold of it -- and thus the second=20 decryption attempt should work since on the second iteration through the = code grabs the key parameters off the request a second time. But I'm obviously missing something because the second volume returns=20 "Wrong key for ...." Ideas? Patch is relative to /usr/src/sys/geom/eli: *** g_eli_ctl.c.orig Wed Sep 3 13:11:52 2014 --- g_eli_ctl.c Wed Sep 3 13:19:15 2014 *************** *** 60,65 **** --- 60,68 ---- int *nargs, *detach, *readonly; int keysize, error; u_int nkey; + char param[16]; + + u_int count; g_topology_assert(); *************** *** 68,74 **** gctl_error(req, "No '%s' argument.", "nargs"); return; } ! if (*nargs !=3D 1) { gctl_error(req, "Invalid number of arguments."); return; } --- 71,77 ---- gctl_error(req, "No '%s' argument.", "nargs"); return; } ! if (*nargs < 1) { gctl_error(req, "Invalid number of arguments."); return; } *************** *** 84,147 **** gctl_error(req, "No '%s' argument.", "readonly"); return; } ! name =3D gctl_get_asciiparam(req, "arg0"); ! if (name =3D=3D NULL) { ! gctl_error(req, "No 'arg%u' argument.", 0); ! return; ! } ! if (strncmp(name, "/dev/", strlen("/dev/")) =3D=3D 0) ! name +=3D strlen("/dev/"); ! pp =3D g_provider_by_name(name); ! if (pp =3D=3D NULL) { ! gctl_error(req, "Provider %s is invalid.", name); ! return; ! } ! error =3D g_eli_read_metadata(mp, pp, &md); ! if (error !=3D 0) { ! gctl_error(req, "Cannot read metadata from %s (error=3D%d).", ! name, error); ! return; ! } ! if (md.md_keys =3D=3D 0x00) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "No valid keys on %s.", pp->name); ! return; ! } ! ! key =3D gctl_get_param(req, "key", &keysize); ! if (key =3D=3D NULL || keysize !=3D G_ELI_USERKEYLEN) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "No '%s' argument.", "key"); ! return; ! } ! ! error =3D g_eli_mkey_decrypt(&md, key, mkey, &nkey); ! bzero(key, keysize); ! if (error =3D=3D -1) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "Wrong key for %s.", pp->name); ! return; ! } else if (error > 0) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "Cannot decrypt Master Key for %s (error=3D%d).= ", ! pp->name, error); ! return; ! } ! G_ELI_DEBUG(1, "Using Master Key %u for %s.", nkey, pp->name); ! ! if (*detach && *readonly) { bzero(&md, sizeof(md)); - gctl_error(req, "Options -d and -r are mutually exclusive."); - return; } - if (*detach) - md.md_flags |=3D G_ELI_FLAG_WO_DETACH; - if (*readonly) - md.md_flags |=3D G_ELI_FLAG_RO; - g_eli_create(req, mp, pp, &md, mkey, nkey); - bzero(mkey, sizeof(mkey)); - bzero(&md, sizeof(md)); } static struct g_eli_softc * --- 87,152 ---- gctl_error(req, "No '%s' argument.", "readonly"); return; } + for (count =3D 0; count < *nargs; count++) { + snprintf(param, sizeof(param), "arg%d", count); + name =3D gctl_get_asciiparam(req, param); + if (name =3D=3D NULL) { + gctl_error(req, "No 'arg%u' argument.", count); + return; + } + if (strncmp(name, "/dev/", strlen("/dev/")) =3D=3D 0) + name +=3D strlen("/dev/"); + pp =3D g_provider_by_name(name); + if (pp =3D=3D NULL) { + gctl_error(req, "Provider %s is invalid.", name); + return; + } + error =3D g_eli_read_metadata(mp, pp, &md); + if (error !=3D 0) { + gctl_error(req, "Cannot read metadata from %s (error=3D%d).= ", + name, error); + return; + } + if (md.md_keys =3D=3D 0x00) { + bzero(&md, sizeof(md)); + gctl_error(req, "No valid keys on %s.", pp->name); + return; + } + + key =3D gctl_get_param(req, "key", &keysize); + if (key =3D=3D NULL || keysize !=3D G_ELI_USERKEYLEN) { + bzero(&md, sizeof(md)); + gctl_error(req, "No '%s' argument.", "key"); + return; + } ! error =3D g_eli_mkey_decrypt(&md, key, mkey, &nkey); ! bzero(key, keysize); ! if (error =3D=3D -1) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "Wrong key for %s.", pp->name); ! return; ! } else if (error > 0) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "Cannot decrypt Master Key for %s=20 (error=3D%d).", ! pp->name, error); ! return; ! } ! G_ELI_DEBUG(1, "Using Master Key %u for %s.", nkey, pp->name); ! ! if (*detach && *readonly) { ! bzero(&md, sizeof(md)); ! gctl_error(req, "Options -d and -r are mutually exclusive."= ); ! return; ! } ! if (*detach) ! md.md_flags |=3D G_ELI_FLAG_WO_DETACH; ! if (*readonly) ! md.md_flags |=3D G_ELI_FLAG_RO; ! g_eli_create(req, mp, pp, &md, mkey, nkey); ! bzero(mkey, sizeof(mkey)); bzero(&md, sizeof(md)); } } static struct g_eli_softc * ------------------------ --=20 -- Karl karl@denninger.net --------------ms020807070100050304090902 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MDMxOTEzNTNaMCMGCSqGSIb3DQEJBDEW BBRo5TI/5OwTSVfaIXgTkF5TYJ4SgTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAaRY+vj1w0w5oF7vPnBRvVsH4a4Nf UZ/S+ua6kEJCoJ7cmzz76oVnp0Nc0huidpnc6+edi8juUDWKluVTGLZnm0LI6EBUm8QoveXV d2NJdW2CDVib4ziQqwlcIUfgkjVV+2aHs11ekxE7TKzwpdVeOmQ5Pso8T2xoWHU+M7blbg1x ozs8v5HI/Em+3rHeulgHJ+PCcsoaTd5g+BD6eI04uwMt9nxulGeotSd23jpEUW1VrII7ZTtn ffOAEyuhWAiOrypiyIX9bQ/1QZThZ6jPgNNqiynz+/WhNzRQ+dhrnyiDJekR9//8yTdlw8wK wTkDclLdqrFznNNf6RVyH81fvUbtSbqgKeYWK27+PWkNjp9WgmUMCgQEDne6OYhObJ+Ud6k8 +/HrYz+RdNzPVD1YJOoQUf2nPkJe80An/506N78H5sOWN+VQxj9JxkYXDRWFwfu7qpIFYvdJ mGvwceHKMEsBcLhFnmq6LaSB873q8nfcrJsjGckGvdyIl0gbGRfwRWaaoQICHz6zcIDz0fO4 4ypjGKt5uUB8rd15UNn6h9zDHKOn3cET4CxwKf+4BYKiP0vUbci0xswXsCw3mugRIQTPqHXm b0TfUGdwHDlCk1cA8m5YooTA5VruybqPaEmQbEa5wSokMxdW7wZr0hzSHh9b109RUzPOyKQL 3rP34KAAAAAAAAA= --------------ms020807070100050304090902-- From owner-freebsd-geom@FreeBSD.ORG Wed Sep 3 19:33:25 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3249F219 for ; Wed, 3 Sep 2014 19:33:25 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D3AF919C2 for ; Wed, 3 Sep 2014 19:33:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s83JXNt5060487 for ; Wed, 3 Sep 2014 14:33:23 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Wed Sep 3 14:33:23 2014 Message-ID: <54076CFE.5010308@denninger.net> Date: Wed, 03 Sep 2014 14:33:18 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Attempt to add multiple device attachment to "geli attach" References: <54076871.5010405@denninger.net> In-Reply-To: <54076871.5010405@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms020901090103060705010404" X-Antivirus: avast! (VPS 140903-0, 09/03/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 19:33:25 -0000 This is a cryptographically signed message in MIME format. --------------ms020901090103060705010404 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Never mind... I know what I missed -- the key generation that is passed=20 in is dependent on the metadata read from the userspace. More work to do here.... will have to pass a separate key structure for=20 each disk and it will also require some more work in the userspace=20 command area so it doesn't prompt a second time for a password. I'll post the completed patch set once I have it if people here think it = would be interesting. On 9/3/2014 14:13, Karl Denninger wrote: > I'm aware of the potential issues here in terms of keying risks, but=20 > there are plenty of reasons to support this capability with the=20 > largest one being ZFS volumes that you wish to run encrypted. > > Take the following: > > label/pool0 > label/pool1 > label/pool2 > label/pool3 > > (all relative to /dev, of course) > > These are all gpt partitions on different devices (typically full=20 > disks less labels.) You "geli init" them and then attach them and=20 > build a raidz2 pool on that. > > OK, now the system is rebooted. If you use the rc.conf file's option=20 > to request geli passwords during the boot you had better not screw up=20 > three times for only ONE of these volumes or the pool WILL come up=20 > degraded! Needless to say that's not nice. It's even worse if it's a = > raidz pool, you blow it, you reattach that disk and allow it to=20 > resilver *and take a disk error on the remaining drives during the=20 > resilver* -- now you're completely hosed. > > So, here's the idea -- if you use the same password and/or keyfile for = > ALL of the volumes then either they ALL mount (if you get it right) or = > NONE of them mount (if you get it wrong.) Now the pool won't import=20 > if you get it wrong and you're safe from the risk of taking a forced=20 > resilver and potential data loss. > > The geom subclass command has a simple "nargs" test (must be "1") in=20 > the attach command; I replaced that with "nargs < 1" for the error=20 > case. Now I can pass multiple devices to the kernel's geom handler=20 > and they get passed to the kernel ctl handler. > > The following patch should, I believe, work -- but it doesn't. The=20 > first disk attaches but the second one that was init'd with the same=20 > passphrase fails. > > As near as I can tell the key components are not picked up off the=20 > metadata until the geom driver gets ahold of it -- and thus the second = > decryption attempt should work since on the second iteration through=20 > the code grabs the key parameters off the request a second time. > > But I'm obviously missing something because the second volume returns=20 > "Wrong key for ...." > > Ideas? > > Patch is relative to /usr/src/sys/geom/eli: > > *** g_eli_ctl.c.orig Wed Sep 3 13:11:52 2014 > --- g_eli_ctl.c Wed Sep 3 13:19:15 2014 > *************** > *** 60,65 **** > --- 60,68 ---- > int *nargs, *detach, *readonly; > int keysize, error; > u_int nkey; > + char param[16]; > + > + u_int count; > > g_topology_assert(); > > *************** > *** 68,74 **** > gctl_error(req, "No '%s' argument.", "nargs"); > return; > } > ! if (*nargs !=3D 1) { > gctl_error(req, "Invalid number of arguments."); > return; > } > --- 71,77 ---- > gctl_error(req, "No '%s' argument.", "nargs"); > return; > } > ! if (*nargs < 1) { > gctl_error(req, "Invalid number of arguments."); > return; > } > *************** > *** 84,147 **** > gctl_error(req, "No '%s' argument.", "readonly"); > return; > } > > ! name =3D gctl_get_asciiparam(req, "arg0"); > ! if (name =3D=3D NULL) { > ! gctl_error(req, "No 'arg%u' argument.", 0); > ! return; > ! } > ! if (strncmp(name, "/dev/", strlen("/dev/")) =3D=3D 0) > ! name +=3D strlen("/dev/"); > ! pp =3D g_provider_by_name(name); > ! if (pp =3D=3D NULL) { > ! gctl_error(req, "Provider %s is invalid.", name); > ! return; > ! } > ! error =3D g_eli_read_metadata(mp, pp, &md); > ! if (error !=3D 0) { > ! gctl_error(req, "Cannot read metadata from %s (error=3D%d).",= > ! name, error); > ! return; > ! } > ! if (md.md_keys =3D=3D 0x00) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "No valid keys on %s.", pp->name); > ! return; > ! } > ! > ! key =3D gctl_get_param(req, "key", &keysize); > ! if (key =3D=3D NULL || keysize !=3D G_ELI_USERKEYLEN) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "No '%s' argument.", "key"); > ! return; > ! } > ! > ! error =3D g_eli_mkey_decrypt(&md, key, mkey, &nkey); > ! bzero(key, keysize); > ! if (error =3D=3D -1) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "Wrong key for %s.", pp->name); > ! return; > ! } else if (error > 0) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "Cannot decrypt Master Key for %s (error=3D%d= ).", > ! pp->name, error); > ! return; > ! } > ! G_ELI_DEBUG(1, "Using Master Key %u for %s.", nkey, pp->name); > ! > ! if (*detach && *readonly) { > bzero(&md, sizeof(md)); > - gctl_error(req, "Options -d and -r are mutually exclusive.");= > - return; > } > - if (*detach) > - md.md_flags |=3D G_ELI_FLAG_WO_DETACH; > - if (*readonly) > - md.md_flags |=3D G_ELI_FLAG_RO; > - g_eli_create(req, mp, pp, &md, mkey, nkey); > - bzero(mkey, sizeof(mkey)); > - bzero(&md, sizeof(md)); > } > > static struct g_eli_softc * > --- 87,152 ---- > gctl_error(req, "No '%s' argument.", "readonly"); > return; > } > + for (count =3D 0; count < *nargs; count++) { > + snprintf(param, sizeof(param), "arg%d", count); > + name =3D gctl_get_asciiparam(req, param); > + if (name =3D=3D NULL) { > + gctl_error(req, "No 'arg%u' argument.", count); > + return; > + } > + if (strncmp(name, "/dev/", strlen("/dev/")) =3D=3D 0) > + name +=3D strlen("/dev/"); > + pp =3D g_provider_by_name(name); > + if (pp =3D=3D NULL) { > + gctl_error(req, "Provider %s is invalid.", name); > + return; > + } > + error =3D g_eli_read_metadata(mp, pp, &md); > + if (error !=3D 0) { > + gctl_error(req, "Cannot read metadata from %s (error=3D%d= ).", > + name, error); > + return; > + } > + if (md.md_keys =3D=3D 0x00) { > + bzero(&md, sizeof(md)); > + gctl_error(req, "No valid keys on %s.", pp->name); > + return; > + } > + > + key =3D gctl_get_param(req, "key", &keysize); > + if (key =3D=3D NULL || keysize !=3D G_ELI_USERKEYLEN) { > + bzero(&md, sizeof(md)); > + gctl_error(req, "No '%s' argument.", "key"); > + return; > + } > > ! error =3D g_eli_mkey_decrypt(&md, key, mkey, &nkey); > ! bzero(key, keysize); > ! if (error =3D=3D -1) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "Wrong key for %s.", pp->name); > ! return; > ! } else if (error > 0) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "Cannot decrypt Master Key for %s=20 > (error=3D%d).", > ! pp->name, error); > ! return; > ! } > ! G_ELI_DEBUG(1, "Using Master Key %u for %s.", nkey, pp->name)= ; > ! > ! if (*detach && *readonly) { > ! bzero(&md, sizeof(md)); > ! gctl_error(req, "Options -d and -r are mutually=20 > exclusive."); > ! return; > ! } > ! if (*detach) > ! md.md_flags |=3D G_ELI_FLAG_WO_DETACH; > ! if (*readonly) > ! md.md_flags |=3D G_ELI_FLAG_RO; > ! g_eli_create(req, mp, pp, &md, mkey, nkey); > ! bzero(mkey, sizeof(mkey)); > bzero(&md, sizeof(md)); > } > } > > static struct g_eli_softc * > > > ------------------------ > --=20 -- Karl karl@denninger.net --------------ms020901090103060705010404 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MDMxOTMzMThaMCMGCSqGSIb3DQEJBDEW BBQZsZ8ZsnAinCIEIb7p7StF64aGZjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIALQFa3eJZZxtMzPFPuRq3fd5Sj78m SGOCuJMrFtxtPiX/IiyqF9UGdfOJ2yGagMw3m+23+Oc4QxKOHr3va2SuqYiBFqh2J9uKX6b+ p1A4YQ7+4by6+it1nXJTg7pcrp/XvEwI3pm7PwnivT47FdAV6/2G+tBM2kQ75XBYZzSHmXx2 YBlNj8LFMAoJE7j3FlmaLwNZXipRZe8C5PbEOKHz/DCYhjd6/kA+yH6z99W4pMe2dyvPAsfY bGb/ahLIW6Jzbvh1JdsC/9jpYgt/r8HTCEhLK5eBX1TwrcH2MS2dtOAcgrNQZnIi4alVfrTj yELdOS3cZz+8kwwG1WZKf+C3S4KOWgzWmx8FTxvNleHZvpFXu/6Z7yfTOJYQ8olR5k1sRArV 7YY50eKWhXD5WGWJ2O2FZA3xGM6NX4OFIXujzOBNFXcil/c0eQjqw+bphuTqoFOFY1WWtHn7 bpxvLLCAph08NJRMq48rZHynshAtev8sTj2zSWdEVrdM6bc0wxeimiqjP5mU1Vw8HvT91Fwo R+7DPcIpVNnJvfAMEjEuW+N2BemgTiiT5cLh4cZnUZiV+RDHjRofmbtgwPdQuf4Tn/bbNnU9 OEaTnbpFDek8B9lwUUTWcAt+lSI095eXvN8aA+8cThfcK7HIpmyxc8MqUjzSnAakyDXi8qhq 2gdREbAAAAAAAAA= --------------ms020901090103060705010404-- From owner-freebsd-geom@FreeBSD.ORG Wed Sep 3 20:00:24 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30753AFD for ; Wed, 3 Sep 2014 20:00:24 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 08EBB1CB6 for ; Wed, 3 Sep 2014 20:00:23 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s83K0EAG082844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Sep 2014 13:00:15 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s83K0E81082843; Wed, 3 Sep 2014 13:00:14 -0700 (PDT) (envelope-from jmg) Date: Wed, 3 Sep 2014 13:00:14 -0700 From: John-Mark Gurney To: Karl Denninger Subject: Re: Attempt to add multiple device attachment to "geli attach" Message-ID: <20140903200014.GB82175@funkthat.com> Mail-Followup-To: Karl Denninger , freebsd-geom@freebsd.org References: <54076871.5010405@denninger.net> <54076CFE.5010308@denninger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54076CFE.5010308@denninger.net> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 03 Sep 2014 13:00:15 -0700 (PDT) Cc: freebsd-geom@freebsd.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 20:00:24 -0000 Karl Denninger wrote this message on Wed, Sep 03, 2014 at 14:33 -0500: > Never mind... I know what I missed -- the key generation that is passed > in is dependent on the metadata read from the userspace. > > More work to do here.... will have to pass a separate key structure for > each disk and it will also require some more work in the userspace > command area so it doesn't prompt a second time for a password. > > I'll post the completed patch set once I have it if people here think it > would be interesting. Just some comments on this as I've thought about this issue... There are two issues here, one is for root and one is for geli volume mounted later... For the later, I personally use a key volume that is encrypted, and uses that "key store" for my large 8 disk raidz pool.. This is less of an issue, but still requires me to type in the password twice... It basicly boils down to: (cd /zkeys && for i in *.key; do geli attach -p -k "$i" "label/${i%.key}"; geli attach -p -k "$i" "gpt/${i%.key}"; done) || exit 5 I have to do both label and gpt since disks are labeled, but things like zlog are on gpt partitions... I haven't reviewed your patch, nor have I looked at how geli keys volumes upon init, but make sure that you have each volume's master key salted seperately... This way if the volumes get seperated from your system, it won't leak that they use the same key... Yes, it'll take a bit more cpu time to unlock, but not that big of an issue IMO... Handling unlocking mirrored roots is a bit more interesting as you now have to touch the geli kernel code... btw, reattaching a single disk that was previously part of a pool is fast... I've done this on more than one occasion where one disk drops out of the raidz and then shortly after I reattach it... It will recognize the original data, so only if new data that got written can't be read will you suffer a loss, but that would be a double failure case, and known limitation of raidz... Thanks for looking at this... I'm definately interested in making multi disk geli more usable... $find /dev -name "*.eli" | wc -l 17 :) 8 (raidz data disks) + 2 (mirrored root) + 1 (swap) + 2 (cache) + 2 (log) + 2 (duplicates from root ada vs ad) > On 9/3/2014 14:13, Karl Denninger wrote: > >I'm aware of the potential issues here in terms of keying risks, but > >there are plenty of reasons to support this capability with the > >largest one being ZFS volumes that you wish to run encrypted. > > > >Take the following: > > > >label/pool0 > >label/pool1 > >label/pool2 > >label/pool3 > > > >(all relative to /dev, of course) > > > >These are all gpt partitions on different devices (typically full > >disks less labels.) You "geli init" them and then attach them and > >build a raidz2 pool on that. > > > >OK, now the system is rebooted. If you use the rc.conf file's option > >to request geli passwords during the boot you had better not screw up > >three times for only ONE of these volumes or the pool WILL come up > >degraded! Needless to say that's not nice. It's even worse if it's a > >raidz pool, you blow it, you reattach that disk and allow it to > >resilver *and take a disk error on the remaining drives during the > >resilver* -- now you're completely hosed. > > > >So, here's the idea -- if you use the same password and/or keyfile for > >ALL of the volumes then either they ALL mount (if you get it right) or > >NONE of them mount (if you get it wrong.) Now the pool won't import > >if you get it wrong and you're safe from the risk of taking a forced > >resilver and potential data loss. > > > >The geom subclass command has a simple "nargs" test (must be "1") in > >the attach command; I replaced that with "nargs < 1" for the error > >case. Now I can pass multiple devices to the kernel's geom handler > >and they get passed to the kernel ctl handler. > > > >The following patch should, I believe, work -- but it doesn't. The > >first disk attaches but the second one that was init'd with the same > >passphrase fails. > > > >As near as I can tell the key components are not picked up off the > >metadata until the geom driver gets ahold of it -- and thus the second > >decryption attempt should work since on the second iteration through > >the code grabs the key parameters off the request a second time. > > > >But I'm obviously missing something because the second volume returns > >"Wrong key for ...." > > > >Ideas? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-geom@FreeBSD.ORG Wed Sep 3 20:30:36 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51FC9A98 for ; Wed, 3 Sep 2014 20:30:36 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A8DE1004 for ; Wed, 3 Sep 2014 20:30:35 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s83KUVkC074013 for ; Wed, 3 Sep 2014 15:30:31 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Wed Sep 3 15:30:31 2014 Message-ID: <54077A62.50606@denninger.net> Date: Wed, 03 Sep 2014 15:30:26 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Attempt to add multiple device attachment to "geli attach" References: <54076871.5010405@denninger.net> <54076CFE.5010308@denninger.net> <20140903200014.GB82175@funkthat.com> In-Reply-To: <20140903200014.GB82175@funkthat.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms030204030709010703090103" X-Antivirus: avast! (VPS 140903-0, 09/03/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 20:30:36 -0000 This is a cryptographically signed message in MIME format. --------------ms030204030709010703090103 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 9/3/2014 15:00, John-Mark Gurney wrote: > Karl Denninger wrote this message on Wed, Sep 03, 2014 at 14:33 -0500: >> Never mind... I know what I missed -- the key generation that is passe= d >> in is dependent on the metadata read from the userspace. >> >> More work to do here.... will have to pass a separate key structure fo= r >> each disk and it will also require some more work in the userspace >> command area so it doesn't prompt a second time for a password. >> >> I'll post the completed patch set once I have it if people here think = it >> would be interesting. > Just some comments on this as I've thought about this issue... > > There are two issues here, one is for root and one is for geli > volume mounted later... > > For the later, I personally use a key volume that is encrypted, and use= s > that "key store" for my large 8 disk raidz pool.. This is less of an > issue, but still requires me to type in the password twice... It > basicly boils down to: > (cd /zkeys && for i in *.key; do geli attach -p -k "$i" "label/${i%.key= }"; geli attach -p -k "$i" "gpt/${i%.key}"; done) || exit 5 > > I have to do both label and gpt since disks are labeled, but things lik= e > zlog are on gpt partitions... > > I haven't reviewed your patch, nor have I looked at how geli keys > volumes upon init, but make sure that you have each volume's master > key salted seperately... This way if the volumes get seperated from > your system, it won't leak that they use the same key... Yes, it'll > take a bit more cpu time to unlock, but not that big of an issue IMO...= Yeah, that's basically the issue. The userspace code (the geli link to /sbin/geom) does that; it grabs the = metadata and uses it as part of the salting, so the key passed to the=20 kernel is unique even if the passphrase is identical. That's why the=20 second pack attach fails the way I tried to do this. This in turn means that the pass to the kernel driver has to have a=20 unique key for each argument (pack) you pass for attachment. Right now=20 the code passes the parameter "key" in the request structure; this will=20 have to be changed to be "key0", "key1", etc. That in turn means that if you pass more than one argument to "geli=20 attach" the userland code has to ask for the password for the first=20 argument but not ask for each subsequent one. That means the userland=20 code has to save the password typed so if the "get the password" routine = is called again during the same invocation (as the code iterates over=20 the listed devices) instead of prompting it just picks up the saved=20 value, iterates over each volume and stuffs the appropriate key=20 structure into the parameter list. I have to put some thought into=20 that though to minimize the risk of that keyed password leaking...... I'm going to noodle on this a bit.... I think I can get where I want to=20 go without adding more risk than the obvious (e.g. if someone gets the=20 password they now have the key to unlock all the volumes involved, but=20 if you're not using unique passwords for each that's already a risk=20 you're accepting.) --=20 -- Karl karl@denninger.net --------------ms030204030709010703090103 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MDMyMDMwMjZaMCMGCSqGSIb3DQEJBDEW BBSb8PBNbY/nrj/e2OPO4vx/nPCQcDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAhI/bhHzeFKp+ViNKxLQZLSW3MZRJ /SMA5otsjMkPAPI+FaVVjF1YcgIcbCQ/bI9RDUMxPcUGsMwU8huvBqr15DKAmEDJkIX032A3 VrBqxs4t1n/hFhLPzX8qkz+m9yFOZXROcGne0xzNA1MRQIzEOOi1AJWKeN0s2m/i5jwrDUQg IZX4oNxm3irDpeYsPXbMKzA6sQOeN55cfM5mfsI6TVd0m0SbUscElEdKJSDqYkXFNurNTS2h RpwaUnc57tEsoeodlCNrOijRMBdjuuEr3eZP2izGVCw1LlexXpFUAMQYxUbTgNGYFN37Y5qn MEnoTODabkW3DTQEA+osFbn5NLhl+/AIA5QozrVXQfPmOWwg6MXHsbPC5ACxfBNor7+88siU NWl8VeDduftmp+nbU6vIO8UgmOl1EYw82W5BtyQAtSYl9ECfsnYqev2vAYVBSfAS/oedqC6s vBggtkgmzM+0AL8UUtCTgIo46xn6QL8IsKLkbo/3BJEZuoQIkwfr/1WyoCATJrTC0Qm5yynE W4wJPV9EkJdoSMwIiJgWRHsfL67We9UmTjI+gjPozpJGq5m7NVLUsr85NvAJwcpDax3fQ9cE 9u5BziEnLPeiOv/je+sS5uV6YvD7ajdmt76Sa0GGNeakw2Ofy/JJQ3Ar0ABAUEq3MCvhdve9 Id8JC5EAAAAAAAA= --------------ms030204030709010703090103-- From owner-freebsd-geom@FreeBSD.ORG Thu Sep 4 02:44:59 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F0FBBFA for ; Thu, 4 Sep 2014 02:44:59 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CBE1D1C8E for ; Thu, 4 Sep 2014 02:44:57 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s842isHk054700 for ; Wed, 3 Sep 2014 21:44:54 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Wed Sep 3 21:44:54 2014 Message-ID: <5407D221.5000609@denninger.net> Date: Wed, 03 Sep 2014 21:44:49 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Attempt to add multiple device attachment to "geli attach" References: <54076871.5010405@denninger.net> <54076CFE.5010308@denninger.net> <20140903200014.GB82175@funkthat.com> In-Reply-To: <20140903200014.GB82175@funkthat.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms080302030507010700050906" X-Antivirus: avast! (VPS 140903-0, 09/03/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Sep 2014 02:44:59 -0000 This is a cryptographically signed message in MIME format. --------------ms080302030507010700050906 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 9/3/2014 15:00, John-Mark Gurney wrote: > Karl Denninger wrote this message on Wed, Sep 03, 2014 at 14:33 -0500: >> Never mind... I know what I missed -- the key generation that is passe= d >> in is dependent on the metadata read from the userspace. >> >> More work to do here.... will have to pass a separate key structure fo= r >> each disk and it will also require some more work in the userspace >> command area so it doesn't prompt a second time for a password. >> >> I'll post the completed patch set once I have it if people here think = it >> would be interesting. > Just some comments on this as I've thought about this issue... > > There are two issues here, one is for root and one is for geli > volume mounted later... > > For the later, I personally use a key volume that is encrypted, and use= s > that "key store" for my large 8 disk raidz pool.. This is less of an > issue, but still requires me to type in the password twice... It > basicly boils down to: > (cd /zkeys && for i in *.key; do geli attach -p -k "$i" "label/${i%.key= }"; geli attach -p -k "$i" "gpt/${i%.key}"; done) || exit 5 > > I have to do both label and gpt since disks are labeled, but things lik= e > zlog are on gpt partitions... > > I haven't reviewed your patch, nor have I looked at how geli keys > volumes upon init, but make sure that you have each volume's master > key salted seperately... This way if the volumes get seperated from > your system, it won't leak that they use the same key... Yes, it'll > take a bit more cpu time to unlock, but not that big of an issue IMO...= > > Handling unlocking mirrored roots is a bit more interesting as you > now have to touch the geli kernel code... > > btw, reattaching a single disk that was previously part of a pool is > fast... I've done this on more than one occasion where one disk drops > out of the raidz and then shortly after I reattach it... It will > recognize the original data, so only if new data that got written > can't be read will you suffer a loss, but that would be a double failur= e > case, and known limitation of raidz... > > Thanks for looking at this... I'm definately interested in making > multi disk geli more usable... > > $find /dev -name "*.eli" | wc -l > 17 > > :) > > 8 (raidz data disks) + 2 (mirrored root) + 1 (swap) + 2 (cache) + > 2 (log) + 2 (duplicates from root ada vs ad) > Try this in /usr/local/etc/rc.d -- it is a modification of the geli=20 script and gets the password, then iterates over the disks and tries to=20 attach them. If it fails it will prompt you again (up to three times as = does the stock code, but you can override that if you want.) This is to = be used in place of the geli option in /etc/rc.conf. Place the disks in /etc/rc.conf as: encrypt_disks=3D"..... " The usual geli overrides also work (since I cribbed the code), EXCEPT=20 the detach-on-close -- I have had serious problems with that when a=20 non-related drive detaches from the bus -- it has on multiple occasions=20 caused all my geli disks to detach on the same adapter! Needless to say=20 I don't set that flag any more -- I let the kernel detach them when the=20 machine shuts down. As long as the password you originally supply is good it will keep=20 iterating through the list and mount them all. Voila -- enter it once! #!/bin/sh # # Copyright 2014 Karl Denninger # Cribbed modified from original as below # # Copyright (c) 2005 Pawel Jakub Dawidek # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution.= # # THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND= # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABL= E # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS= # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ # # PROVIDE: disks # REQUIRE: initrandom # KEYWORD: nojail =2E /etc/rc.subr name=3D"encrypt" start_cmd=3D"encrypt_start" stop_cmd=3D"encrypt_stop" required_modules=3D"geom_eli:g_eli" encrypt_start() { devices=3D${encrypt_disks} echo -n 'Geli attach Password: ' stty -echo read password stty echo echo if [ -z "${encrypt_tries}" ]; then if [ -n "${encrypt_attach_attempts}" ]; then # Compatibility with rc.d/gbde. encrypt_tries=3D${encrypt_attach_attempts} else encrypt_tries=3D`${SYSCTL_N} kern.geom.eli.tries= ` fi fi for provider in ${devices}; do provider_=3D`ltr ${provider} '/-' '_'` eval "flags=3D\${encrypt_${provider_}_flags}" if [ -z "${flags}" ]; then flags=3D${encrypt_default_flags} fi if [ -e "/dev/${provider}" -a ! -e "/dev/${provider}.eli" ]; the= n echo "Geli attach ${provider}." count=3D1 while [ ${count} -le ${encrypt_tries} ]; do echo $password | geli attach -j - ${flags} ${provider} if [ -e "/dev/${provider}.eli" ]; then break fi echo "Attach failed; attempt ${count} of ${encrypt_tries= }." count=3D$((count+1)) if [ ${count} -gt ${encrypt_tries} ]; then echo "KEY MISMATCH ERROR - Abort" exit 1 fi echo -n 'Geli attach Password: ' stty -echo read password stty echo echo done else if [ -e "/dev/${provider}" ]; then echo "${provider} is already attached." else echo "${provider} does not exist." fi fi done } encrypt_stop() { devices=3D${encrypt_disks} for provider in ${devices}; do if [ -e "/dev/${provider}.eli" ]; then umount "/dev/${provider}.eli" 2>/dev/null geli detach "${provider}" fi done } load_rc_config $name run_rc_command "$1" --=20 -- Karl karl@denninger.net --------------ms080302030507010700050906 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA5MDQwMjQ0NDlaMCMGCSqGSIb3DQEJBDEW BBSvKticq39C5sphylyhv8EKbrUl1jBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIArm1/g/DAM+dUD9EgdCnKQHXvDg/F dWFcXD4P0ei+SZJaeE9iqarT7NdWzBiFtqeA+cFqzCT2R2QkUwhNjE1ejvinK4LQO+32LDl8 klHnxtfKgo13jkFVxLyn1vQ8S+EzL5yPKf/eoT50VsfOrWGrL7LzEEdk7Tlsbj8MxhqqM7S3 +8Kj+0Tjjcs8iXnECJU1W5e3aKsnqv9aNDL2UXDNHDzkc7Zd2QvQwvWAxJlDFwYFijrpSzZ7 5vZ2tCgaLAo1+DDPPXKZ0PHj2/jvjLlP8PXDEuXLwQwR0nNhhAkT6uN6hx/hL1e8dGuc4eO6 RiZP+Nc3k83H12jJ38Z4TtxKr4dfLp/x85TQ/T85cHplrkz+5fyCyGnXJBn0ZniHeoOYbfWE GR9lmlmTzcTJ+9ZwMaLBJqGa5RI1ZU8lAoRCKB6DGVXxtaabadFX9OT8kI/WwFxHcuK25tGh mq+xj1CkoYbSWhrl+z9TFNmMlpmRTzZb2e+75nQsgNdC3McmR2+LPtoKoXd4InKoqahOU2KI GME7EHTWVfnJdNJSjUusC2DaCV0nD1r8YS23mYgksg1k8VgRJVj6Hj2IYI5VHB4IGxLk0Qd9 gpNnSYZlZzGwDKj4P3lEEul/uDNEtihu0NVgajJpFnk6g0DHMYHb0DAQMg1dIIABvUqDx12n i+p4BWkAAAAAAAA= --------------ms080302030507010700050906-- From owner-freebsd-geom@FreeBSD.ORG Thu Sep 4 08:53:11 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C980D4F5 for ; Thu, 4 Sep 2014 08:53:11 +0000 (UTC) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [IPv6:2a02:b90:3002:e550::3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91A861833 for ; Thu, 4 Sep 2014 08:53:11 +0000 (UTC) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1XPSmy-0007vL-0p; Thu, 04 Sep 2014 08:53:08 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XPSmx-0005PO-Uk; Thu, 04 Sep 2014 09:53:07 +0100 To: freebsd-geom@freebsd.org, karl@denninger.net Subject: Re: Attempt to add multiple device attachment to "geli attach" In-Reply-To: <54076871.5010405@denninger.net> Message-Id: From: Pete French Date: Thu, 04 Sep 2014 09:53:07 +0100 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Sep 2014 08:53:11 -0000 > Take the following: > > label/pool0 > label/pool1 > label/pool2 > label/pool3 > > (all relative to /dev, of course) > > These are all gpt partitions on different devices (typically full disks=20 > less labels.) You "geli init" them and then attach them and build a=20 > raidz2 pool on that. > > OK, now the system is rebooted. If you use the rc.conf file's option to > request geli passwords during the boot you had better not screw up three > times for only ONE of these volumes or the pool WILL come up degraded! I hit this ne when I used to run ZFs on top of geli. The solution I came up with was to have a tiny partition which requested the password, and then for that devide to be used as the key file for all the others. That way the password is only requested once, but decrypts all drives if successful. We ran that way for a long tme and it worked well. Irrelevent to the oatch of course, which is a good thing, but just pounting ut one way round it in practice using the current setup. -pete.