Date: Thu, 22 Jan 2009 13:24:17 -0800 From: David Ehrmann <ehrmann@gmail.com> To: Pete French <petefrench@ticketswitch.com> Cc: freebsd-stable@freebsd.org Subject: Re: zfs drive keeps failing between export and import Message-ID: <6e0e5340901221324o33f1e2b1l53c842ebf9dad9a8@mail.gmail.com> In-Reply-To: <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com> References: <6e0e5340901151158n5108ba8ct6af8fb270b10b75b@mail.gmail.com> <E1LNmxP-0003vM-Lx@dilbert.ticketswitch.com> <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jan 16, 2009 at 3:21 PM, David Ehrmann <ehrmann@gmail.com> wrote: > On Fri, Jan 16, 2009 at 3:33 AM, Pete French > <petefrench@ticketswitch.com> wrote: >>> a software problem before hardware. Both drives are encrypted geli >>> devices. I tried to reproduce the error with 1GB disk images (vs >> >> This is probably a silly question, but are you sure that the drives >> are not auto detaching ? I had big problems with a zfs mirror on top >> of geli which turned out to be that drives mounted using "geli_devices" >> in rc.conf will auto detach unless you set "geli_autodetach" to NO. > > Not silly at all. I didn't know that could be an issue, but they > weren't mounted with "geli_devices," they were mounted by hand with > "geli attach /dev/ad<disk>." I did not set the -d flag on attach, and > I don't think I used the -l flag on detach, either. Listing the > device says this: > > Geom name: ad10.eli > EncryptionAlgorithm: AES-CBC > KeyLength: 128 > Crypto: hardware > UsedKey: 0 > Flags: NONE > > (and more stuff) > > One more interesting thing: I accidentally rebooted the system without > any detaching/exporting (it involved a different, bad drive). When it > came up, I was able to re-import tank without any problems. > Ok, here's where it gets interesting: The next time I saw the import error, I ran zdb -l on the actual dev. It couldn't find the labels. So I used dd to grab the first 4k of the .eli device and the actual device. Once I got it working, I repeated. The data in the first 4k of /dev/ad8 were all 0x00 both times. I'm guessing this is reserved, or something. The data in the first 4k of /dev/ad8.eli differed between runs (so zdb -l is probably right about not finding the label). In the /dev/ad8.eli that zfs doesn't recognize, I found a 16 byte string that was repeated a lot, but it was also repeated in another place: the good /dev/ad10.eli (though the offsets were different). The other weird thing: the good and bad /dev/ad8.eli look a lot alike: one 16 byte string, then another that gets repeated, then another 16 byte string randomly shows up at 0x200. Why the same data appear in the bad ad8.eli as the good ad10.eli, I'm not sure (I do have the same password and no keyfile with geli), but the patterns of data looking the same make me think something's wrong with the encryption. It's using 128 bit AES-CBC, and these patterns would not be hidden by it (128 bits == 16 bytes). I'm using a Via C7 CPU's padlock cryptographic accelerator, and geli reports this. I'm guessing this is either a padlock or a geli bug. I can't reliably reproduce this problem, but doing it with padlock off might be a good test.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6e0e5340901221324o33f1e2b1l53c842ebf9dad9a8>