Date: Tue, 14 Jan 2014 10:07:29 -0800 From: Freddie Cash <fjwcash@gmail.com> To: Thomas Hoffmann <trh411@gmail.com> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: zpool import taking weeks ... Message-ID: <CAOjFWZ6r2LYWAw_eB4f9AEoy_w313rRy44t9uBdNMK2MJTmQwg@mail.gmail.com> In-Reply-To: <CAB7-odmXb%2B0KvEh6vDPstt89-O=ccL81MAtX2hRLWk%2BRcVZzUA@mail.gmail.com> References: <CAFSDvD2hLNN_CupeVKu02bLLUS1EFuTdZZw5Z9xoiNFZ0p4_fg@mail.gmail.com> <CAOjFWZ5dgw0nZzWFhWOCS-HSseBC=uavc=aQM%2Byht-R5a4Xfow@mail.gmail.com> <CAB7-odmXb%2B0KvEh6vDPstt89-O=ccL81MAtX2hRLWk%2BRcVZzUA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jan 14, 2014 at 8:58 AM, Thomas Hoffmann <trh411@gmail.com> wrote: > On Tue, Jan 14, 2014 at 11:52 AM, Freddie Cash <fjwcash@gmail.com> wrote: > >> Dedupe enabled on this server? >> >> >> -- >> Freddie Cash >> fjwcash@gmail.com > > > Buried in his rather lengthy text was: > > "The storage is using one zpool with multiple dataset with dedup on (i > know it EATS RAM :s )" > > So the answer is yes. > =E2=80=8BWhoops, missed that. In that case, it's "normal". The pool was interrupted in the middle of a send and a rollback. The import will now do the necessary cleanup to return the pool to the correct state, during the import process. To do so, it has to remove all the blocks from the send, remove all the blocks from the rollback, and update the DDT entries for all those blocks. It will load the DDT into ARC as needed, and eventually blow out all the RAM in the system and lockup. Reboot, re-do the import, it will continue from where it left off, load the DDT into ARC, and eventually blow out all the RAM in the system and lockup. Rinse, repeat, reboot, blah blah blah. After several iterations, it will eventually finish the cleanup, and the pool will import. This is a known limitation of dedupe and interrupted operations. Similar to the "destroy lots of filesystems" and then lockup issue. Basically, dedupe makes a lot of things very slow in recovery. I once had to spend 2 weeks doing the reboot/import/wait/lockup dance to get through a similar situation.=E2=80=8B =E2=80=8B-=E2=80=8B - Freddie Cash fjwcash@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ6r2LYWAw_eB4f9AEoy_w313rRy44t9uBdNMK2MJTmQwg>