From owner-freebsd-fs@FreeBSD.ORG Wed Jan 15 08:29:28 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73D90808 for ; Wed, 15 Jan 2014 08:29:28 +0000 (UTC) Received: from mail-oa0-x235.google.com (mail-oa0-x235.google.com [IPv6:2607:f8b0:4003:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A85B179F for ; Wed, 15 Jan 2014 08:29:28 +0000 (UTC) Received: by mail-oa0-f53.google.com with SMTP id i7so851709oag.40 for ; Wed, 15 Jan 2014 00:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=j3djeQMOXnj+KWc8mftwRxyGxpahVu5Uyk76yYaeJWc=; b=zDEldFEfhLzVf9f62lcWhuNN4k+mbYFQ8Me5O6Avw56OGYshpeuRV/YhYVPimE84rc 6hVUqzCZepMEhoHnYnMzRTLy9vph0iKuWIqceLweREhZsgP4tN3yg9jx2EEcCy4mWd+c vdu8t6CbeleX/yQUTZ2iUSodBsjoSRSS8DlTvYDUtB6hSb9lOBf60DVEZDy41bvmss+O uSUgqZzOrR2FmJMKoTOZ9Xzm0hbF6p1gxi3mgioSrW9Pb6JuCGPUu7Z/ZR8ypCSc7Pg5 69fVPCzGDEEn6HMYKNMQtyUgcgYCgKUHeRo6ZCaiS232CnMkrfczU5wz3/JtUa7PqccJ NAHQ== MIME-Version: 1.0 X-Received: by 10.60.37.33 with SMTP id v1mr769455oej.2.1389774567442; Wed, 15 Jan 2014 00:29:27 -0800 (PST) Received: by 10.60.171.145 with HTTP; Wed, 15 Jan 2014 00:29:27 -0800 (PST) In-Reply-To: References: Date: Wed, 15 Jan 2014 09:29:27 +0100 Message-ID: Subject: Re: zpool import taking weeks ... From: Ulysse 31 To: Freddie Cash Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jan 2014 08:29:28 -0000 Thanks for the answer. I was beginning to ... well ... have a little start of panic ^^' The import still running but no real activity on disks : sometimes, maybe something like each hour, there is some really brief led activity on the hdd, but nothing more. one thing that would be great is to have an overall percentage or just counting of what is done or what is left ... for now i don't even know if it's gonna end on the next 5 mins or on the next 5 weeks :s So now since i have no real activity on the actual zpool import (the last import started a week ago, it do not have go out of swap since i added lot of swap (260G HDD swap)), should i just let it run, or make a reboot and start over ? Also once imported, and since i don't care about the data actually on the incriminated dataset (the one that crashed with the recv/rollback), I was planning to simply delete that dataset and make a new full sync, what do you think about it ? Thanks again for your help. 2014/1/14 Freddie Cash : > On Tue, Jan 14, 2014 at 8:58 AM, Thomas Hoffmann wrote: >> >> On Tue, Jan 14, 2014 at 11:52 AM, Freddie Cash wrote: >>> >>> Dedupe enabled on this server? >>> >>> >>> -- >>> Freddie Cash >>> fjwcash@gmail.com >> >> >> Buried in his rather lengthy text was: >> >> "The storage is using one zpool with multiple dataset with dedup on (i >> know it EATS RAM :s )" >> >> So the answer is yes. > > > Whoops, missed that. > > In that case, it's "normal". The pool was interrupted in the middle of a > send and a rollback. The import will now do the necessary cleanup to return > the pool to the correct state, during the import process. To do so, it has > to remove all the blocks from the send, remove all the blocks from the > rollback, and update the DDT entries for all those blocks. > > It will load the DDT into ARC as needed, and eventually blow out all the RAM > in the system and lockup. > > Reboot, re-do the import, it will continue from where it left off, load the > DDT into ARC, and eventually blow out all the RAM in the system and lockup. > > Rinse, repeat, reboot, blah blah blah. > > After several iterations, it will eventually finish the cleanup, and the > pool will import. > > This is a known limitation of dedupe and interrupted operations. Similar to > the "destroy lots of filesystems" and then lockup issue. Basically, dedupe > makes a lot of things very slow in recovery. > > I once had to spend 2 weeks doing the reboot/import/wait/lockup dance to get > through a similar situation. > > - > - > Freddie Cash > fjwcash@gmail.com -- Ulysse31