From owner-freebsd-stable@FreeBSD.ORG Tue Oct 14 11:30:33 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3F196B2; Tue, 14 Oct 2014 11:30:33 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 2FF25DDF; Tue, 14 Oct 2014 11:30:33 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3jHF1N4138z1Gc; Tue, 14 Oct 2014 13:30:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= content-transfer-encoding:content-type:content-type:in-reply-to :references:subject:subject:mime-version:user-agent:organization :from:from:date:date:message-id:received:received:received; s= jakla4; t=1413286227; x=1415878228; bh=zLHNxpSja50NlONXqgXlIWA5a gJI6zlDpAKAum9KRSE=; b=HAYylhABxzW6VWCfGn9b6LoSwYcUJNfTmPn3i0nX4 ePT8QdCCpmpBbtHW5HkdogcMjcwkrWljoxn7+9/KfzAvMKsaB6iWurKO1n00vaDJ 0AhgKSz59CUml2BZhnVAMaM7hAeXBtmO2JkHa38y6lUhc7KmzUiKauTLoEUYEuZM vk= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id YIifCGDKsbao; Tue, 14 Oct 2014 13:30:27 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Tue, 14 Oct 2014 13:30:27 +0200 (CEST) Received: from sleepy.ijs.si (msleepy-1-pt.tunnel.tserv27.prg1.ipv6.he.net [IPv6:2001:470:6e:18e::2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id 3jHF1H48xzzDy; Tue, 14 Oct 2014 13:30:27 +0200 (CEST) Message-ID: <543D0953.1070604@ijs.si> Date: Tue, 14 Oct 2014 13:30:27 +0200 From: Mark Martinec Organization: J. Stefan Institute User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: zpool import hangs when out of space - Was: zfs pool import hangs on [tx->tx_sync_done_cv] References: <54372173.1010100@ijs.si> <644FA8299BF848E599B82D2C2C298EA7@multiplay.co.uk> <54372EBA.1000908@ijs.si> <543731F3.8090701@ijs.si> <543AE740.7000808@ijs.si> <6E01BBEDA9984CCDA14F290D26A8E14D@multiplay.co.uk> <14ADE02801754E028D9A0EAB4A16527E@multiplay.co.uk> <543C3C47.4010208@ijs.si> <138CF459AA0B41EB8CB4E11B3DE932CF@multiplay.co.uk> In-Reply-To: <138CF459AA0B41EB8CB4E11B3DE932CF@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Oct 2014 11:30:33 -0000 On 10/14/2014 13:19, Steven Hartland wrote: > Well interesting issue I left this pool alone this morning literally doing > nothing, and its now out of space. > zpool list > NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH > ALTROOT > sys1boot 3.97G 3.97G 190K 0% - 99% 1.00x ONLINE - > sys1copy 3.97G 3.97G 8K 0% - 99% 1.00x ONLINE - > > There's something very wrong here as nothing has been accessing the pool. > > pool: zfs > state: ONLINE > status: One or more devices are faulted in response to IO failures. > action: Make sure the affected devices are connected, then run 'zpool > clear'. > see: http://illumos.org/msg/ZFS-8000-HC > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > zfs ONLINE 0 2 0 > md1 ONLINE 0 0 0 > > I tried destroying the pool and ever that failed, presumably because > the pool has suspended IO. That's exactly how trouble started here. Got the "One or more devices are faulted in response to IO failures" on all three small cloned boot pools one day, out of the blue. There was no activity there, except for periodic snapshoting every 10 minutes. Mark