From owner-freebsd-fs@FreeBSD.ORG Sun Sep 20 22:22:55 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C586106566B for ; Sun, 20 Sep 2009 22:22:55 +0000 (UTC) (envelope-from ktouet@gmail.com) Received: from mail-yw0-f199.google.com (mail-yw0-f199.google.com [209.85.211.199]) by mx1.freebsd.org (Postfix) with ESMTP id D181C8FC12 for ; Sun, 20 Sep 2009 22:22:54 +0000 (UTC) Received: by ywh37 with SMTP id 37so2854451ywh.28 for ; Sun, 20 Sep 2009 15:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=YI/+B76HedaPioctwmcvygBwY1ODGsbc6VNixFMjPak=; b=g1ibe9Dldq5ia1Xr+A10YVYED1CvlUlIixqFplWbTKcB29hGxNFivG7UFZ4YEyI+R7 MUzFPAaND26TQGN0fmhdYTqH66yylZqcvJ9gcMMY7vdDzi/83nLFlMbeej7ISmOGG+6G EVALZ1KqZ7d0XD8St1SXzIuVZPj4j3op1s/oc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=MmK2n1MgC8gjQJpYyF+ehH5NY/iTiByuwglQ1d9TEu9CC/AbjpGnhU0B749EbHrzob /7/5mLlne9lbJ4KWYPnhyguh0kzzSbpnrRM4mR28O4FJixyiLtfMxrb5xK8pNYC+beDR ZN4/42pyBlyq6hGvKqsuWA5dE5yD7rbHy7r6U= MIME-Version: 1.0 Received: by 10.90.13.15 with SMTP id 15mr2828487agm.74.1253484010343; Sun, 20 Sep 2009 15:00:10 -0700 (PDT) Date: Sun, 20 Sep 2009 16:00:10 -0600 Message-ID: <2a5e326f0909201500w1513aeb5ra644f1c748e22f34@mail.gmail.com> From: Kurt Touet To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: ZFS - Unable to offline drive in raidz1 based pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2009 22:22:55 -0000 I am using ZFS pool based on a 4-drive raidz1 setup for storage. I believe that one of the drives is failing, and I'd like to remove/replace it. The drive has been causing some issues (such as becoming non-responsive and hanging the system with timeouts), so I'd like to offline it, and then run in degraded mode until I can grab a new drive (tomorrow). However, when I disconnected the drive (pulled the plug, not using a zpool offline command), the following occurred: NAME STATE READ WRITE CKSUM storage FAULTED 0 0 1 raidz1 DEGRADED 0 0 0 ad14 ONLINE 0 0 0 ad6 UNAVAIL 0 0 0 ad12 ONLINE 0 0 0 ad4 ONLINE 0 0 0 Note: That's my recreation of the output... not the actual text. At this point, I was unable to to do anything with the pool... and all data was inaccessible. Fortunately, the after sitting pulled for a bit, I tried putting the failing drive back into the array, and it booted properly. Of course, I still want to replace it, but this is what happens when I try to take it offline: monolith# zpool status storage pool: storage state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad14 ONLINE 0 0 0 ad6 ONLINE 0 0 0 ad12 ONLINE 0 0 0 ad4 ONLINE 0 0 0 errors: No known data errors monolith# zpool offline storage ad6 cannot offline ad6: no valid replicas monolith# uname -a FreeBSD monolith 8.0-RC1 FreeBSD 8.0-RC1 #2 r197370: Sun Sep 20 15:32:08 CST 2009 k@monolith:/usr/obj/usr/src/sys/MONOLITH amd64 If the array is online and healthy, why can't I simply offline a drive and then replace it afterwards? Any thoughts? Also, how does a degraded raidz1 array end up faulting the entire pool? Thanks, -kurt