Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 28 Apr 2017 15:52:59 +0200 (CEST)
From:      =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no>
To:        Paul Macdonald <paul@ifdnrg.com>
Cc:        freebsd-questions@freebsd.org
Subject:   Re: A seperate ZFS question
Message-ID:  <alpine.BSF.2.20.1704281532300.10691@mail.fig.ol.no>
In-Reply-To: <adb1b78d-2e03-d561-dac2-7c5da9a53a1c@ifdnrg.com>
References:  <adb1b78d-2e03-d561-dac2-7c5da9a53a1c@ifdnrg.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 28 Apr 2017 14:14+0100, Paul Macdonald wrote:

> 
> I fat fingered adding a cache drive and was unable to remove the SSD cache
> drive (ada2)
> 
> I added a 2nd SSD (ada3) for safety as i didn;t want to risk the pool at all
> and exporting wasn;t feasible in timescale.
> 
> Can someone explain :
> 
>             what i've actually managed to set up here (!),
> 
>             the ramifications of forceably offlining/removing ada2
> 
>             and if i should worry when ada2 gets full?
> 
>                  capacity     operations    bandwidth
> pool          alloc   free   read  write   read  write
> ------------  -----  -----  -----  -----  -----  -----
> zrootnew       692G   460G     15    174   351K  2.40M
>   mirror       683G   237G     14    138   328K  1.43M
>     gpt/zfs0      -      -      6     50   155K  1.44M
>     ada1p3        -      -      7     50   176K  1.44M
>   ada2        9.17G   223G      0     36  22.9K   989K
> cache             -      -      -      -      -      -
>   ada3        39.0G   194G     23      5   168K   264K

First, you have gpt/zfs0 and ada1p3 in a mirrored configuration. Next, 
you have ada2 striped (RAID 0) to the first pair of mirrors. This is a 
very weird "RAID 1 + 0" combo. Guess what will happen when you lose 
ada2!

Maybe you should describe what you wanted to happen in the first 
place.

If you really wanted a three-way mirror, you have just learned the 
hard way about the difference between zpool add and zpool attach.

The latter is for adding additional members to existing mirrors, while 
the former is for adding raidz{1,2,3} groups to a pool, and the add 
command got you into this mess.

"zpool attach zrootnew ada1p3 ada2" is the right command if you wanted 
to create a three-way mirror in the first place.

I suggest you make good backups of this pool, destroy it, recreate it 
properly, and restore the missing data.

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+
From owner-freebsd-questions@freebsd.org  Fri Apr 28 14:28:44 2017
Return-Path: <owner-freebsd-questions@freebsd.org>
Delivered-To: freebsd-questions@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67E82D5322B
 for <freebsd-questions@mailman.ysv.freebsd.org>;
 Fri, 28 Apr 2017 14:28:44 +0000 (UTC)
 (envelope-from ultima1252@gmail.com)
Received: from mail-yb0-x229.google.com (mail-yb0-x229.google.com
 [IPv6:2607:f8b0:4002:c09::229])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 25E421F89
 for <freebsd-questions@freebsd.org>; Fri, 28 Apr 2017 14:28:44 +0000 (UTC)
 (envelope-from ultima1252@gmail.com)
Received: by mail-yb0-x229.google.com with SMTP id 8so15840855ybw.1
 for <freebsd-questions@freebsd.org>; Fri, 28 Apr 2017 07:28:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=mMrJvqJDFsI3bl1BNQB+T7PkNiH2+HEOzHuIqO4wE/k=;
 b=huvzfyJqN7yYCHu8XTHbKDOARqMH6X92KMAeil6R71UwGoApqsUE3GOIVYiOBg1P+J
 fzNR8Pytr+vlLGqzvWFiKce0+HmnY71Nb7l/hNphxXGRrR5/6IhfEQuCgoLOtDDl3RrN
 NRm9LoWnaJ67u6A6erAxISok1kiz5pt9PrOQhkCeMu3U/TAsYeMTWfGUibaznodeciGF
 rouc8QsDvEhwPeA15QwzqBMgQVzbRxj/pgVuw6lV/7Xtl5cR8GP3dZ2yrfL84GSMXqZy
 yxMjv1Mma6rNj305Yfnv/Yzk6R+h+K3MJgRR+AvMpygXlPd0onW5dp8NGHnoCr6nRWni
 JkXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=mMrJvqJDFsI3bl1BNQB+T7PkNiH2+HEOzHuIqO4wE/k=;
 b=KYyHNMvVcb29d7gFKIdWbeZOu8vuGIUk5F4Y3AIu6X1+00qoq/Pru2TfYsGkh/ZiCd
 Nr0nDhR59tn83el+khOYQZE0J4miPB5THIb33lADqGLFNH87E0YdDhwrdjusp4kWgwdv
 DZrnSNp/0dnunUEgqwcv7WPuRGlrR4KgOh5H4G7G2KAHULWsIMfTyziAkStDhXA9h8I0
 gORzD2J4La/dD9v3eQfXEQ4LCIgwmpDxa2BMwkz1xPctEt2o6QcJ0ynN5d5xT/cj4KLN
 jQp2d5HmU0F8XMZeHFJPPcEq7AnCY3Uwmbw3b6HwPiDu/8TmYPvv08z7MS2GopxP4SRA
 Xc2w==
X-Gm-Message-State: AN3rC/7z+gmIO8JBAWlWdkzIkitaKrxAQeG/VkY5ZAkcpIvxymxi+HpI
 +FWDSaGtchbol7WNLvrRCzVsjklzai2cKmI=
X-Received: by 10.37.1.4 with SMTP id 4mr9505859ybb.91.1493389723175; Fri, 28
 Apr 2017 07:28:43 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.129.72.144 with HTTP; Fri, 28 Apr 2017 07:28:42 -0700 (PDT)
In-Reply-To: <alpine.BSF.2.20.1704281532300.10691@mail.fig.ol.no>
References: <adb1b78d-2e03-d561-dac2-7c5da9a53a1c@ifdnrg.com>
 <alpine.BSF.2.20.1704281532300.10691@mail.fig.ol.no>
From: Ultima <ultima1252@gmail.com>
Date: Fri, 28 Apr 2017 10:28:42 -0400
Message-ID: <CANJ8om5Y6Q9hFVny12TyjJvMJ6La+NekEfKvH+j4Za5Z8nhdiw@mail.gmail.com>
Subject: Re: A seperate ZFS question
To: Paul Macdonald <paul@ifdnrg.com>
Cc: FreeBSD Mailing List <freebsd-questions@freebsd.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.23
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions/>;
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Apr 2017 14:28:44 -0000

-n is your friend. Never used zpool add without it to verify changes are as
you intend. Not using it first is similar to jumping off a plane without
verifying your parachute is on properly.

On Fri, Apr 28, 2017 at 9:52 AM, Trond Endrest=C3=B8l <
Trond.Endrestol@fagskolen.gjovik.no> wrote:

> On Fri, 28 Apr 2017 14:14+0100, Paul Macdonald wrote:
>
> >
> > I fat fingered adding a cache drive and was unable to remove the SSD
> cache
> > drive (ada2)
> >
> > I added a 2nd SSD (ada3) for safety as i didn;t want to risk the pool a=
t
> all
> > and exporting wasn;t feasible in timescale.
> >
> > Can someone explain :
> >
> >             what i've actually managed to set up here (!),
> >
> >             the ramifications of forceably offlining/removing ada2
> >
> >             and if i should worry when ada2 gets full?
> >
> >                  capacity     operations    bandwidth
> > pool          alloc   free   read  write   read  write
> > ------------  -----  -----  -----  -----  -----  -----
> > zrootnew       692G   460G     15    174   351K  2.40M
> >   mirror       683G   237G     14    138   328K  1.43M
> >     gpt/zfs0      -      -      6     50   155K  1.44M
> >     ada1p3        -      -      7     50   176K  1.44M
> >   ada2        9.17G   223G      0     36  22.9K   989K
> > cache             -      -      -      -      -      -
> >   ada3        39.0G   194G     23      5   168K   264K
>
> First, you have gpt/zfs0 and ada1p3 in a mirrored configuration. Next,
> you have ada2 striped (RAID 0) to the first pair of mirrors. This is a
> very weird "RAID 1 + 0" combo. Guess what will happen when you lose
> ada2!
>
> Maybe you should describe what you wanted to happen in the first
> place.
>
> If you really wanted a three-way mirror, you have just learned the
> hard way about the difference between zpool add and zpool attach.
>
> The latter is for adding additional members to existing mirrors, while
> the former is for adding raidz{1,2,3} groups to a pool, and the add
> command got you into this mess.
>
> "zpool attach zrootnew ada1p3 ada2" is the right command if you wanted
> to create a three-way mirror in the first place.
>
> I suggest you make good backups of this pool, destroy it, recreate it
> properly, and restore the missing data.
>
> --
> +-------------------------------+------------------------------------+
> | Vennlig hilsen,               | Best regards,                      |
> | Trond Endrest=C3=B8l,              | Trond Endrest=C3=B8l,             =
      |
> | IT-ansvarlig,                 | System administrator,              |
> | Fagskolen Innlandet,          | Gj=C3=B8vik Technical College, Norway, =
 |
> | tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
> | sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
> +-------------------------------+------------------------------------+
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-
> unsubscribe@freebsd.org"
>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.20.1704281532300.10691>