From owner-freebsd-questions@FreeBSD.ORG  Mon Apr 27 13:37:39 2009
Return-Path: <owner-freebsd-questions@FreeBSD.ORG>
Delivered-To: freebsd-questions@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id A3E1D106566B
	for <freebsd-questions@freebsd.org>;
	Mon, 27 Apr 2009 13:37:39 +0000 (UTC)
	(envelope-from psteele@maxiscale.com)
Received: from exprod7og123.obsmtp.com (exprod7og123.obsmtp.com [64.18.2.24])
	by mx1.freebsd.org (Postfix) with SMTP id 5F18A8FC1D
	for <freebsd-questions@freebsd.org>;
	Mon, 27 Apr 2009 13:37:39 +0000 (UTC)
	(envelope-from psteele@maxiscale.com)
Received: from source ([209.85.146.180]) by exprod7ob123.postini.com
	([64.18.6.12]) with SMTP
	ID DSNKSfW1Ih8IjPszOEoerNAVklEg3iO/gRhf@postini.com;
	Mon, 27 Apr 2009 06:37:39 PDT
Received: by wa-out-1112.google.com with SMTP id j5so910841wah.2
	for <freebsd-questions@freebsd.org>;
	Mon, 27 Apr 2009 06:37:38 -0700 (PDT)
Received: by 10.114.133.1 with SMTP id g1mr2543801wad.21.1240839458664;
	Mon, 27 Apr 2009 06:37:38 -0700 (PDT)
Received: from localhost ([76.231.178.131])
	by mx.google.com with ESMTPS id m29sm14702289poh.25.2009.04.27.06.37.37
	(version=SSLv3 cipher=RC4-MD5); Mon, 27 Apr 2009 06:37:38 -0700 (PDT)
Date: Mon, 27 Apr 2009 06:37:36 -0700 (PDT)
From: Peter Steele <psteele@maxiscale.com>
To: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl>
Message-ID: <2732371.101240839456548.JavaMail.HALO$@halo>
In-Reply-To: <18933005.81240838051541.JavaMail.HALO$@halo>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.5
Cc: #freebsd-questions <freebsd-questions@freebsd.org>
Subject: Re: Unexpected gmirror behavior: Is this a bug?
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions>
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Apr 2009 13:37:39 -0000

> i think it's a bug but only happens with such massive mirror. very few 
>people do more than 2-way mirrors that's probably it wasn't catched. 
> 
>please do report the bug - it's critical. 

In fact I just confirmed that if we reduce our mirror to just two members the problem does not occur. The returning member, even if it is the first drive, is always re-synced with the data from the other drive and no data is lost. 

And yes, it's definitely a critical bug. I'm filing a bug report now, but we may have to fix this in-house before we can release our product with this problem. There is too great a risk for customers to lose data. 

Peter