From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 20:20:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 482EC1065673 for ; Tue, 4 Jan 2011 20:20:35 +0000 (UTC) (envelope-from cliftonr@oz.volcano.org) Received: from oz.volcano.org (oz.volcano.org [64.65.105.124]) by mx1.freebsd.org (Postfix) with ESMTP id D481B8FC12 for ; Tue, 4 Jan 2011 20:20:34 +0000 (UTC) Received: by oz.volcano.org (Postfix, from userid 1001) id 174F15081F; Tue, 4 Jan 2011 10:01:49 -1000 (HST) Date: Tue, 4 Jan 2011 10:01:49 -1000 From: Clifton Royston To: Daniel Braniss Message-ID: <20110104200149.GA33991@lava.net> Mail-Followup-To: Daniel Braniss , freebsd-stable@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 20:20:35 -0000 On Tue, Jan 04, 2011 at 04:21:31PM +0200, Daniel Braniss wrote: > Hi, > I have 2 ada disks striped: > > # gstripe list > Geom name: s1 > State: UP > Status: Total=2, Online=2 > Type: AUTOMATIC > Stripesize: 65536 > ID: 2442772675 > Providers: > 1. Name: stripe/s1 > Mediasize: 1000215674880 (932G) > Sectorsize: 512 > Stripesize: 65536 > Stripeoffset: 0 > Mode: r0w0e0 > Consumers: > 1. Name: ada0 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r0w0e0 > Number: 0 > 2. Name: ada1 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r0w0e0 > Number: 1 > > boot complains: > > GEOM_STRIPE: Device s1 created (id=2442772675). > GEOM_STRIPE: Disk ada0 attached to s1. > GEOM: ada0: corrupt or invalid GPT detected. > GEOM: ada0: GPT rejected -- may not be recoverable. > GEOM_STRIPE: Disk ada1 attached to s1. > GEOM_STRIPE: Device s1 activated. > > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 1953546045 - free - (932G) > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 1953546045 - free - (932G) > > # gpart add -t freebsd-ufs -s 20g stripe/s1 > GEOM: ada0: corrupt or invalid GPT detected. > GEOM: ada0: GPT rejected -- may not be recoverable. > stripe/s1p2 added > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 41943040 2 freebsd-ufs (20G) > 41943202 1911603005 - free - (912G) > > if I go the MBR road, all seems ok, but as soon as I try to write > the boot block (boot0cfg -B /dev/stripe/s1) again the kernel > starts to complain about corrupted GEOM too. So are you trying to partition the drives and then stripe the partitions within the drives, or are you trying to partition the stripe? It seems here as though you might be trying to first partition the drives (not clear on that) then stripe the whole drives - which will mean the partition info is wrong for the resulting striped drive set - and then repartition the striped drive set, and neither is ending up valid. If what you are intending is to partition after striping the raw drives, then you are doing the right steps, but when the geom layer tries to look at the info on the individual drives as at boot, it will find it invalid. If it the gpart layer is actually refusing to write partition info to the drives which is wrong for the drives taken individually, that would account for your problems. One valid order to do things in would be partition the drives with gpart, creating identical sets of partitions on both drives, then stripe the partitions created within them (syntax not exact): gpart add -t freebsd-ufs0 -s 10g ada0 gpart add -t freebsd-ufs1 -s 10g ada1 gstripe label freebsd-ufs freebsd-ufs0 freebsd-ufs1 That would give you a 20GB stripe, with valid partition info on each drive. If this will be your boot drive, depending on how much needs to be read from the drive before the geom_stripe kernel module gets loaded, I would think there could also be a problem booting from the drive. This is not like gmirroring two drives or partitions, where the info read from either disk early in boot will be identical, and identical (except for the last block of the partition) to what the OS sees later after the mirror is formed. I assume you're bearing in mind that if you lose either drive to a hardware fault you lose the whole thing, and consider the risk worth the potential speed/size gain. -- Clifton -- Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services