From owner-freebsd-geom@FreeBSD.ORG Sun Oct 19 17:44:36 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDDF51065674 for ; Sun, 19 Oct 2008 17:44:35 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.182]) by mx1.freebsd.org (Postfix) with ESMTP id BBDD28FC18 for ; Sun, 19 Oct 2008 17:44:35 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: by wa-out-1112.google.com with SMTP id n4so805618wag.27 for ; Sun, 19 Oct 2008 10:44:35 -0700 (PDT) Received: by 10.115.32.8 with SMTP id k8mr4655307waj.217.1224437046324; Sun, 19 Oct 2008 10:24:06 -0700 (PDT) Received: by 10.115.89.3 with HTTP; Sun, 19 Oct 2008 10:24:06 -0700 (PDT) Message-ID: Date: Sun, 19 Oct 2008 19:24:06 +0200 From: "=?ISO-8859-1?Q?Marius_N=FCnnerich?=" To: "Cyrus Rahman" In-Reply-To: <9e77bdb50810011331y7216eac3yf85907f96f5e8370@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <9e77bdb50810011331y7216eac3yf85907f96f5e8370@mail.gmail.com> Cc: freebsd-geom@freebsd.org Subject: Re: Experiences with Gpart X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Oct 2008 17:44:36 -0000 Hello Cyrus, thanks for your experimentation. On Wed, Oct 1, 2008 at 10:31 PM, Cyrus Rahman wrote: > I recently had occasion to set up a system that needed to contain > numerous partitions, and I thought it would be a good occasion to try > gpart to divide the disk. By and large it went very well, with the > exception of bug in 'gpart bootcode' which made it necessary to > manually dd in the bootstrap. > > I did discover a few oddities that I will mention here. > > My intent was to mirror the disks with gmirror. It didn't make sense > to individually mirror each gpart partition and have some 30 > simultaneous restores going should the mirror ever break. My first > idea was to mirror the entire disks (ad4 and ad6) with gmirror and > put gpart over the mirror. > > This sort of works, except that gpart itself if it finds itself at the > beginning of the disk even when it doesn't properly taste itself on > the last block of the disk. As a result, the same gpart partitions > get attached to ad4 and ad6, as well as to the mirror. The gpart > geoms attached directly to the disks complain about the last sector > being incorrect (the backup table) and request that you fix it (which > would not likely go well). But in addition to that, any further geoms > contained within the mirror get attached to the outer gpart geoms on > ad4 and ad6 and not the mirror. They don't work properly, because the > mirror is already attached to ad4 and ad6, so you end up with a > non-functional setup. I don't know of a solution for this. Maybe GEOM itself should be extended with something like a taste priority? So that gmirror always tastes before gpart. But I do not really like this because it adds complexity. > In experimenting, I also discovered that if you directly nest gpart > tables, things work fine until you reboot. Then the internal > partitions vanish because gpart doesn't attach to them. > > Fortunately, if you put the inner gpart inside a gmirror inside the > outer gpart, things do work fine. > > This I did for two reasons: First, to avoid the aforementioned > trouble with an outermost gmirror containing a gpart geom, and second, > so that I could better define how the disk was to be mirrored. I was > able to put a number of smaller filesystems inside a single mirrored > gpart partition, and then most of the really big ones inside a second > mirrored gpart partition. This second partition used the -F option to > gmirror so that it would not need to be re-synchonized in a crash. > The filesystems within it are gjournal'd to make this possible. > > Booting was yet another issue, since the loader objects to nested > gpart partitions. I handled this by simply making a third partition > on the outermost gpt that contained the root. > > Since this is likely confusing, here's a diagram that will help if you > use a fixed font: > > ad4 ad6 > ------ ------ > gpart 1 (ad4p1) gpart 1 (ad6p1) > freebsd-boot freebsd-boot > > gpart 2 (ad4p2) gpart 2 (ad6p2) > gmirror (gm0 = ad4p2 + ad6p2) > root (gm0.p1) > > gpart 3 (ad4p3) gpart3 (ad6p3) > gmirror (gm1 = ad4p3 + ad6p3) > gpart 1 (gm1p1) > gpart 2 (gm1p2 > ... > gpart n (gm1pn) > > gpart 3 (ad4p4) gpart 3 (ad6p4) > gmirror (gm2 = ad4p4 + ad6p4) (mirrored with -F) > gpart 1 (gm2p1) > gjournal (gm2p1.journal) > gpart 2 (gm2p2) > gjournal (gm2p2.journal) > ... > > Despite the intent of gpt's being to make such nesting unnecessary, as > a means of defining the structure of gmirrors, which take up the > entire extent of whatever encloses them, the nesting was very helpful. Maybe nesting simply works if you comment the first if in g_part_gpt_probe() in sys/geom/part/g_part_gpt.c ? I don't get why this is restricted, should be my decision to nest or not imo. Kind regards Marius