Date: Sat, 3 Nov 2007 02:54:35 +0100 From: Ulf Lilleengen <lulf@stud.ntnu.no> To: Marco Haddad <freebsd-lists@ideo.com.br> Cc: Joe Koberg <joe@rootnode.com>, Peter Giessel <pgiessel@mac.com>, freebsd-geom@freebsd.org Subject: Re: gvinum and raid5 Message-ID: <20071103015435.GB22755@stud.ntnu.no> In-Reply-To: <8d4842b50711021832g7ad7cec9x48d2f114b1e41f5f@mail.gmail.com> References: <8d4842b50710310814w3880f7d3ldf8abe3a236cbcc8@mail.gmail.com> <20071031215756.GB1670@stud.ntnu.no> <472AA59F.3020103@rootnode.com> <0001DFFC-0115-1000-9A80-3F81219C1B16-Webmail-10013@mac.com> <8d4842b50711021832g7ad7cec9x48d2f114b1e41f5f@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On fre, nov 02, 2007 at 10:32:54 -0300, Marco Haddad wrote: > Hi, > > I must say that I had a strong faith on vinum too. I used it on a dozen > servers to build raid5 volumes, specially when the hard drives were small > and unreliable. So I had a few crashes naturally, but replacing the failed > disk was easy and rebuild worked all times. > [...] > > Suppose ad1 crashes and gvinum marks it as down. With the command "gvinum l" > we would get something like this: > > 3 drives: > D AD0 State: up /dev/ad0s1d ... > D AD1 State: down /dev/ad1s1d ... > D AD2 State: up /dev/ad2s1d ... > > 1 volumes: > V VOL State: up ... > > 1 plexes: > P VOL.p0 R5 State: degraded ... > > 3 subdisks: > S VOL.p0.s0 State: up D: AD0 ... > S VOL.p0.s1 State: down D: AD1 ... > S VOL.p0.s2 State: up D: AD2 ... > > > First thing I do: edit fstab and comment out the line mounting > /dev/gvinum/VOL wherever it was mounted. It is necessary because once > mounted gvinum can not operate most commands, and umount doesn't do the > trick. Then I shutdown the system and replace the hard disk and bring it up > again. True, this was a bit of a pain, also because the geom_vinum module can't be kldunloaded. This have been fixed in the "experimental" version :) > > At this point the first weird thing can be noted. With 'gvinum l' you would > get: > > 2 drives: > D AD0 State: up /dev/ad0s1d ... > D AD2 State: up /dev/ad2s1d ... > > 1 volumes: > V VOL State: up ... > > 1 plexes: > P VOL.p0 R5 State: up ... > > 3 subdisks: > S VOL.p0.s0 State: up D: AD0 ... > S VOL.p0.s1 State: up D: AD1 ... > S VOL.p0.s2 State: up D: AD2 ... > > What? The AD1 is gonne, ok, but why the subdisk VOL.p0.s1 is up? And it > makes the plex up instead of degraded. The first time I saw it I got the > shivers. This is fixed in new gvinum changes from Summer of Code. The current version of gvinum is very bad at keeping correct configuration data, which have many issues. > > Next step is to fdisk and label the new disk just like the old one. The new > disk can be bigger but, I think, the partition ad1s1d must be the same size > as before. Yes, at least the same size or bigger. > [...] > to rebuild it. It may take about 1 hour per 100GB of volume space, so we > better grab some lunch... > > The progress can be seen at any time with: > > gvinum ls > > After that, a 'fsck -t ufs /dev/gvinum/VOL' will probably catch some errors > left behind when the drive came down. > > Now we just need to uncomment that line in fstab and reboot. > > I think there's no easier way... Yes there is. Replacing a drive in gvinum follows the following procedure: 1. Create config for the new drive and name the drive _differently_ than the old one. 2. Use the gvinum 'move' command to move the stale subdisk to the new drive. 3. Make sure the the subdisk now points to the new drive and that it's in the 'stale' state. 4. Start the plex (gvinum start). The other issues you encountered have been fixed in my gvinum work this summer. Also, replacing a drive and rebuilding a plex can happen without unmounting your volume in the new gvinum. brave users can find patches here: http://people.freebsd.org/~lulf/patches/gvinum All testing is very much appreciated. -- Ulf Lilleengen
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20071103015435.GB22755>