Date: Sun, 27 Jun 2010 09:36:44 +0100 From: Matthew Seaman <m.seaman@infracaninophile.co.uk> To: Matthew Lear <matt@bubblegen.co.uk> Cc: Adam Vande More <amvandemore@gmail.com>, freebsd-stable@freebsd.org, Jeremy Chadwick <freebsd@jdc.parodius.com> Subject: Re: 7.2-RELEASE-p4, IO errors & RAID1 failure Message-ID: <4C270D9C.3000801@infracaninophile.co.uk> In-Reply-To: <1277593488.1884.107.camel@almscliff.bubblegen.co.uk> References: <1276889330.2210.44.camel@almscliff.bubblegen.co.uk> <1277155992.1860.3.camel@almscliff.bubblegen.co.uk> <20100622074541.GA71157@icarus.home.lan> <82A96ECD-676C-4A4D-A328-0CFAABD64D50@gid.co.uk> <1277401934.1874.12.camel@almscliff.bubblegen.co.uk> <20100624181535.GA58443@icarus.home.lan> <1277417182.1874.30.camel@almscliff.bubblegen.co.uk> <AANLkTimo1Vb461DHw3ZXNwK5BxDcgzKSkdxc3Dnqizge@mail.gmail.com> <20100625071644.GA75910@icarus.home.lan> <1277567868.1870.21.camel@almscliff.bubblegen.co.uk> <20100626171251.GA26022@icarus.home.lan> <1277593488.1884.107.camel@almscliff.bubblegen.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 27/06/2010 24:04:48, Matthew Lear wrote:
> Incidentally, is there a way to easily migrate from a atacontrol created
> array to a gmirror created array? I'm running FreeBSD 8.0 on another
> machine with a gmirror created RAID1 array with no problem whatsoever (I
> chose gmirror as the choice for this machine over atacontrol after
> reading various postings about software RAID under recent releases of
> FreeBSD). I was planning on upgrading the 7.2 machine to 8.0-RC1 anyway
> so if I could easily move to using gmirror then I would. That said,
> atacontrol should (I assume) function correctly with 8.x, shouldn't it,
> or is support of it dwindling somewhat?
So long as your ataraid setup is a simple mirror, this is actually quite
easy.
Lets assume you have two identical disks, ad0 and ad2 from which you
create a RAID ar0. To switch over to gmirror based RAID:
* Edit /etc/fstab changing all references to ar0 to read ad0, so
eg. change:
/dev/ar0s1a / ufs rw 1 1
to read:
/dev/ad0s1a / ufs rw 1 1
and similarly for any other partitions on ar0 (don't forget the
swap area)
* Edit /boot/loader.conf to load the necessary gmirror kernel modules
add:
geom_mirror_load="YES"
* Reboot
If necessary, drop into the system BIOS and make sure that which
ever drive corresponding to your ad0 is set as the primary drive
to boot from.
This should bring your system up on just your ad0 drive -- so no
resilience for a few minutes.
(Effectively, you're back at the default system on a single
disk layout now)
* Delete the old ar0 RAID:
# atacontrol delete ar0
This should have no effect on the running system as all it does
is remove some meta-data stored on the disk.
* Configure a new geom mirror device. You need to know the sysctl
magic to let you futz with a mounted drive:
# sysctl kern.geom.debugflags=16
# gmirror label -v -b load gm0 /dev/ad0
(If you're a fan of the classic article on OnLamp
http://onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html) which
recommends using round-robin as the balancing algorithm: that's
fine as well, but 'load' as a mechanism wasn't available when
that article was written, and it is generally recognised as
superior nowadays.)
* Re-edit /etc/fstab to change all references from ad0 to mirror/gm0
so eg. change:
/dev/ad0s1a / ufs rw 1 1
to:
/dev/mirror/gm0s1a / ufs rw 1 1
and similarly for any other partitions on ad0
* Reboot. You should see messages towards the end of the kernel
initialisation sequence saying gmirror raid created successfully.
* Now add your second drive to the mirror:
# gmirror insert gm0 /dev/ad2
* The system will spend some time synchronizing the data from ad0 to
ad0 -- the system can be put back into use immediately if needed,
but it's better to let the resynch finish first, as you won't be
resilient until it has. Also, active disk use will slow down the
synchronisation process. Monitor synchronisation progress by
'gmirror status gm0' or by running gstat(8).
* Once everything is synch'd it's all ready for immediate use.
> How easy is it to upgrade an array to use larger disks - atacontrol or
> gmirror? Feel free to respond with RTFM :-) I suppose one possible
> solution would be to use something like GpartEd (example Linux land
> tool) to grow a certain partition on an array (eg the partition mounted
> on /usr/local). That way both partitions on each of the separate array
> subdisks would be grown transparently since the operation would be
> performed on partition ar0s1<n> (ie, taken care of by atacontrol /
> gmirror).
gmirror doesn't really help you with resizing disks or filesystems.
Note the the recipe above assumes you want to mirror the entire contents
of one disk onto another -- which is the usual idea for the system drive
used by a server -- so ideally you want two identical drives. If one of
your drives fails and you need to replace it, but can't get the exact
same model, you will have to use a *bigger* drive -- ie. a larger number
of sectors. Be wary when buying, as disks of different models but with
nominally the same capacity can easily vary by a few thousand sectors.
Anyhow, it is certainly possible to replace one of the drives in a
gmirror setup with another of much larger capacity, synchronise the
drives, then replace the second drive with a similar larger one and
synchronise back again. If you have hot-swap capability, you can do all
that without downtime. However, you will end up with a system layout
exactly the same size as the original disks. In order to grow the
filesystems to fill the drives, you will have to do all sorts of
high-risk invasive procedures to rewrite partition tables and run
growfs(8) to expand your UFS filesystems to fill the available space.
Note: you can't shrink UFS partitions easily -- generally you have to
backup the data, and then delete and recreate the partition as a smaller
size and restore the data.
If you want to do this sort of migration from ar0 to a much bigger
gmirror setup, probably the lowest risk, least effort route is to mount
the two new drives on the system, configure a gmirror across the two,
partition and create filesystems on the gmirror, use dump(8) and
restore(8) to copy the filesystems from your live disks, then shutdown,
take out the old ar0 disks and boot from the new gmirror. Remember
you're going to have to fix /etc/fstab on the gmirror before rebooting.
If you can't fit 4 drives in your system at once, then take out one
drive (which will break the ar0 mirror), replace it with one of your new
disks, build the gmirror on it as above, then shutdown, remove the 2nd
ar0 drive, reboot, add the 2nd gmirror drive and resynch.
If you really want to be able to resize partitions easily or swap out
the hardware and have the system utilize the increased space without a
great deal of faffing around, then ZFS is your friend. See:
http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror
You should be able to adapt those instructions to install a ZFS system
on some fresh drives attached to an existing system which uses
atacontrol or gmirror raid + UFS rather than booting from installation
media, or you can split your original mirror and build half a ZFS mirror
in the space that creates, then resynch disks afterwards.
It's more work to set up ZFS initially, but once it is done, it just
ticks along happily and it makes any subsequent disk manipulations much
easier and safer.
Cheers,
Matthew
- --
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matthew@infracaninophile.co.uk Kent, CT11 9PW
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkwnDZwACgkQ8Mjk52CukIzBmwCcC1bA1h0Ux80LuW+Ip4W6YKbn
tcEAn14MxpNRBgUSIYfK6CRb5eF3WiT9
=oFID
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C270D9C.3000801>