Date: Tue, 29 May 2001 13:47:24 -0400 From: David Gilbert <dgilbert@velocet.ca> To: doug.poland@omniresources.com Cc: avatar+freebsd@huey.kew.com, freebsd-questions@freebsd.org Subject: [FBSD-Q] RE: Building a Raid 1 or Raid 5 system? Message-ID: <15123.57516.832209.744900@trooper.velocet.net> In-Reply-To: <DF5018B3C986D11196FF0060089A903C3C306A@omnigbwww.omniresource.com> References: <DF5018B3C986D11196FF0060089A903C3C306A@omnigbwww.omniresource.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[note: anyone has permission to put this in an FAQ somewhere as long
as I get credit]
>>>>> "doug" == doug poland <doug.poland@omniresources.com> writes:
>> Does someone have a cookbook procedure for building and/or
>> upgrading to a Raid 1 or Raid 5 configuration using VINUM? I want
>> as much as of the system on RAID as possible, specifically
>> including /usr and (if possible) root (/).
doug> This will get you going with vinum:
doug> http://www.freebsddiary.org/vinum.php
I have a different recipe that I've used successfully on 2 drive
(raid-1) and 8 drive (raid-5) systems.
For the following ad? will be a 2 drive raid-1 and da? will be an 8
drive raid-5.
1. install the bare minimum on drive da0 (or ad0).
1a. Use the custom install
1b. fdisk all drives.
1c. (ad) disklabel
ad0s1a 128M /
ad0s1b 256M swap
ad0s1g rest of disk /d1
Disklabel ad1 with anything, we'll change it in step 2.
(da) disklabel
da0s1a 128M /
da0s1g rest of disk /d1
disklabel rest with anything, we'll change it later.
1d. install minimum from menu.
2. Boot from your minimum install, edit disklabels
2a. unmount /d1 and any other partitions you've made
2b. edit your disk 0 partion (disklabel -e) to change the g
partition to be "vinum" instead of "4.2BSD"
2c. (ad) "disklabel ad0 >ad0.label"
"disklabel -R -r ad1 ad0.label"
(da) "disklabel da0 >da0.label"
"disklabel -R -r da1 da0.label"
"disklabel -R -r da2 da0.label"
edit (disklabel -e) your da2 label. Change the 'a'
partition to a 'b' partition and change it's type to
swap. Then copy it to da3 thru da7 (or whatever)
using the same method above.
2d. (ad) you have two swap partitions now ... ad0s1b and
ad1s1b. Add them to fstab and swapon -a
(da) you have 6 possible swap partitions (da2s1b thru
da7s1b). Add 4 of them to fstab and swapon -a.
I don't know why you can't add six yet, but your
raid sever probably shouldn't swap that much :).
3. vinum config. Run "vinum create" and add all your disks together
into whatever RAID(s) you want. Make sure you start them, etc.
Also... use mkfs. Finally mount your new vinum partition on /mnt.
(if you're creating multiple raid partitions, you might mkdir
/mnt/usr /mnt/var /mnt/u, etc.) Put "vinum_start="YES"" in
your /etc/rc.conf
4. Copy /usr and/or /var to the new raid partitions. Edit fstab to
put the new partitions into the right place. mv old /usr and /var
to /usr-old and /var-old. Reboot.
5. Fire up /stand/sysinstall and install everything _but_ bin that you
want on the server. If everything's working, you can also remove
/usr-old and /var-old.
(bonus steps)
Your server is now working, but there are a few bonus steps:
6. (ad) copy / to ad1s1a and make ad1s1a also bootable.
6. (da) copy / to da1s1a and make da1s1a also bootable. You could
also have more than one other drive that was bootable. It's also
possible to use the other 2 disks that don't have swap as another
raid partition for /var (raid-0 or raid-1).
7. cvsup and make world (it is FreeBSD, not Linux, after all)!
Here are the two configurations I have:
-- 8 drive SCSI RAID-5 --
[1:14:2014]root@arbiter:/usr/ports/lang/python-doc-html> vinum dumpconfig
Drive d0: Device /dev/da0s1g
Created on at Sat May 5 12:42:19 2001
Config last updated Mon May 28 17:09:41 2001
Size: 18232800256 bytes (17388 MB)
volume usr state up
plex name usr.p0 state up org raid5 128s vol usr
sd name usr.p0.s0 drive d0 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 0s
sd name usr.p0.s1 drive d1 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 128s
sd name usr.p0.s2 drive d2 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 256s
sd name usr.p0.s3 drive d3 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 384s
sd name usr.p0.s4 drive d4 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 512s
sd name usr.p0.s5 drive d5 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 640s
sd name usr.p0.s6 drive d6 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 768s
sd name usr.p0.s7 drive d7 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 896s
[1:7:2007]root@arbiter:/usr/ports/lang/python-doc-html> vinum list
8 drives:
D d0 State: up Device /dev/da0s1g Avail: 0/17388 MB (0%)
D d1 State: up Device /dev/da1s1g Avail: 0/17388 MB (0%)
D d2 State: up Device /dev/da2s1g Avail: 0/17388 MB (0%)
D d3 State: up Device /dev/da3s1g Avail: 0/17388 MB (0%)
D d4 State: up Device /dev/da4s1g Avail: 0/17388 MB (0%)
D d5 State: up Device /dev/da5s1g Avail: 0/17388 MB (0%)
D d6 State: up Device /dev/da6s1g Avail: 0/17388 MB (0%)
D d7 State: up Device /dev/da7s1g Avail: 0/17388 MB (0%)
1 volumes:
V usr State: up Plexes: 1 Size: 118 GB
1 plexes:
P usr.p0 R5 State: up Subdisks: 8 Size: 118 GB
8 subdisks:
S usr.p0.s0 State: up PO: 0 B Size: 16 GB
S usr.p0.s1 State: up PO: 64 kB Size: 16 GB
S usr.p0.s2 State: up PO: 128 kB Size: 16 GB
S usr.p0.s3 State: up PO: 192 kB Size: 16 GB
S usr.p0.s4 State: up PO: 256 kB Size: 16 GB
S usr.p0.s5 State: up PO: 320 kB Size: 16 GB
S usr.p0.s6 State: up PO: 384 kB Size: 16 GB
S usr.p0.s7 State: up PO: 448 kB Size: 16 GB
[1:8:2008]root@arbiter:/usr/ports/lang/python-doc-html> cat /etc/rc.conf
inetd_enable="NO"
sendmail_enable="NO"
sshd_enable="YES"
usbd_enable="NO"
start_vinum="YES"
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 8"
nfs_client_enable="YES"
ifconfig_rl0="inet blah netmask 255.255.255.252 up"
defaultrouter="blah"
hostname="blah.velocet.net"
[1:9:2009]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da0
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 16*)
c: 35873082 0 unused 0 0 # (Cyl. 0 - 2232*)
g: 35610938 262144 vinum # (Cyl. 16*- 2232*)
[1:11:2011]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da2
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
b: 262144 0 swap # (Cyl. 0 - 16*)
c: 35873082 0 unused 0 0 # (Cyl. 0 - 2232*)
g: 35610938 262144 vinum # (Cyl. 16*- 2232*)
[1:12:2012]root@arbiter:/usr/ports/lang/python-doc-html> df -k
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/da0s1a 127023 112459 4403 96% /
/dev/vinum/usr 120802676 10283498 100854964 9% /usr
[1:13:2013]root@arbiter:/usr/ports/lang/python-doc-html> pstat -s
Device 1K-blocks Used Avail Capacity Type
/dev/da2s1b 130944 8 130936 0% Interleaved
/dev/da3s1b 130944 0 130944 0% Interleaved
/dev/da4s1b 130944 0 130944 0% Interleaved
/dev/da5s1b 130944 0 130944 0% Interleaved
Total 523776 8 523768 0%
-- 2 drive IDE RAID-1 --
[1:13:326]root@sheppard:~> vinum dumpconfig
Drive d0: Device /dev/ad0s1f
Created on freenet.velocet.ca at Fri Nov 3 13:52:16 2000
Config last updated Tue May 29 12:26:36 2001
Size: 30064460288 bytes (28671 MB)
volume var state up
volume usr state up
volume u state up
plex name var.p0 state up org concat vol var
plex name var.p1 state up org concat vol var
plex name usr.p0 state up org concat vol usr
plex name usr.p1 state up org concat vol usr
plex name u.p0 state up org concat vol u
plex name u.p1 state up org concat vol u
sd name var.p0.s0 drive d0 plex var.p0 state up len 4194304s driveoffset 265s plexoffset 0s
sd name var.p1.s0 drive d1 plex var.p1 state up len 4194304s driveoffset 265s plexoffset 0s
sd name usr.p0.s0 drive d0 plex usr.p0 state up len 8388608s driveoffset 4194569s plexoffset 0s
sd name usr.p1.s0 drive d1 plex usr.p1 state up len 8388608s driveoffset 4194569s plexoffset 0s
sd name u.p0.s0 drive d0 plex u.p0 state up len 46136472s driveoffset 12583177s plexoffset 0s
sd name u.p1.s0 drive d1 plex u.p1 state up len 46136472s driveoffset 12583177s plexoffset 0s
[1:4:317]root@sheppard:~> vinum list
2 drives:
D d0 State: up Device /dev/ad0s1f Avail: 0/28671 MB (0%)
D d1 State: up Device /dev/ad1s1f Avail: 0/28671 MB (0%)
3 volumes:
V var State: up Plexes: 2 Size: 2048 MB
V usr State: up Plexes: 2 Size: 4096 MB
V u State: up Plexes: 2 Size: 21 GB
6 plexes:
P var.p0 C State: up Subdisks: 1 Size: 2048 MB
P var.p1 C State: up Subdisks: 1 Size: 2048 MB
P usr.p0 C State: up Subdisks: 1 Size: 4096 MB
P usr.p1 C State: up Subdisks: 1 Size: 4096 MB
P u.p0 C State: up Subdisks: 1 Size: 21 GB
P u.p1 C State: up Subdisks: 1 Size: 21 GB
6 subdisks:
S var.p0.s0 State: up PO: 0 B Size: 2048 MB
S var.p1.s0 State: up PO: 0 B Size: 2048 MB
S usr.p0.s0 State: up PO: 0 B Size: 4096 MB
S usr.p1.s0 State: up PO: 0 B Size: 4096 MB
S u.p0.s0 State: up PO: 0 B Size: 21 GB
S u.p1.s0 State: up PO: 0 B Size: 21 GB
[1:5:318]root@sheppard:~> cat /etc/rc.conf
# This file now contains just the overrides from /etc/defaults/rc.conf
# please make all changes to this file.
# -- sysinstall generated deltas -- #
ifconfig_rl0="inet blah netmask 255.255.255.0"
defaultrouter="199.71.188.30"
inetd_enable="YES"
#ifconfig_rl0="DHCP"
hostname="blah"
sendmail_enable="YES"
sshd_enable="YES"
usbd_enable="NO"
start_vinum="YES"
portmap_enable="NO"
[1:7:320]root@sheppard:~> disklabel ad0
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 260*)
b: 1048576 262144 swap # (Cyl. 260*- 1300*)
c: 60030369 0 unused 0 0 # (Cyl. 0 - 59553*)
f: 58719649 1310720 vinum # (Cyl. 1300*- 59553*)
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/ad0s1a 127023 114415 2447 98% /
mfs:29 507639 3 467025 0% /tmp
/dev/vinum/var 2032623 34395 1835619 2% /var
/dev/vinum/usr 4065262 1523152 2216890 41% /usr
/dev/vinum/u 22358577 4457713 16112178 22% /u
[1:10:323]root@sheppard:~> pstat -s
Device 1K-blocks Used Avail Capacity Type
/dev/ad0s1b 524160 0 524160 0% Interleaved
/dev/ad1s1b 524160 0 524160 0% Interleaved
Total 1048320 0 1048320 0%
Dave.
--
============================================================================
|David Gilbert, Velocet Communications. | Two things can only be |
|Mail: dgilbert@velocet.net | equal if and only if they |
|http://www.velocet.net/~dgilbert | are precisely opposite. |
=========================================================GLO================
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15123.57516.832209.744900>