From owner-freebsd-questions Sat Jul 20 22:40:24 1996 Return-Path: owner-questions Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id WAA14205 for questions-outgoing; Sat, 20 Jul 1996 22:40:24 -0700 (PDT) Received: from salsa.habaneros.com ([207.34.140.99]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id WAA14193 for ; Sat, 20 Jul 1996 22:40:21 -0700 (PDT) Received: from jalapeno.habaneros.com (jalapeno.habaneros.com [207.34.140.98]) by salsa.habaneros.com (8.6.12/8.6.12) with SMTP id WAA01513; Sat, 20 Jul 1996 22:26:35 -0700 Received: by jalapeno.habaneros.com with Microsoft Mail id <01BB768B.BCD618E0@jalapeno.habaneros.com>; Sat, 20 Jul 1996 22:35:22 -0700 Message-ID: <01BB768B.BCD618E0@jalapeno.habaneros.com> From: "Neil C. Jensen" To: "'Satoshi Asami'" Cc: "questions@freebsd.org" Subject: RE: 2.1.5R and ccd questions Date: Sat, 20 Jul 1996 22:35:15 -0700 Encoding: 277 TEXT Sender: owner-questions@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Sorry for the long post, I thought I'd include the info below for reference... I've been attempting to get disk mirroring working following the directions below. I can make it to the point where I create the 1-disk ccd, however, I have troubles disklabeling it. When I attempt "disklabel -w -r ccd0 auto" I get back "disklabel: open(): No such file or directory" ---------- From: Satoshi Asami[SMTP:asami@cs.berkeley.edu] Sent: Friday, July 19, 1996 9:33 AM To: njensen@salsa.habaneros.com Cc: questions@freebsd.org Subject: Re: 2.1.5R and ccd questions * Just finished installing 2.1.5R (went very smoothly, thanks * Team FreeBSD!) and thought I'd try disk mirroring using * ccd. I want to mirror the contents of my entire file system * so that I can reboot from one of the disk if the other fails. * * I have read the documentation; ccd home page and man * pages and am still unclear on exactly what I need to do. * * I have two idential SCSI drives, sd0 & sd1. Right now the OS * is installed on sd0 and I have done nothing to sd1. I'm afraid you are somewhat misunderstanding the way ccd mirroring works. Despite the name, ccd actually concatenates disk partitions, not disks themselves. The result of concatenation is a meta-device that looks like a disk, which you can further partition up by editing the disklabel. As far as I can tell, there are two ways to accomplish what you want to do. One is to mirror every partition you want as a separate ccd, and the other is to first create a mega-partition that is the size of everything except root & swap (more on this later) and divide this thing up into pieces. I'll give you an example on how to do the latter. The strategy here is to first create a 1-disk ccd on sd1, partition it up, copy everything you need onto it from sd0, then set up a 2-disk ccd with mirroring using both disks. Beware, I'm not sure at all if this will work, so make a backup of your system before you try this! * 1. recompile a new kernel with 'pseudo-device ccd 4' ? Yes. The ccd lkm unfortunately doesn't work right now. * 2. use dd to copy the contents of sd0 to sd1? (do I need * to use disklabel,partition first?) If we were just interested in copying things from one disk to another identical one, I'd say "dd if=/dev/rsd0 of=/dev/rsd1" should probably work, although I have never copied anything larger than a single partition myself. However, there is a little problem here: a filesystem on a regular partition is not the same as that on a ccd on the same partition. The ccd partition needs its own disklabel and such, so the start of the filesystem is shifted back somewhat. What I recommend would be like: (1) dd if=/dev/rsd0 of=/dev/rsd1 bs=512 count=100 This should copy the fdisk table and disklabel to sd1. The number "100" is arbirtary, I'm not sure how many is needed here. (2) disklabel -e sd1 Edit the disklabel, subtract the size of "c" from the offset of "e" (or "d" or whatever the first partition after "c" is), and use that as the size of the new "e" partition. For instance, if your partition table looked like this: 7 partitions: # size offset fstype [fsize bsize bps/cpg] a: 65536 0 4.2BSD 1024 8192 0 # (Cyl. 0 - 4*) b: 262144 65536 swap # (Cyl. 4*- 20*) c: 4199697 0 unused 0 0 # (Cyl. 0 - 261*) e: 32768 327680 4.2BSD 1024 8192 0 # (Cyl. 20*- 22*) f: 196608 360448 4.2BSD 1024 8192 0 # (Cyl. 22*- 34*) g: 3642641 557056 4.2BSD 1024 8192 0 # (Cyl. 34*- 261*) You should change it to: 5 partitions: # size offset fstype [fsize bsize bps/cpg] a: 65536 0 4.2BSD 1024 8192 0 # (Cyl. 0 - 4*) b: 262144 65536 swap # (Cyl. 4*- 20*) c: 4199697 0 unused 0 0 # (Cyl. 0 - 261*) e: 3872017 327680 4.2BSD 1024 8192 0 # (Cyl. 20*- 22*) ^^^^^^^ = 4199697 - 327680 (The number of partitions is actually the index of the last partition.) Also, don't worry about the cylinder numbers, they are comments and disklabel will fix them up for you. * 3. create the /etc/ccd.conf file with CCDF_MIRROR flag? Yes. This is what it eventually should look like: ccd0 128 CCDF_MIRROR /dev/sd0e /dev/sd1e ^^^ The interleave size shouldn't really matter, we are mirroring only two disks.... * 4. now run ccdconfig, creating device ccd0? ccdconfig won't create the devices. Use /dev/MAKEDEV to create them. "sh ./MAKEDEV ccd0" will create /dev/ccd0* and /dev/rccd0*, etc. ccdconfig is automatically run from /etc/rc if you have a /etc/ccd.conf file on your system. Now, this is what you need to do now. Remember we need to copy everything onto the 1-disk ccd on sd1 first. (1) create /etc/ccd.conf with the entry ccd0 128 0 /dev/sd1e (2) create a 1-disk ccd ccdconfig -Cv (3) edit the disklabel disklabel -wr ccd0 auto disklabel -e ccd0 Now, remember how the disklabel looked like before? # size offset fstype [fsize bsize bps/cpg] ... e: 32768 327680 4.2BSD 1024 8192 0 # (Cyl. 20*- 22*) f: 196608 360448 4.2BSD 1024 8192 0 # (Cyl. 22*- 34*) g: 3642641 557056 4.2BSD 1024 8192 0 # (Cyl. 34*- 261*) Take these numbers, subtract "327680" from all the offsets, and write that as the new disklabel: # size offset fstype [fsize bsize bps/cpg] ... e: 32768 0 4.2BSD 1024 8192 0 # (Cyl. 20*- 22*) f: 196608 32768 4.2BSD 1024 8192 0 # (Cyl. 22*- 34*) g: 3642641 229376 4.2BSD 1024 8192 0 # (Cyl. 34*- 261*) You will also need to decrement the size of the last partition to make the whole thing fit into the size of "c". I'm not exactly sure what it's going to be, it will be a few dozen sectors less than the 3872017 we gave as the size of the whole thing. So, make sure that size(g) + offset(g) = size(c). (4) Create filesystems newfs /dev/ccd0e newfs /dev/ccd0f newfs /dev/ccd0g (5) Mount them Assuming these three partitions are mounted as /var, /usr and /a on sd0: mkdir /mnt/var /mnt/usr /mnt/a mount -o async /dev/ccd0e /mnt/var mount -o async /dev/ccd0f /mnt/usr mount -o async /dev/ccd0g /mnt/a The "-o async" mounts the drives asynchronously, which is dangerous for normal use but should be ok for this situation. It's much faster. (6) Copy the stuff over cd /var; find -x . | cpio -pmV /mnt/var cd /usr; find -x . | cpio -pmV /mnt/usr cd /a; find -x . | cpio -pmV /mnt/a * 5. is ccd0 now my device with the original filesystem installed, * except now I am mirrored? Not yet. :) Now we need to boot with root/swap from sd0 and the rest from ccd0 (sd1), and then copy the stuff back to sd0. It will be like this: (1) vi /etc/fstab Change "/dev/sd0e" to "/dev/ccd0e", etc. Don't touch the references to sd0a and sd0b! Actually, you can add a new line so that it says /dev/sd0b none swap sw 0 0 /dev/sd1b none swap sw 0 0 if you want to use the "b" partition of both disks as swap. (2) reboot The system should now come up with something like this: % mount /dev/sd0a on / (local) /dev/ccd0e on /var (local) /dev/ccd0f on /usr (local) /dev/ccd0g on /a (local) procfs on /proc (local) In addition, "pstat -s" will show both sd0b and sd1b if you added sd1b as swap. (3) edit sd0's disklabel Do a "disklabel -e sd0" and make it look identical to sd1. Now it should have sd0a, sd0b, sd0c (the whole disk) and one big partition (sd0e in our example). (4) copy stuff over to sd0 dd if=/dev/rsd1e of=/dev/rsd0e bs=65536 (5) vi /etc/ccd.conf Change it to ccd0 128 CCDF_MIRROR /dev/sd0e /dev/sd1e (6) reboot Pray it works. ;) * 6. by default, I think the system will boot from sd0. If sd0 fails, * how do I get the system to boot from sd1, will it automatically * check for a valid system there? Unfortunately, I don't think you can boot from a concatenated disk. The ccdconfig utility is run from /etc/rc, after the root partition is mounted (read-only), so any filesystem other than root should work fine. (You can even swap to a ccd if you want, the "swapon -a" is after ccdconfig in /etc/rc. ;) Of course you probably want to keep a copy of the root partition as well, so maybe you can run a find -x | cpio from cron or something. I think the easiest way to be able to boot from sd1 when sd0 goes bad would be to install the boot manager. That way you can select "disk 2" to boot from the second disk easily. Assuming you have maintained a copy of sd0a in sd1a, you should be able to boot from there. Of course, if sd0 goes totally bad and you have to take it out, sd1 will now become sd0 so it will boot from there. ;) Beware, ccd doesn't automatically check if a disk is valid, so you will have to boot the system with -s and edit the ccd.conf file to use only one disk if one of the two goes bad. Also, if you did the find -x | cpio from cron, fstab will say root is sd0a so if the (bad) sd0 is still in the system and you are booting from sd1, you have to change it to sd1a (and take out the sd0b reference as well). * 7. how do I test if ccd is working? Write something to /usr/tmp, and see lights on both disks flash. :) * Thanks in advance for your help. I will compile any responses I * get for possible inclusion in the handbook. Well, I'm not sure if this was the kind of answer you wanted, but this is what I *think* should work. Again, no guarantees, and if you don't understand the phrase "boot the system with -s", you may not want to go this far. ;) Satoshi