Date: Tue, 09 Sep 2008 17:04:37 -0700 From: chris#@1command.com To: freebsd-stable@freebsd.org Subject: Re: bugged sysinstall, bsdlabel, zfs, gmirror - recept for disaster :) Message-ID: <20080909170437.zarijodao8gcggoo@webmail.1command.com> In-Reply-To: <48BE82F6.9070103@kkip.pl> References: <48BE82F6.9070103@kkip.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
Congratulations! Thanks for sharing, I'm guessing that you've probably prevented a few ppl from tripping on this by posting this. Me; I'm still waiting for: make zfs ad0s1, ad1s1, ad2s1 install. :-) Best wishes. --Chris Quoting Bartosz Stec <admin@kkip.pl>: > Hello there! > Here's my story, hopefully some of you won't follow my steps and > avoid some troubles :) > > Yesterday I've decided that's about time to test zfs functionality on > my home server PC (i386 FreeBSD 7.1-pre) . A couple of weeks ago I > bought new desktop PC (with SATA), so I had a bunch of PATA disks > from old one to use in server. Lucky me - there was 3 HDD at size > 40GB - RAIDZ was on approch! > > So after a thirty minutes I had a plan, and my server had 4 disks > connected - one 20GB with actual system (ad1), and three 40GB to > replace actual system (ad[023]). > Plan was simple: > > 1. csup freebsd-stable > 2. follow the tuning guide for zfs, rebuild world, kernel, and > follow system upgrade > 3. Reboot in single user mode > 4. fdisk new disks with sysinstall using one big slice for every disk > 5. bsdlabel every new disk with sysinstall using: 1GB for /, 512MB > for swap, and rest unused (for ZFS) > 6. gmirror -n -v -b round-robin boot ad0s1a ad2s1a ad3s1a > 7. newfs /dev/mirror/boot > 8. mount /dev/mirror/boot /mnt && cd /mnt > 9. dump -h 0 -L -f - -C 32 / | restore rf - > 10. zpool create tank raidz ad0s1d ad2s1d ad3s1d > 11. zfs create new cool filesystems :) > 12. dump | restore old ufs2 filesystem to new cool zfs filesystems :) > 13. changing mount points from tank/foo to /foo > 14. edit new fstab on mirror by replacing root mount point by "boot" > mirror, adding new swaps and remove ald ones and all fs now placed > on zpool > 15. power off system, detach ad1 and power on new system in mixed > gmirror - raidz environment. Yay! > > Well...it has almost works. Sysinstall screw it up. I was always too > lazy to read man bsdlabel, that's why I've been using this "nice" > tool for disk related tasks. Such a mistake! > Problem with labels created with sysinstal, is that it aks for a > mount point for every partition in slice. Well, in my case it was > unwanted behaviour, so on every disk I created first: > > a: / b: swap > c: none > d: /foo > > Then by using "M" key I removed mount points and saved changes with > "W". At this point everything seems ok. So I've added gmirror to > loader.conf and run "gmirror label -n -v -b round-robin boot ad0s1a > ad2s1a ad3s1a". Still ok until now. Next step - kldload geom_mirror. > Here's disaster! System became unresponsible and hangs after a while. > Reboot didn't help, just after gmirror module was loaded by kernel, > screen was flooded with messages: > > WARNING: Expected rawoffset 0, found 63 > > andy didn't boot. I've made system start only because an old drive > ad1 has no gmirror module added to loader.conf. So after reboot I've > cleared metadata on providers and made some another attempts, but > results were always the same. Finally I have found explanation for > this issue. Man bsdlabel says: > > /offset/ The offset of the start of the partition from the beginning = of > =09 the drive in sectors, or *** to have *bsdlabel* calculate > the correct > =09 offset to use (the end of the previous partition plus one, ign= or- > =09 ing partition `c'). For partition `c', *** will be interprete= d as > =09 an offset of 0. The first partition should start at offset 16= , > =09 because the first 16 sectors are reserved for metadata. > > > So proper labels for disks should be (and they are now): > > # /dev/ad0s1: > 8 partitions: > # size offset fstype [fsize bsize bps/cpg] > a: 2097152 16 4.2BSD 0 0 0 > b: 1048576 2097168 swap > c: 78156162 0 unused 0 0 # "raw" > part, don't edit > d: 75010418 3145744 unused 0 0 > > Problem was - Sysinstall has placed partition "a:" starting with > offset 0! This is what happens when you don't RTFM :) I assume that > this bug occured because I created mount point for root on ad[023]s1a > and removed it after, than saved label. It seems that GEOM framework > didn't expect this, neither maual for bsdlabel. I think that should > be fixed somehow. > Fortunately manually editing labels by "bsdlabel -e" wasn't so hard > as I expected. This is how I made everything back to normal: > > a: 1024M * 4.2BSD 0 0 0 > b: 512M * swap > c: 78156162 0 unused 0 0 # "raw" > part, don't edit > d: * * unused 0 0 > > After that, gmirror has stopped pissing me off, and I finished my > plan, as below: > > # zpool status > pool: tank > state: ONLINE > scrub: scrub completed with 0 errors on Wed Sep 3 10:10:07 2008 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > raidz1 ONLINE 0 0 0 > ad0s1d ONLINE 0 0 0 > ad2s1d ONLINE 0 0 0 > ad3s1d ONLINE 0 0 0 > > errors: No known data errors > > # gmirror status > Name Status Components > mirror/boot COMPLETE ad0s1a > ad2s1a > ad3s1a > > Good luck with ZFS everyone! :) And RTFM ;) > > -- > Bartosz Stec - specjalista ds. IT > > AUXILIA Sp=C3=B3=C5=82ka z o.o. > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080909170437.zarijodao8gcggoo>