From owner-freebsd-questions@FreeBSD.ORG Mon Feb 16 03:04:48 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 543A216A4CE for ; Mon, 16 Feb 2004 03:04:48 -0800 (PST) Received: from mail008.syd.optusnet.com.au (mail008.syd.optusnet.com.au [211.29.132.212]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65CB143D2D for ; Mon, 16 Feb 2004 03:04:47 -0800 (PST) (envelope-from tfrank@optushome.com.au) Received: from marvin.home.local (c211-28-241-189.eburwd5.vic.optusnet.com.au [211.28.241.189])i1GB4id12133 for ; Mon, 16 Feb 2004 22:04:45 +1100 Received: by marvin.home.local (Postfix, from userid 1001) id 71B5DD7; Mon, 16 Feb 2004 22:04:44 +1100 (EST) Date: Mon, 16 Feb 2004 22:04:44 +1100 From: Tony Frank To: freebsd-questions@freebsd.org Message-ID: <20040216110444.GA83416@marvin.home.local> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: vinum raid5 subdisks keep changing length? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Feb 2004 11:04:48 -0000 Hi all, Running vinum on RELENG_4. No changes to source, vinum loaded as a KLD as per instructions in handbook & man pages. As per the subject I am having what appears to be strange behaviour with vinum. After a number of teething problems I eventually setup a vinum root for my system which covers / swap /var and /usr. This works just fine, though I am yet to experiment with failure scenarios. However I also configured a raid5 volume and may be having issues with it. Currently just a test system and no obvious data loss but I have some concerns before I put it into production. Specifically I originally configured five subdisks (on different drives) with identical lengths and then combined them into a raid5 plex. This worked ok and I successfully initialised the plex and used newfs on it. Successfully able to read&write to the volume (1G/14G used at the moment) The original config looked as so: volume data plex name data.p0 org raid5 984s vol data sd name data.p0.s0 drive eightgig plex data.p0 len 8802864s driveoffset 6335201s plexoffset 0s sd name data.p0.s1 drive vinumdrive0 plex data.p0 len 8802864s driveoffset 265s plexoffset 984s sd name data.p0.s2 drive vinumdrive1 plex data.p0 len 8802864s driveoffset 265s plexoffset 1968s sd name data.p0.s3 drive vinumdrive2 plex data.p0 len 8802864s driveoffset 265s plexoffset 2952s sd name data.p0.s4 drive vinumdrive3 plex data.p0 len 8802864s driveoffset 265s plexoffset 3936s Now after each system reboot I see messages on system console from vinum. (they are not logged in vinum_history or messages) (hand transcript): vinum: updating configuration from /dev/ad0s1h #(repeated for all 6 drives) vinum: removing 2560 blocks of partial stripe at the end of data.p0 vinum: data.p0 must have equal sized subdisks vinum: data.p0 must have equal sized subdisks vinum: data.p0 must have equal sized subdisks vinum: data.p0 must have equal sized subdisks Correcting length of data.p0: was 35191776, is 35197088 vinum: using volume root for root device Mounting root from ufs:/dev/vinum/root A "vinum printconfig" now shows difference values for the subdisks. This value has changed after each reboot. Current relevant parts of 'printconfig' are: volume data plex name data.p0 org raid5 984s vol data sd name data.p0.s0 drive eightgig plex data.p0 len 8799978s driveoffset 6335201s plexoffset 0s sd name data.p0.s1 drive vinumdrive0 plex data.p0 len 8799486s driveoffset 265s plexoffset 984s sd name data.p0.s2 drive vinumdrive1 plex data.p0 len 8799158s driveoffset 265s plexoffset 1968s sd name data.p0.s3 drive vinumdrive2 plex data.p0 len 8798420s driveoffset 265s plexoffset 2952s sd name data.p0.s4 drive vinumdrive3 plex data.p0 len 8797600s driveoffset 265s plexoffset 3936s Each subdisk len is now a different size which is of some concern to me, especially as the specific len values change after each reboot. Some other perhaps 'relevant' details: uname -a output: FreeBSD raider.home.local 4.9-STABLE FreeBSD 4.9-STABLE #1: Sat Feb 7 01:50:58 EST 2004 tony@marvin.home.local:/data/ad3/obj/usr/src/sys/RAIDER i386 Tail of /var/log/messages output: Feb 16 20:14:42 raider /kernel: da2: 20.000MB/s transfers (10.000MHz, offset 8, 16bit), Tagged Queueing Enabled Feb 16 20:14:42 raider /kernel: da2: 4303MB (8813870 512 byte sectors: 255H 63S/T 548C) Feb 16 20:14:42 raider /kernel: Mounting root from ufs:/dev/vinum/root Feb 16 20:14:43 raider savecore: no core dump Feb 16 20:14:37 raider ntpd[115]: ntpd 4.1.0-a Sat Feb 7 00:06:01 EST 2004 (1) Feb 16 20:14:37 raider ntpd[115]: kernel time discipline status 2040 Feb 16 20:14:54 raider su: tony to root on /dev/ttyp0 Feb 16 20:15:03 raider su: tony to root on /dev/ttyp0 Feb 16 20:19:10 raider ntpd[115]: time set 0.007497 s /var/log/vinum_history extract (shows init and subsequent prints) 8 Feb 2004 13:38:58.679668 attach data.p0 data 8 Feb 2004 13:39:01.616409 lv 8 Feb 2004 13:39:02.891476 lv -r 8 Feb 2004 13:39:04.099017 lv 8 Feb 2004 13:39:09.498986 lv -r 8 Feb 2004 13:39:17.697480 init data.p0 8 Feb 2004 13:40:24.011586 lv -r 8 Feb 2004 13:41:23.409672 lv -r 8 Feb 2004 13:42:06.320951 lv -r 8 Feb 2004 13:43:17.928172 *** vinum started *** 8 Feb 2004 13:43:17.943327 lv -r 8 Feb 2004 13:44:26.669794 *** vinum started *** 8 Feb 2004 13:44:26.675611 lv -r 8 Feb 2004 13:46:06.942916 *** vinum started *** 8 Feb 2004 13:46:06.945251 lv -r 8 Feb 2004 13:49:36.588518 *** vinum started *** 8 Feb 2004 13:49:36.590516 lv -r 8 Feb 2004 13:53:32.719039 *** vinum started *** # Rebooted here 8 Feb 2004 15:26:57.032467 *** vinum started *** 8 Feb 2004 15:26:57.053653 list 8 Feb 2004 15:27:00.942329 *** vinum started *** 8 Feb 2004 15:27:00.943709 printconfig 8 Feb 2004 15:27:22.021212 *** vinum started *** 8 Feb 2004 15:27:22.022626 printconfig after_reboot.config 8 Feb 2004 15:29:53.377752 *** vinum started *** 8 Feb 2004 15:29:53.379181 l 8 Feb 2004 15:31:16.065588 *** vinum started *** 8 Feb 2004 15:31:16.067022 ld # Rebooted here 11 Feb 2004 23:15:51.299376 *** vinum started *** 11 Feb 2004 23:15:51.317598 printconfig latest_reboot.config # Rebooted here 16 Feb 2004 20:15:14.296975 *** vinum started *** 16 Feb 2004 20:15:14.323960 printconfig boot3.config 16 Feb 2004 20:37:55.783001 *** vinum started *** 16 Feb 2004 20:37:55.797851 list Current output of "vinum list": 12 drives: D eightgig State: up Device /dev/ad0h Avail: 8056/8055 MB (100%) D threegig State: up Device /dev/ad2h Avail: 3093/3093 MB (100%) D vinumdrive0 State: up Device /dev/da0h Avail: 4298/4298 MB (100%) D vinumdrive1 State: up Device /dev/da1h Avail: 4298/4298 MB (100%) D vinumdrive2 State: up Device /dev/da2h Avail: 4298/4298 MB (100%) D vinumdrive3 State: up Device /dev/da3h Avail: 4298/4298 MB (100%) 6 volumes: V root State: up Plexes: 2 Size: 256 MB V swap State: up Plexes: 2 Size: 255 MB V usr State: up Plexes: 2 Size: 2048 MB V var State: up Plexes: 2 Size: 533 MB V data State: up Plexes: 1 Size: 16 GB V junk State: up Plexes: 1 Size: 664 MB 10 plexes: P root.p0 C State: up Subdisks: 1 Size: 256 MB P swap.p0 C State: up Subdisks: 1 Size: 255 MB P usr.p0 C State: up Subdisks: 1 Size: 2048 MB P var.p0 C State: up Subdisks: 1 Size: 533 MB P root.p1 C State: up Subdisks: 1 Size: 256 MB P swap.p1 C State: up Subdisks: 1 Size: 255 MB P usr.p1 C State: up Subdisks: 1 Size: 2048 MB P var.p1 C State: up Subdisks: 1 Size: 533 MB P data.p0 R5 State: up Subdisks: 5 Size: 16 GB P junk.p0 C State: up Subdisks: 1 Size: 664 MB 14 subdisks: S root.p0.s0 State: up PO: 0 B Size: 256 MB S swap.p0.s0 State: up PO: 0 B Size: 255 MB S usr.p0.s0 State: up PO: 0 B Size: 2048 MB S var.p0.s0 State: up PO: 0 B Size: 533 MB S root.p1.s0 State: up PO: 0 B Size: 256 MB S swap.p1.s0 State: up PO: 0 B Size: 255 MB S usr.p1.s0 State: up PO: 0 B Size: 2048 MB S var.p1.s0 State: up PO: 0 B Size: 533 MB S data.p0.s0 State: up PO: 0 B Size: 4296 MB S data.p0.s1 State: up PO: 492 kB Size: 4296 MB S data.p0.s2 State: up PO: 984 kB Size: 4296 MB S data.p0.s3 State: up PO: 1476 kB Size: 4296 MB S data.p0.s4 State: up PO: 1968 kB Size: 4295 MB S junk.p0.s0 State: up PO: 0 B Size: 664 MB Any assistance is appreciated, Tony