From owner-freebsd-current@FreeBSD.ORG Wed Jun 16 14:32:44 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC54E16A4CE; Wed, 16 Jun 2004 14:32:44 +0000 (GMT) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5080E43D39; Wed, 16 Jun 2004 14:32:42 +0000 (GMT) (envelope-from craig@xfoil.gank.org) Received: from owen1492.uf.corelab.com (pix.corelab.com [12.45.169.2]) by ion.gank.org (mail) with ESMTP id 4B68D2C4C1; Wed, 16 Jun 2004 09:31:14 -0500 (CDT) From: Craig Boston To: freebsd-current@freebsd.org Date: Wed, 16 Jun 2004 09:31:10 -0500 User-Agent: KMail/1.6.2 References: <20040615191702.GA56695@gromit.dlib.vt.edu> <20040615235935.GA57617@gromit.dlib.vt.edu> <200406152218.14770.craig@xfoil.gank.org> In-Reply-To: <200406152218.14770.craig@xfoil.gank.org> MIME-Version: 1.0 Content-Disposition: inline X-UID: 616 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200406160931.10205.craig@xfoil.gank.org> cc: Paul Mather cc: Lukas Ertl Subject: Re: HEADSUP: geom_vinum committed X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2004 14:32:45 -0000 ARGH! Sorry, did it again. Paul and Lukas, please disregard the duplicate. Need to set a dummy account as my default... (beware, long email) Well, we're getting closer. With the commits from last night I was able to get it to mount to root partition. Unfortunately, it seems all of the plexes are not getting detected in time, so the configuration is slightly weird (details below). I didn't notice this at first and ended up with a corrupted root partition because one of the disks in the mirror was out of sync, eventually leading to a ufs_dirbad panic. Fortunately, using regular vinum, I was able to shut down all but one of them, run an fsck, and get things up and running again. Excerpts from dmesg: acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0% ad0: 9541MB [19386/16/63] at ata0-master UDMA33 acd0: DVDROM at ata0-slave UDMA33 ad2: 9541MB [19386/16/63] at ata1-master UDMA33 Waiting 7 seconds for SCSI devices to settle FOO: sd root.p1.s0 is stale FOO: sd swap.p0.s1 is up FOO: sd usr.p0.s1 is up FOO: sd var.p0.s1 is up FOO: sd tmp.p0.s1 is up (pauses for 7 seconds here) SMP: AP CPU #1 Launched! da0 at ahc0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged Queueing Enabled da0: 8678MB (17773500 512 byte sectors: 255H 63S/T 1106C) cd0 at ahc0 bus 0 target 1 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: 10.000MB/s transfers (10.000MHz, offset 15) cd0: Attempt to query device size failed: NOT READY, Medium not present FOO: sd root.p2.s0 is up FOO: sd swap.p0.s2 is up FOO: sd usr.p0.s2 is up FOO: sd var.p0.s2 is up FOO: sd tmp.p0.s2 is up For some reason, the objects on ad0 don't get detected during bootup -- those are the only FOO: lines that appear. Output of gvinum list: D drive3 State: up /dev/da0s1e A: 0/8675 MB (0%) D drive2 State: up /dev/ad2s1e A: 0/8675 MB (0%) D drive1 State: up /dev/ad0s1e A: 8675/8675 MB (100%) 5 volumes: V root State: up Plexes: 2 Size: 128 MB V swap State: up Plexes: 1 Size: 0 B V usr State: up Plexes: 1 Size: 0 B V var State: up Plexes: 1 Size: 0 B V tmp State: up Plexes: 1 Size: 0 B 7 plexes: P root.p0 C State: up Subdisks: 0 Size: 0 B P root.p1 C State: up Subdisks: 1 Size: 128 MB P root.p2 C State: up Subdisks: 1 Size: 128 MB P swap.p0 R5 State: up Subdisks: 2 Size: 512 MB P usr.p0 R5 State: up Subdisks: 2 Size: 7651 MB P var.p0 R5 State: up Subdisks: 2 Size: 256 MB P tmp.p0 R5 State: up Subdisks: 2 Size: 128 MB 15 subdisks: S root.p0.s0 State: up D: drive1 Size: 128 MB S root.p1.s0 State: up D: drive2 Size: 128 MB S root.p2.s0 State: up D: drive3 Size: 128 MB S swap.p0.s0 State: up D: drive1 Size: 512 MB S swap.p0.s1 State: up D: drive2 Size: 512 MB S swap.p0.s2 State: up D: drive3 Size: 512 MB S usr.p0.s0 State: up D: drive1 Size: 7651 MB S usr.p0.s1 State: up D: drive2 Size: 7651 MB S usr.p0.s2 State: up D: drive3 Size: 7651 MB S var.p0.s0 State: up D: drive1 Size: 256 MB S var.p0.s1 State: up D: drive2 Size: 256 MB S var.p0.s2 State: up D: drive3 Size: 256 MB S tmp.p0.s0 State: up D: drive1 Size: 128 MB S tmp.p0.s1 State: up D: drive2 Size: 128 MB S tmp.p0.s2 State: up D: drive3 Size: 128 MB Note the Plexes: 2 and the strange Subdisks: 0. gvinum printconfig confirms that all the linkages for anything on drive1 are wrong: drive drive3 device da0s1e drive drive2 device ad2s1e drive drive1 device ad0s1e volume root volume swap volume usr volume var volume tmp plex name root.p0 org concat plex name root.p1 org concat vol root plex name root.p2 org concat vol root plex name swap.p0 org raid5 256s vol swap plex name usr.p0 org raid5 256s vol usr plex name var.p0 org raid5 256s vol var plex name tmp.p0 org raid5 256s vol tmp sd name root.p0.s0 drive drive1 len 262144s driveoffset 265s sd name root.p1.s0 drive drive2 len 262144s driveoffset 265s plex root.p1 plexoffset 0s sd name root.p2.s0 drive drive3 len 262144s driveoffset 265s plex root.p2 plexoffset 0s sd name swap.p0.s0 drive drive1 len 1048576s driveoffset 262409s sd name swap.p0.s1 drive drive2 len 1048576s driveoffset 262409s plex swap.p0 plexoffset 256s sd name swap.p0.s2 drive drive3 len 1048576s driveoffset 262409s plex swap.p0 plexoffset 512s sd name usr.p0.s0 drive drive1 len 15669248s driveoffset 1310985s sd name usr.p0.s1 drive drive2 len 15669248s driveoffset 1310985s plex usr.p0 plexoffset 256s sd name usr.p0.s2 drive drive3 len 15669248s driveoffset 1310985s plex usr.p0 plexoffset 512s sd name var.p0.s0 drive drive1 len 524288s driveoffset 16980233s sd name var.p0.s1 drive drive2 len 524288s driveoffset 16980233s plex var.p0 plexoffset 256s sd name var.p0.s2 drive drive3 len 524288s driveoffset 16980233s plex var.p0 plexoffset 512s sd name tmp.p0.s0 drive drive1 len 262144s driveoffset 17504521s sd name tmp.p0.s1 drive drive2 len 262144s driveoffset 17504521s plex tmp.p0 plexoffset 256s sd name tmp.p0.s2 drive drive3 len 262144s driveoffset 17504521s plex tmp.p0 plexoffset 512s Lots of missing plex and vol entries. At least the driveoffsets seem to be correct. I was unable to mount any of the raid-5 volumes, I'm guessing as a raid5 plex with only 2 subdisks doesn't make a whole lot of sense. For comparison, here is the printconfig from non-GEOMified vinum: drive drive1 device /dev/ad0s1e drive drive2 device /dev/ad2s1e drive drive3 device /dev/da0s1e volume tmp volume var volume usr volume swap volume root plex name tmp.p0 org raid5 256s vol tmp plex name var.p0 org raid5 256s vol var plex name usr.p0 org raid5 256s vol usr plex name swap.p0 org raid5 256s vol swap plex name root.p2 org concat vol root plex name root.p1 org concat vol root plex name root.p0 org concat vol root sd name tmp.p0.s2 drive drive3 plex tmp.p0 len 262144s driveoffset 17504521s plexoffset 512s sd name tmp.p0.s1 drive drive2 plex tmp.p0 len 262144s driveoffset 17504521s plexoffset 256s sd name tmp.p0.s0 drive drive1 plex tmp.p0 len 262144s driveoffset 17504521s plexoffset 0s sd name var.p0.s2 drive drive3 plex var.p0 len 524288s driveoffset 16980233s plexoffset 512s sd name var.p0.s1 drive drive2 plex var.p0 len 524288s driveoffset 16980233s plexoffset 256s sd name var.p0.s0 drive drive1 plex var.p0 len 524288s driveoffset 16980233s plexoffset 0s sd name usr.p0.s2 drive drive3 plex usr.p0 len 15669248s driveoffset 1310985s plexoffset 512s sd name usr.p0.s1 drive drive2 plex usr.p0 len 15669248s driveoffset 1310985s plexoffset 256s sd name usr.p0.s0 drive drive1 plex usr.p0 len 15669248s driveoffset 1310985s plexoffset 0s sd name swap.p0.s2 drive drive3 plex swap.p0 len 1048576s driveoffset 262409s plexoffset 512s sd name swap.p0.s1 drive drive2 plex swap.p0 len 1048576s driveoffset 262409s plexoffset 256s sd name swap.p0.s0 drive drive1 plex swap.p0 len 1048576s driveoffset 262409s plexoffset 0s sd name root.p2.s0 drive drive3 plex root.p2 len 262144s driveoffset 265s plexoffset 0s sd name root.p1.s0 drive drive2 plex root.p1 len 262144s driveoffset 265s plexoffset 0s sd name root.p0.s0 drive drive1 plex root.p0 len 262144s driveoffset 265s plexoffset 0s Again, thanks for all the work you've done on this! If there's anything I can do to help, I'd be glad to volunteer the time :) Craig