From owner-freebsd-questions@FreeBSD.ORG Wed Jul 28 09:55:52 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BB00106567A; Wed, 28 Jul 2010 09:55:52 +0000 (UTC) (envelope-from vince@unsane.co.uk) Received: from unsane.co.uk (unsane-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:110::2]) by mx1.freebsd.org (Postfix) with ESMTP id AE8A78FC12; Wed, 28 Jul 2010 09:55:51 +0000 (UTC) Received: from vhoffman.lon.namesco.net (150.117-84-212.staticip.namesco.net [212.84.117.150]) (authenticated bits=0) by unsane.co.uk (8.14.4/8.14.4) with ESMTP id o6S9tm04001454 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 28 Jul 2010 10:55:49 +0100 (BST) (envelope-from vince@unsane.co.uk) Message-ID: <4C4FFEA3.7060307@unsane.co.uk> Date: Wed, 28 Jul 2010 10:55:47 +0100 From: Vincent Hoffman User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-questions@freebsd.org, geom@freebsd.org References: In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: gmultipath woes X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 09:55:52 -0000 On 28/07/2010 08:17, Corey Smith wrote: > Has anyone tried gmultipath under 8.1-RELEASE/amd64 yet? This is my > first attempt at multipathing so it is possible I'm doing something > wrong but anyways: > > # uname -a > FreeBSD host 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 > UTC 2010 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC > amd64 > > # dmesg | egrep '^(isp|da1|da2)' > isp0: port 0xd800-0xd8ff mem > 0xdf4f8000-0xdf4fbfff irq 38 at device 0.0 on pci6 > isp1: port 0xdc00-0xdcff mem > 0xdf4fc000-0xdf4fffff irq 45 at device 0.1 on pci6 > da1 at isp0 bus 0 scbus1 target 0 lun 0 > da1: Fixed Direct Access SCSI-4 device > da1: 400.000MB/s transfers > da1: Command Queueing enabled > da1: 2097152MB (4294967296 512 byte sectors: 255H 63S/T 267349C) > da2 at isp1 bus 0 scbus2 target 0 lun 0 > da2: Fixed Direct Access SCSI-4 device > da2: 400.000MB/s transfers > da2: Command Queueing enabled > da2: 2097152MB (4294967296 512 byte sectors: 255H 63S/T 267349C) > > # camcontrol devlist > at scbus0 target 0 lun 0 (pass0,da0) > at scbus0 target 8 lun 0 (pass1,ses0) > at scbus1 target 0 lun 0 (pass2,da1) > at scbus2 target 0 lun 0 (pass3,da2) > > # camcontrol inquiry da1 -S > 870405180007 > # camcontrol inquiry da2 -S > 870405180007 > > # gmultipath label -v FRED /dev/da1 /dev/da2 > gmultipath: Unknown verb add > > # gmultipath status > Name Status Components > multipath/FRED N/A da1 > > # gmultipath list > Geom name: FRED > Providers: > 1. Name: multipath/FRED > Mediasize: 2199023255040 (2.0T) > Sectorsize: 512 > Mode: r0w0e0 > Consumers: > 1. Name: da1 > Mediasize: 2199023255552 (2.0T) > Sectorsize: 512 > Mode: r0w0e0 > > ------ EOT ------ > > I'm wondering what is going on with the "Unknown verb add" and why is > it only listing da1 in the components? Does the disk need to be in a > particular state (unpartitioned, glabel, etc...) or does it not > I dont have any storage with multiple paths handy other than creating md devices, however doing that as you have (and as the manpage says) gives the same results. [root@ostracod /mnt]# mdconfig -a -t vnode -f /scratch/test.dsk -u 12 [root@ostracod /mnt]# mdconfig -a -t vnode -f /scratch/test.dsk -u 13 [root@ostracod /mnt]# sysctl kern.geom.multipath.debug=1 kern.geom.multipath.debug: 1 -> 1 [root@ostracod /mnt]# gmultipath label -v TEST /dev/md12 /dev/md13 gmultipath: Unknown verb add [root@ostracod /mnt]# gmultipath list Geom name: TEST Providers: 1. Name: multipath/TEST Mediasize: 52428288 (50M) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: md12 Mediasize: 52428800 (50M) Sectorsize: 512 Mode: r0w0e0 [root@ostracod /mnt]# gmultipath status Name Status Components multipath/TEST N/A md12 However doing echo 1 > /dev/md13 made geom re-taste the device so I get [root@ostracod /mnt]# gmultipath list Geom name: TEST Providers: 1. Name: multipath/TEST Mediasize: 52428288 (50M) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: md12 Mediasize: 52428800 (50M) Sectorsize: 512 Mode: r0w0e0 2. Name: md13 Mediasize: 52428800 (50M) Sectorsize: 512 Mode: r0w0e0 Also doing mdconfig -a -t vnode -f /scratch/test.dsk -u 12 gmultipath load gmultipath label -v FRED /dev/md12 mdconfig -a -t vnode -f /scratch/test.dsk -u 13 gave me Jul 28 10:32:43 ostracod kernel: md12 is not MULTIPATH Jul 28 10:32:43 ostracod kernel: MULTIPATH: FRED/12a41222-9a2b-11df-aa80-00012e273257 Jul 28 10:32:43 ostracod kernel: GEOM_MULTIPATH: adding md12 to FRED/12a41222-9a2b-11df-aa80-00012e273257 Jul 28 10:32:43 ostracod kernel: GEOM_MULTIPATH: md12 now active path in FRED Jul 28 10:32:43 ostracod kernel: multipath/FRED is not MULTIPATH Jul 28 10:33:37 ostracod kernel: MULTIPATH: FRED/12a41222-9a2b-11df-aa80-00012e273257 Jul 28 10:33:37 ostracod kernel: GEOM_MULTIPATH: adding md13 to FRED/12a41222-9a2b-11df-aa80-00012e273257 [root@ostracod /mnt]# gmultipath list Geom name: FRED Providers: 1. Name: multipath/FRED Mediasize: 52428288 (50M) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: md12 Mediasize: 52428800 (50M) Sectorsize: 512 Mode: r0w0e0 2. Name: md13 Mediasize: 52428800 (50M) Sectorsize: 512 Mode: r0w0e0 [root@ostracod /mnt]# gpart create -s gpt multipath/FRED multipath/FRED created [root@ostracod /mnt]# gpart add -s 10M -t freebsd-ufs multipath/FRED multipath/FREDp1 added [root@ostracod /mnt]# newfs /dev/multipath/FREDp1 /dev/multipath/FREDp1: 10.0MB (20480 sectors) block size 16384, fragment size 2048 using 4 cylinder groups of 2.52MB, 161 blks, 384 inodes. super-block backups (for fsck -b #) at: 160, 5312, 10464, 15616 [root@ostracod /mnt]# mount /dev/multipath/FREDp1 /mnt/foo/ [root@ostracod /mnt] so it all seems to works but not as expected/documented. Also I get these various errors in /var/log/messages Jul 28 10:41:17 ostracod kernel: GEOM: md13: the secondary GPT table is corrupt or invalid. Jul 28 10:41:17 ostracod kernel: GEOM: md13: using the primary only -- recovery suggested. Jul 28 10:41:17 ostracod kernel: GEOM: md12: the secondary GPT table is corrupt or invalid. Jul 28 10:41:17 ostracod kernel: GEOM: md12: using the primary only -- recovery suggested. Jul 28 10:41:17 ostracod kernel: multipath/FRED is not MULTIPATH Jul 28 10:42:00 ostracod kernel: multipath/FREDp1 is not MULTIPATH Jul 28 10:42:00 ostracod kernel: multipath/FRED is not MULTIPATH Jul 28 10:42:00 ostracod kernel: gptid/5ed86f82-9a2c-11df-aa80-00012e273257 is not MULTIPATH Jul 28 10:42:14 ostracod kernel: GEOM: md13: the secondary GPT table is corrupt or invalid. Jul 28 10:42:14 ostracod kernel: GEOM: md13: using the primary only -- recovery suggested. Jul 28 10:42:14 ostracod kernel: GEOM: md12: the secondary GPT table is corrupt or invalid. Jul 28 10:42:14 ostracod kernel: GEOM: md12: using the primary only -- recovery suggested. Jul 28 10:42:14 ostracod kernel: multipath/FRED is not MULTIPATH Jul 28 10:42:14 ostracod kernel: multipath/FREDp1 is not MULTIPATH Jul 28 10:42:14 ostracod kernel: md13p1 is not MULTIPATH Jul 28 10:42:14 ostracod kernel: md12p1 is not MULTIPATH Jul 28 10:42:14 ostracod kernel: ufsid/4c4ffb754e3cdb5c is not MULTIPATH Jul 28 10:42:14 ostracod kernel: gptid/5ed86f82-9a2c-11df-aa80-00012e273257 is not MULTIPATH so I'd say that gmultipath needs some attention, and would guess using gpt with geom_multipath isnt a great idea I'd file a pr if I were you. I'm cross posting to geom@ in case I'm being a muppet somewhere though. Vince > matter? > > -Corey Smith > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" >