Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 04 Oct 2009 16:22:30 -0500
From:      Aaron Hurt <aaron@goflexitllc.com>
To:        Artem Belevich <fbsdlist@src.cx>
Cc:        freebsd-fs@freebsd.org, Pawel Jakub Dawidek <pjd@freebsd.org>
Subject:   Re: Help needed! ZFS I/O error recovery?
Message-ID:  <4AC91216.9070200@goflexitllc.com>
In-Reply-To: <ed91d4a80910041335o5b23019du39faa96ba3590a2d@mail.gmail.com>
References:  <683849754.20091001110503@pyro.de>	 <20091004174746.GF1660@garage.freebsd.pl>	 <4AC8FFE9.90606@goflexitllc.com> <ed91d4a80910041335o5b23019du39faa96ba3590a2d@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
--------------050008030608050201080905
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Artem Belevich wrote:
> Similar issue has been discussed on various zfs-related lists.
>
> Following post contains number of useful links on the issue itself and
> the ways to recover.
> http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg26704.html
>
> --Artem
>   
Thank you for the links, I read through them and read victor's 
post...however, I am not even able to get the command he mentioned to 
start on my particular machine:

schroeder# zdb -bbccsv 5666150306755460943
zdb: can't open 5666150306755460943: Input/output error

I do have some more hope than before after reading those threads, but I 
am unsure of how to proceed past this point.  Not sure if it will help 
but here is the zdb -ll for all the devices in this pool.

schroeder# zdb -ll /dev/ad4s1
--------------------------------------------
LABEL 0
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14725838678957352699
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 1
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14725838678957352699
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 2
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14725838678957352699
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 3
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14725838678957352699
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
schroeder# zdb -ll /dev/ad6s1
--------------------------------------------
LABEL 0
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478204
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14258118189710079071
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 1
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478204
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14258118189710079071
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 2
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478204
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14258118189710079071
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 3
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478204
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=14258118189710079071
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
schroeder# zdb -ll /dev/ad8s1
--------------------------------------------
LABEL 0
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=3285817555281532863
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 1
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=3285817555281532863
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 2
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=3285817555281532863
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 3
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=3285817555281532863
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
schroeder# zdb -ll /dev/ad10s1
--------------------------------------------
LABEL 0
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=4760880554491391388
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 1
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=4760880554491391388
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 2
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=4760880554491391388
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
--------------------------------------------
LABEL 3
--------------------------------------------
    version=13
    name='tank0'
    state=0
    txg=478240
    pool_guid=5666150306755460943
    hostid=2440235366
    hostname='unset'
    top_guid=2037719367177844310
    guid=4760880554491391388
    vdev_tree
        type='raidz'
        id=0
        guid=2037719367177844310
        nparity=1
        metaslab_array=23
        metaslab_shift=32
        ashift=9
        asize=1600332496896
        is_log=0
        children[0]
                type='disk'
                id=0
                guid=14725838678957352699
                path='/dev/ad4s1'
                whole_disk=0
                DTL=323
        children[1]
                type='disk'
                id=1
                guid=14258118189710079071
                path='/dev/ad6s1'
                whole_disk=0
                DTL=327
                removed=1
        children[2]
                type='disk'
                id=2
                guid=3285817555281532863
                path='/dev/ad8s1'
                whole_disk=0
                DTL=326
        children[3]
                type='disk'
                id=3
                guid=4760880554491391388
                path='/dev/ad10s1'
                whole_disk=0
                DTL=325
schroeder#


Thank You,
Aaron

--------------050008030608050201080905--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4AC91216.9070200>