From owner-freebsd-stable@FreeBSD.ORG Mon Jan 23 19:30:01 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A3B4106566C for ; Mon, 23 Jan 2012 19:30:01 +0000 (UTC) (envelope-from prvs=1369c89d38=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 8F31A8FC18 for ; Mon, 23 Jan 2012 19:30:00 +0000 (UTC) X-Spam-Processed: mail1.multiplay.co.uk, Mon, 23 Jan 2012 19:29:00 +0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50017676863.msg for ; Mon, 23 Jan 2012 19:28:59 +0000 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1369c89d38=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-stable@freebsd.org Message-ID: <7F1E17105173440B8C52DB9AAFA7A400@multiplay.co.uk> From: "Steven Hartland" To: "Matthew Seaman" , References: <520B9285BCC0498286196195933D67E9@multiplay.co.uk> <4F1DA991.1060601@infracaninophile.co.uk> Date: Mon, 23 Jan 2012 19:29:12 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Subject: Re: ZFS: i/o error - all block copies unavailable on large disk number machines X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 19:30:01 -0000 ----- Original Message ----- From: "Matthew Seaman" > Even if you do split up your pool into vdevs using 8 drives, you will > still run into the problem with zfs being unable to assemble the pool > unless it sees all of the drives in it. > > Interesting that this only appeared as part of a minor kernel update. > I ran into this myself with 8-STABLE, no indication that there was a fix > possible by juggling kernels. Indeed this was nothing to do with the changes in the kernel its purely down to which disks the physical copies of the data lives on within the boot zpool, at least that's what I believe is the key here. For reference the layout here is the following:- pool: tank2 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 gptid/aad3bd9f-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 gptid/abbe61d0-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 gptid/aca6dba7-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 gptid/ad90c2ba-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 gptid/ae773314-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 gptid/af5dea39-05a2-11e1-8d4a-0025903b854c ONLINE 0 0 0 raidz2 ONLINE 0 0 0 da0p1 ONLINE 0 0 0 da1p1 ONLINE 0 0 0 da2p1 ONLINE 0 0 0 da3p1 ONLINE 0 0 0 gptid/c21396ba-05a5-11e1-bce9-0025903b854c ONLINE 0 0 0 gptid/c21b30b9-05a5-11e1-bce9-0025903b854c ONLINE 0 0 0 cache ada0p3 ONLINE 0 0 0 ada1p3 ONLINE 0 0 0 spares gptid/4eb3ef4c-05a6-11e1-bce9-0025903b854c AVAIL gptid/c2ba092d-05a5-11e1-bce9-0025903b854c AVAIL Initially the zpool was just the first raidz2. Only after install was the second raidz2 added to increase capacity. So what I believe has happened is the new kernel when installed happens to have data be located on the second raidz2 which consists of disks not available to the BIOS and hence results in "all block copies unavailable" from the boot code. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.