From owner-freebsd-fs@FreeBSD.ORG Mon Jan 6 17:27:06 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F038CD; Mon, 6 Jan 2014 17:27:06 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 346BC179C; Mon, 6 Jan 2014 17:27:05 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa01.fnfis.com (8.14.5/8.14.5) with ESMTP id s06H6hLm019091 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Mon, 6 Jan 2014 11:06:45 -0600 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.118]) by LTCFISWMSGHT03.FNFIS.com ([10.132.206.31]) with mapi id 14.03.0174.001; Mon, 6 Jan 2014 11:06:42 -0600 From: "Teske, Devin" To: Darren Pilgrim Subject: Re: zfs upgrade hang upgrading from v3 to v5 Thread-Topic: zfs upgrade hang upgrading from v3 to v5 Thread-Index: AQHPCwGr3TjDJfr2/keI6KkHRTPCWg== Date: Mon, 6 Jan 2014 17:06:42 +0000 Message-ID: References: <52C89140.5040700@bluerosetech.com> In-Reply-To: <52C89140.5040700@bluerosetech.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.126] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87, 1.0.14, 0.0.0000 definitions=2014-01-06_02:2014-01-06,2014-01-06,1970-01-01 signatures=0 Cc: freebsd-fs , Devin Teske , "Teske, Devin" , freebsd-questions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Devin Teske List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 17:27:06 -0000 On Jan 4, 2014, at 2:54 PM, Darren Pilgrim wrote: > I'm upgrading a system from 8.3 to 9.2 using a fresh install onto a paral= lel set of filesystems on the ZFS pool. The system is a root on ZFS config= uration with GPT-labeled AHCI disks. The zpool upgrade step worked fine. = When I did `zfs upgrade -a` it didn't return right away, but this system is= a little smaller so I left it to work. >=20 > An hour later, it's still not done. Ctrl-T shows zfs upgrade is in tx->t= x_sync_done_cv and using no CPU. Normally I expect to see "runnable" and u= sing some CPU. I can still work in open SSH sessions, but other zfs comman= ds hang. New SSH logins don't work. Console logins hang between me enteri= ng the username and it printing the password prompt. Even though I know th= ere are active processes on the system, there is no disk activity. Network= ing is still fine--the machine acts as a router, and the LAN behind it hasn= 't loss internet access. The unbound instance running on it is also respon= sive, but it never touches the disk when running (it syslogs). >=20 > Figuring it's livelocked on disk I/O, I try to reboot, but neither Contro= l-Alt-Delete nor the power button do anything. I ended up hard resetting t= he system. >=20 > The system rebooted without issue. Zfs upgrade showed a few of the v3 fi= lesystems had been upgraded, but most hadn't. Upgrading filesystems one by= one got me most of the way there. By dumb luck I got all the way to the b= ase filesystem without anything hanging. The base filesystem, however, did= hang. >=20 > I read Devin Teske's messages to freebsd-fs from Sept 20, 2013 about the = same scenario. Interestingly, the base filesystem on this box is the only = one that has mountpoint=3Dnone. Later today I'll try setting a mountpoint = on it see if the upgrade will succeed then. >=20 > In the meantime, is this a known issue by now? The only things I could f= ind were the aforementioned emails from Devin, and no one answered him. I can chime in with the ugly work-around that allowed us to migrate from v3= to v5. Quite unceremoniously, we rsync'd all the data to a new v5 dataset and then destroyed the existing v3 dataset, only to rebuild the pool from scratch. Certainly less than ideal; I'll be very interested in your testing to see i= f you can find a way to around the issue (which we still think is centered around datasets= lacking a mountpoint). --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.