From owner-freebsd-fs@FreeBSD.ORG Sun Apr 26 03:26:06 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B2EC868 for ; Sun, 26 Apr 2015 03:26:06 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 253AD1CF0 for ; Sun, 26 Apr 2015 03:26:06 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3Q3Q6rw038591 for ; Sun, 26 Apr 2015 03:26:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198242] [zfs] L2ARC degraded. Checksum errors, I/O errors Date: Sun, 26 Apr 2015 03:26:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lacey.leanne@gmail.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Apr 2015 03:26:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198242 Lacey Powers changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lacey.leanne@gmail.com --- Comment #5 from Lacey Powers --- This bug, and the related bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195746 looks like they were fixed in Illumos as of a few days ago. https://www.illumos.org/issues/5701 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sun Apr 26 10:26:11 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEB55720 for ; Sun, 26 Apr 2015 10:26:11 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A8EC712E9 for ; Sun, 26 Apr 2015 10:26:11 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3QAQBGP004428 for ; Sun, 26 Apr 2015 10:26:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198242] [zfs] L2ARC degraded. Checksum errors, I/O errors Date: Sun, 26 Apr 2015 10:26:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: gkontos@aicom.gr X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Apr 2015 10:26:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198242 gkontos@aicom.gr changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gkontos@aicom.gr --- Comment #6 from gkontos@aicom.gr --- When can we expect this to 10.1-STABLE? Thanks -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sun Apr 26 21:00:16 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79269C3E for ; Sun, 26 Apr 2015 21:00:16 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5243510CA for ; Sun, 26 Apr 2015 21:00:16 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3QL0G3G069348 for ; Sun, 26 Apr 2015 21:00:16 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201504262100.t3QL0G3G069348@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 26 Apr 2015 21:00:16 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Apr 2015 21:00:16 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 3 problems total for which you should take action. From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 08:59:24 2015 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40461FE for ; Mon, 27 Apr 2015 08:59:24 +0000 (UTC) Received: from forward2l.mail.yandex.net (forward2l.mail.yandex.net [IPv6:2a02:6b8:0:1819::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF5E6157A for ; Mon, 27 Apr 2015 08:59:23 +0000 (UTC) Received: from smtp17.mail.yandex.net (smtp17.mail.yandex.net [95.108.252.17]) by forward2l.mail.yandex.net (Yandex) with ESMTP id 3A5601AC10C0; Mon, 27 Apr 2015 11:59:12 +0300 (MSK) Received: from smtp17.mail.yandex.net (localhost [127.0.0.1]) by smtp17.mail.yandex.net (Yandex) with ESMTP id B2292190018E; Mon, 27 Apr 2015 11:59:11 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:5::3a]) by smtp17.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id Iz8BsBKnzB-xBZWpLFF; Mon, 27 Apr 2015 11:59:11 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1430125151; bh=8fiDyZcxhtloVe7AIvDzNagqNaFpa8wYRcHrttAD+fM=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: References:In-Reply-To:Content-Type; b=Oh5L3pJ8d+1UEcMjvmLFCuAJd1AzgzwjlYRZM7TWteFSXKBBp/5xniL1uCp4JoP0B mFuWUzl4b+ziiHvOrJBIf0uG6JO1ocD9nbijHHXWu7ePFMvpcnLn8hB4QjVEz9Zvte dPPplM74hyKPfz59JzyCuMNsdeiMRi0PhQONJ47A= Authentication-Results: smtp17.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <553DF9E3.1090609@yandex.ru> Date: Mon, 27 Apr 2015 11:57:07 +0300 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Willem Jan Withagen , fs@freebsd.org Subject: Re: resampeling of a ZVOL that has been resized References: <55381127.4090603@digiware.nl> <5539B0C4.6070000@yandex.ru> <553B7200.7090002@digiware.nl> In-Reply-To: <553B7200.7090002@digiware.nl> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="aOfp1KQW1LkD2LC3WVA9wDLHbcxd9oaFr" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 08:59:24 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --aOfp1KQW1LkD2LC3WVA9wDLHbcxd9oaFr Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 25.04.2015 13:52, Willem Jan Withagen wrote: > So it seems that although gpart understands that the ZVOL volume has > grown, it does not take it far enough and set it to CORRUPTED and then > let the user correct/grow it. Can you try this patch? Index: head/sys/geom/part/g_part_gpt.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- head/sys/geom/part/g_part_gpt.c (revision 282044) +++ head/sys/geom/part/g_part_gpt.c (working copy) @@ -760,7 +760,7 @@ g_part_gpt_resize(struct g_part_table *basetable, struct g_part_gpt_entry *entry; if (baseentry =3D=3D NULL) - return (EOPNOTSUPP); + return (g_part_gpt_recover(basetable)); entry =3D (struct g_part_gpt_entry *)baseentry; baseentry->gpe_end =3D baseentry->gpe_start + gpp->gpp_size - 1; --=20 WBR, Andrey V. Elsukov --aOfp1KQW1LkD2LC3WVA9wDLHbcxd9oaFr Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJVPfnjAAoJEAHF6gQQyKF6q1AIAKPD/O4Yf2lhSFY8K3uYVVJd JJB+yXpUzuRCFWo8xpir31XnOztwBPK3qD5gIibHWlpExqvTWz9G51OQSNz4W9BE hwNFNe5r/tW0m9KvY3jG6GaOOJHBPHNOLskstdeHlzCPoJrlsRwxtk4hWDWgp3+l FxvptQnxvetZ2WE/3EbEkLinH40apyAivNye7IA6DnCUYAh5E24LiQjRFwZ+dJ6u GTBaKoRYNwDYRP7DiIUN24mwqlsbYo8gvK9eiWNRCRVU50HH7HH0KPhIO7eW7iS8 JpcPeX9Ca/qyYhRW5TxEYRewVF4QV4rFwjO8mYva/YBpF0C12RSR+FVYUIJlqJA= =h1L9 -----END PGP SIGNATURE----- --aOfp1KQW1LkD2LC3WVA9wDLHbcxd9oaFr-- From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 10:23:48 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB382AA6 for ; Mon, 27 Apr 2015 10:23:48 +0000 (UTC) Received: from platinum.linux.pl (platinum.edu.pl [81.161.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id 7C8F51F04 for ; Mon, 27 Apr 2015 10:23:47 +0000 (UTC) Received: by platinum.linux.pl (Postfix, from userid 87) id B85E545219A; Mon, 27 Apr 2015 12:18:15 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on platinum.linux.pl X-Spam-Level: X-Spam-Status: No, score=-1.3 required=3.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.4.0 Received: from [10.255.1.10] (c38-073.client.duna.pl [83.151.38.73]) by platinum.linux.pl (Postfix) with ESMTPA id 43A5E452198 for ; Mon, 27 Apr 2015 12:18:15 +0200 (CEST) Message-ID: <553E0CE0.8070803@platinum.linux.pl> Date: Mon, 27 Apr 2015 12:18:08 +0200 From: Adam Nowacki User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: resampeling of a ZVOL that has been resized References: <55381127.4090603@digiware.nl> <5539B0C4.6070000@yandex.ru> <553B7200.7090002@digiware.nl> In-Reply-To: <553B7200.7090002@digiware.nl> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 10:23:48 -0000 On 2015-04-25 12:52, Willem Jan Withagen wrote: > On 24/04/2015 04:56, Andrey V. Elsukov wrote: >> On 23.04.2015 00:22, Willem Jan Withagen wrote: >>> Now the question: >>> How can I get GEOM to resample the zvol, and have it really detect that >>> the disk has changed.... It sort of does, but not enough to actually >>> allow it to grow to the new size. >> >> You need to read dmesg, where you will find the message: >> GEOM_PART: zvol/zfsdata/vol was automatically resized. >> Use `gpart commit zvol/zfsdata/vol` to save changes or >> `gpart undo zvol/zfsdata/vol` to revert them. >> > > This does not really resolve the issue. > after growing the volume in ZFS, the new size is reported in ' > gpart show': > > freetest# gpart show zvol/zfsdata/vol > => 40 209715120 zvol/zfsdata/vol GPT (200G) > 40 8 - free - (4.0K) > 48 209715104 1 freebsd-ufs (100G) > 209715152 8 - free - (4.0K) > > However the free space at the end stays at a mere 4.0K, not allowing > gpart resize to take any value other than less than 100G, effectively > shrinking the partition. > > And there is no incantation of any of commit, recover, etc... to get > gpart to actually do "the right thing" > > When I did this on a VMware VM, gpart show reported the volume as > [CORRUPTED] and gpart recover fixed that. > Supposedly the backup blocks were at the wrong place in the grown disk > and recover again placed them at the end. > > But the ZFS case does not go into the [CORRUPTED] state. > Perhaps that is due to also missing the message that you suggest that > can be found in dmesg after resizing the ZVOL. > And thus a recover is not needed, nor dies issueing it fix anything. > > Now after a reboot the bootup log tells: > GEOM: zvol/zfsdata/vol: the secondary GPT header is not in the last LBA. > > And now gpart report as expected: > => 40 209715120 zvol/zfsdata/vol GPT (200G) [CORRUPT] > 40 8 - free - (4.0K) > 48 209715104 1 freebsd-ufs (100G) > 209715152 8 - free - (4.0K) > > gpart recover sets the free space to 100G: > => 40 419430320 zvol/zfsdata/vol GPT (200G) > 40 8 - free - (4.0K) > 48 209715104 1 freebsd-ufs (100G) > 209715152 209715208 - free - (100G) > > And from now on I can resize the partition 1 to 200G... > > So it seems that although gpart understands that the ZVOL volume has > grown, it does not take it far enough and set it to CORRUPTED and then > let the user correct/grow it. You can tell kernel to retaste the device without rebooting by running: true > /dev/zvol/zfsdata/vol From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 15:21:18 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAE097FE for ; Mon, 27 Apr 2015 15:21:18 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B580D12D9 for ; Mon, 27 Apr 2015 15:21:18 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3RFLIc5002099 for ; Mon, 27 Apr 2015 15:21:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 121898] [nullfs] pwd(1)/getcwd(2) fails with Permission denied Date: Mon, 27 Apr 2015 15:21:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jilles@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 15:21:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=121898 Jilles Tjoelker changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Progress |Closed Resolution|--- |DUPLICATE CC| |jilles@FreeBSD.org --- Comment #3 from Jilles Tjoelker --- *** This bug has been marked as a duplicate of bug 161424 *** -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 15:21:21 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 316FC802 for ; Mon, 27 Apr 2015 15:21:21 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C7C512DD for ; Mon, 27 Apr 2015 15:21:21 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3RFLKjc002122 for ; Mon, 27 Apr 2015 15:21:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 161424] [nullfs] __getcwd() calls fail when used on nullfs mount Date: Mon, 27 Apr 2015 15:21:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jilles@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 15:21:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161424 Jilles Tjoelker changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ota@j.email.ne.jp --- Comment #7 from Jilles Tjoelker --- *** Bug 121898 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 16:15:16 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7647BF39 for ; Mon, 27 Apr 2015 16:15:16 +0000 (UTC) Received: from smtp101-5.vfemail.net (eightfive.vfemail.net [96.30.253.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4689C1991 for ; Mon, 27 Apr 2015 16:15:15 +0000 (UTC) Received: (qmail 28676 invoked by uid 89); 27 Apr 2015 16:08:32 -0000 Received: by simscan 1.4.0 ppid: 28669, pid: 28673, t: 0.0715s scanners:none Received: from unknown (HELO d3d3MTEwQDE0MzAxNTA5MTI=) (cmlja0BoYXZva21vbi5jb21AMTQzMDE1MDkxMg==@MTcyLjE2LjEwMC45MkAxNDMwMTUwOTEy) by 172.16.100.61 with ESMTPA; 27 Apr 2015 16:08:32 -0000 Date: Mon, 27 Apr 2015 11:08:32 -0500 Message-ID: <20150427110832.Horde.MAoPtcoic1-3sfV0OhkyxQ1@www.vfemail.net> From: Rick Romero To: freebsd-fs Subject: 10.1 + ZFS snapshot eating diskspace User-Agent: Internet Messaging Program (IMP) H5 (6.2.2) X-VFEmail-Originating-IP: MTIuMzEuMTAwLjE0Ng== X-VFEmail-AntiSpam: Notify admin@vfemail.net of any spam, and include VFEmail headers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes Content-Transfer-Encoding: 8bit Content-Disposition: inline Content-Description: Plaintext Message X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 16:15:16 -0000 Try number two.   I built another new system, no encryption this time. I replacated ONE snapshot that is about 562GB of data. (I just found Ronalds reply in my Spam folder, sorry!) This new 10.1 system has the exact same 3 drives in RAIDZ1 as the original source (9.2).  What's confusing is the original RAIDZ1 is replicated correctly to a 10 drive RAIDZ2 (10.1), but the RAIDZ2 source cannot replicate data correctly to a new 3 drive RAIDZ1. So not only is this a problem with the new system, but it concerns me that if there were a problem with the old system that a full restore from backup would eat all the disk space. Source: # zfs get all sysvol/primessd_home |grep -i used sysvol/primessd_home  used                  822G                   - sysvol/primessd_home  usedbysnapshots       260G                   - sysvol/primessd_home  usedbydataset         562G                   - sysvol/primessd_home  usedbychildren        0                      - sysvol/primessd_home  usedbyrefreservation  0                      - sysvol/primessd_home  logicalused           811G                   - Right? 562 is the 'current' amount of space used?  So I send it to a new box, and this is the result # zfs list -t all NAME                        USED  AVAIL  REFER  MOUNTPOINT sysvol                      919G      0  12.5G  /sysvol sysvol/home                 906G      0   898G  /sysvol/home sysvol/home@remrep-Week16  8.53G      -   898G  - I can see a possible sector size diff or recordsize affecting a few bytes, but 400G is a bit excessive. The fact that it more closely matches the full dataset+snapshots, IMHO, is much more telling. # zfs get all sysvol/home | grep used sysvol/home  used                  906G                   - sysvol/home  usedbysnapshots       8.53G                  - sysvol/home  usedbydataset         898G                   - sysvol/home  usedbychildren        0                      - sysvol/home  usedbyrefreservation  0                      - sysvol/home  logicalused           574G                   - logical used is actual used, correct?   Why is it the 'full' amount, when only one snapshot was replicated? So I thought maybe it's not reporting correctly # zfs list NAME          USED  AVAIL  REFER  MOUNTPOINT sysvol        907G  12.3G   256M  /sysvol sysvol/home   906G  12.3G   898G  /sysvol/home # dd bs=1M count=12560 if=/dev/zero of=test2 dd: test2: No space left on device 12558+0 records in 12557+1 records out 13167886336 bytes transferred in 33.499157 secs (393081126 bytes/sec) # zfs list NAME          USED  AVAIL  REFER  MOUNTPOINT sysvol        919G      0  12.5G  /sysvol sysvol/home   906G      0   898G  /sysvol/home # dd bs=1M count=12560 if=/dev/zero of=test3 dd: test3: No space left on device So what's going on?  Is this a known issue? I suppose I can take the new server down to the colo and replicate from the original, but that doesn't resolve the 'restore from backup' issue that I see happening... From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 17:26:38 2015 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 275263BA for ; Mon, 27 Apr 2015 17:26:38 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE6AA11FD for ; Mon, 27 Apr 2015 17:26:37 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 685D016A51F; Mon, 27 Apr 2015 19:26:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9YBdJQCxn11a; Mon, 27 Apr 2015 19:26:22 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:7460:5f9a:4d6f:40c1] (unknown [IPv6:2001:4cb8:3:1:7460:5f9a:4d6f:40c1]) by smtp.digiware.nl (Postfix) with ESMTPA id C859216A502; Mon, 27 Apr 2015 19:26:22 +0200 (CEST) Message-ID: <553E713C.2000602@digiware.nl> Date: Mon, 27 Apr 2015 19:26:20 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: "Andrey V. Elsukov" , fs@freebsd.org Subject: Re: resampeling of a ZVOL that has been resized References: <55381127.4090603@digiware.nl> <5539B0C4.6070000@yandex.ru> <553B7200.7090002@digiware.nl> <553DF9E3.1090609@yandex.ru> In-Reply-To: <553DF9E3.1090609@yandex.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 17:26:38 -0000 On 27/04/2015 10:57, Andrey V. Elsukov wrote: > On 25.04.2015 13:52, Willem Jan Withagen wrote: >> So it seems that although gpart understands that the ZVOL volume has >> grown, it does not take it far enough and set it to CORRUPTED and then >> let the user correct/grow it. > > Can you try this patch? > > Index: head/sys/geom/part/g_part_gpt.c > =================================================================== > --- head/sys/geom/part/g_part_gpt.c (revision 282044) > +++ head/sys/geom/part/g_part_gpt.c (working copy) > @@ -760,7 +760,7 @@ g_part_gpt_resize(struct g_part_table *basetable, > struct g_part_gpt_entry *entry; > > if (baseentry == NULL) > - return (EOPNOTSUPP); > + return (g_part_gpt_recover(basetable)); > > entry = (struct g_part_gpt_entry *)baseentry; > baseentry->gpe_end = baseentry->gpe_start + gpp->gpp_size - 1; > That actually generates on the console: (And probably in dmesg) Apr 27 15:35:37 freetest kernel: GEOM_PART: zvol/zfsdata/vol was automatically resized. Apr 27 15:35:37 freetest kernel: Use `gpart commit zvol/zfsdata/vol` to save changes or `gpart undo zvol/zfsdata/vol` to revert them. And after gpart commit, it allows to gpart resize, growfs.. Exactly like I wanted. But I need to set freetest# sysctl kern.geom.debugflags=16 before resize actually works. And I did get a panic in one of the attempts, but with no keyboard to the console. So I had very little to act on, and needed to reset. I'll finish the test script and run it in a loop to see if the panic reoccurs. But so far so good. --WjW From owner-freebsd-fs@FreeBSD.ORG Mon Apr 27 19:38:26 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C87432F3 for ; Mon, 27 Apr 2015 19:38:26 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B379411F9 for ; Mon, 27 Apr 2015 19:38:26 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t3RJcQUA018347 for ; Mon, 27 Apr 2015 19:38:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199189] SWAP on ZFS can crash server Date: Mon, 27 Apr 2015 19:38:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: marcus@blazingdot.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2015 19:38:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199189 --- Comment #13 from marcus@blazingdot.com --- > Currently only 'compression=lz4' comes to my mind. It would surprise nobody if turning on compression caused problems. If it's any consolation, here are the default locally set flags on the swap zvol on a Solaris 10 machine: rpool/swap volsize 4G local rpool/swap checksum off local rpool/swap compression off local rpool/swap refreservation 4G local rpool/swap primarycache metadata local -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Apr 28 14:22:03 2015 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BAE5A67 for ; Tue, 28 Apr 2015 14:22:03 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E09C146E for ; Tue, 28 Apr 2015 14:22:03 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 1FF8516A4F4; Tue, 28 Apr 2015 16:22:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ERJNdhkvgcf2; Tue, 28 Apr 2015 16:21:31 +0200 (CEST) Received: from [192.168.101.176] (vpn.ecoracks.nl [31.223.170.173]) by smtp.digiware.nl (Postfix) with ESMTPA id EE6B716A4DC; Tue, 28 Apr 2015 16:21:31 +0200 (CEST) Message-ID: <553F976B.5070608@digiware.nl> Date: Tue, 28 Apr 2015 16:21:31 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: "Andrey V. Elsukov" , fs@freebsd.org Subject: Re: resampeling of a ZVOL that has been resized References: <55381127.4090603@digiware.nl> <5539B0C4.6070000@yandex.ru> <553B7200.7090002@digiware.nl> <553DF9E3.1090609@yandex.ru> <553E713C.2000602@digiware.nl> In-Reply-To: <553E713C.2000602@digiware.nl> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 14:22:03 -0000 On 27/04/2015 19:26, Willem Jan Withagen wrote: > On 27/04/2015 10:57, Andrey V. Elsukov wrote: >> On 25.04.2015 13:52, Willem Jan Withagen wrote: >>> So it seems that although gpart understands that the ZVOL volume has >>> grown, it does not take it far enough and set it to CORRUPTED and then >>> let the user correct/grow it. >> >> Can you try this patch? >> >> Index: head/sys/geom/part/g_part_gpt.c >> =================================================================== >> --- head/sys/geom/part/g_part_gpt.c (revision 282044) >> +++ head/sys/geom/part/g_part_gpt.c (working copy) >> @@ -760,7 +760,7 @@ g_part_gpt_resize(struct g_part_table *basetable, >> struct g_part_gpt_entry *entry; >> >> if (baseentry == NULL) >> - return (EOPNOTSUPP); >> + return (g_part_gpt_recover(basetable)); >> >> entry = (struct g_part_gpt_entry *)baseentry; >> baseentry->gpe_end = baseentry->gpe_start + gpp->gpp_size - 1; >> > > That actually generates on the console: (And probably in dmesg) > Apr 27 15:35:37 freetest kernel: GEOM_PART: zvol/zfsdata/vol was > automatically resized. > Apr 27 15:35:37 freetest kernel: Use `gpart commit zvol/zfsdata/vol` to > save changes or `gpart undo zvol/zfsdata/vol` to revert them. > > And after gpart commit, it allows to gpart resize, growfs.. > Exactly like I wanted. But I need to set > freetest# sysctl kern.geom.debugflags=16 > before resize actually works. > > And I did get a panic in one of the attempts, but with no keyboard to > the console. So I had very little to act on, and needed to reset. > > I'll finish the test script and run it in a loop to see if the panic > reoccurs. > > But so far so good. It has been running for most the day and night, in all kinds of incarnations.... And it has not failed yet and without crashes. So I'd say: that is a fix. --WjW From owner-freebsd-fs@FreeBSD.ORG Tue Apr 28 15:53:49 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF470CDC for ; Tue, 28 Apr 2015 15:53:49 +0000 (UTC) Received: from mail-ig0-f179.google.com (mail-ig0-f179.google.com [209.85.213.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FE451026 for ; Tue, 28 Apr 2015 15:53:49 +0000 (UTC) Received: by igblo3 with SMTP id lo3so93272414igb.1 for ; Tue, 28 Apr 2015 08:53:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:reply-to:organization :user-agent:mime-version:to:subject:content-type :content-transfer-encoding; bh=O5Llz+XOIOhjgD1bx/hNBReaNoHs1aJCPU/ddaH7yR4=; b=HNbIeUAcS6XGC8RMF4GM7iex2OPJHcdCRcRwJ9BY2wCEe25hyWoahI4NB00dF8vJK8 pf4ZSFmEN22NK2EH+W1uDPEw1Qk3bzq+ptgWSgL1hMDEUKKjwKXs8vzztWeTimkUk60W SKFkeFioFxCCGWvjIUJpDqRMuru9yYJhgk8dg7Xm6iL2G28I7shpAw50jX5QDt70XHf0 Ww6c3B6Ij1Cxp3GePirub6z9cvIABxY2xDtBL86KuqYFoMP56ovrMMsbCv9qv4xWfdGu cuU7Z2x780oLI6HLjzOk2XzH/zbiFQr3Ni7rOhM7bWL9bzfJCTOpSfJcvc+1IOsmGsY9 BUsQ== X-Gm-Message-State: ALoCoQn584y6iU9uO2t3X5E/tAzjh+gQ57Q/plHPGQaaPyHVB2VbX3ShadfAtwIy2aki+mzoE32A X-Received: by 10.107.149.196 with SMTP id x187mr924381iod.4.1430236428008; Tue, 28 Apr 2015 08:53:48 -0700 (PDT) Received: from [192.168.0.18] ([63.231.252.189]) by mx.google.com with ESMTPSA id j39sm14967195iod.41.2015.04.28.08.53.46 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Apr 2015 08:53:47 -0700 (PDT) Message-ID: <553FAD0A.3070408@kateley.com> Date: Tue, 28 Apr 2015 10:53:46 -0500 From: Linda Kateley Reply-To: linda@kateley.com Organization: Kateley Company User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: zfs-discuss@list.zfsonlinux.org, freebsd-fs@freebsd.org, omnios-discuss@lists.omniti.com Subject: Open-ZFS Europe May 26 Paris Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 15:53:49 -0000 All, I wanted to make sure everyone was aware of the Open-ZFS Europe conference in Paris on May 26th. Speakers will also be webcast. Festivities start at 8 am, Central European Summer time UTC +02 http://www.meetup.com/OpenZFS-Europe/events/218873174/ There is also an openzfs hackathon the next day May 27th. Details can also be seen from the open-zfs.org page. linda From owner-freebsd-fs@FreeBSD.ORG Wed Apr 29 21:41:00 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D526B93 for ; Wed, 29 Apr 2015 21:41:00 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BFEF31ADB for ; Wed, 29 Apr 2015 21:40:59 +0000 (UTC) Received: by qcbii10 with SMTP id ii10so20207849qcb.2 for ; Wed, 29 Apr 2015 14:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=GRB8FDZRQ6MbiROlYt12Hc21F/NonxRbT6+fGazfzFw=; b=sPPox3DjUHvBdN7Byw5gAwBSjDFU81meJzWNEwNG5ZXYY0XZiCnopxRc9VuhV/Lca6 4G9TESst8B/NbHv4oTI9ypuu4XCUInTNyl+J0H1DXzt8142fQjycUXyPwzW5zAOSQNW9 DpUb3F40m8jmhhkwiXoJ9l630QozncmIOcU0qk627RjEVKeQi0AMPuIWehzT9TErILPH Cgr2KsnKPG+DJI2i34Dct4h/vrr3npT5W/suMkTl7WfohXNs4t5Iv+oBBsHYtx24MhKS 7qPETKqseiqLJCXwT09/itzYYIPzh3tOQ41f/TPot00Zuaft/f83w6MjnaRPMIkSCH15 z50Q== X-Received: by 10.140.236.147 with SMTP id h141mr1346726qhc.77.1430343658436; Wed, 29 Apr 2015 14:40:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.96.118 with HTTP; Wed, 29 Apr 2015 14:40:38 -0700 (PDT) From: Gabor Radnai Date: Wed, 29 Apr 2015 23:40:38 +0200 Message-ID: Subject: vfs.zfs.write_to_degraded missing in 10.1? To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Apr 2015 21:41:00 -0000 Hi, I run 10.1-stable (r281854), is it possible I missed the announcement of removing vfs.zfs.write_to_degraded sysctl variable? The Handbook's Advanced Topics 20.6 is mentioning this variable as "to allow writing to degraded vdevs to prevent the amount of free space across the vdevs from becoming unbalanced, which will reduce read and write performance." How can this be achieved currently? Thanks. From owner-freebsd-fs@FreeBSD.ORG Thu Apr 30 13:56:09 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7B9BFE6 for ; Thu, 30 Apr 2015 13:56:09 +0000 (UTC) Received: from smtp.hostage.nl (smtp.hostage.nl [109.72.93.221]) by mx1.freebsd.org (Postfix) with ESMTP id 758F81CDE for ; Thu, 30 Apr 2015 13:56:09 +0000 (UTC) Date: Thu, 30 Apr 2015 15:47:00 +0200 From: Martijn To: freebsd-fs@freebsd.org Subject: ZFS stuck on write Message-ID: <20150430134659.GA4950@kobol.office.hostage.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2015 13:56:09 -0000 Hi, I've been trying to get an important production machine stable again since yesterday afternoon, but to no avail (so far). It seems ZFS is the problem on this box. Situation is as follows: - it used to be fbsd 8.3. after reading about deadlocks which have been fixed in the meantime, i upgrade to 10.1, no change, did zpool upgrade, no change and did zfs upgrade -a also no change... - Its a machine on which each user has a separate zfs filesystem with refquota set. It also did periodic zfs snapshots, every hour (48), day (14) week (8) month (24), which is way too much but at the time of setting it up i thought it couldn't hurt. - After some usage the machine gets stuck when trying to write a file. The process just stops and can't be killed. After some time the whole machine used to become unresponsive in 8.3, but nice 10.1 i can still reach it, although processes attempting to write get stuck for ever. - nothing scary shows in dmesg What can i do? The machine has 24GB of registered ECC ram (17GB free), its a RAID-Z pool with 4 sata hdd's on a LSI SAS3442E-R (1068 chip) in IT mode. loader.conf: vfs.zfs.arc_max=8G vfs.zfs.txg.timeout="5" vfs.zfs.prefetch_disable="1" vfs.zfs.vdev.min_pending="3" vfs.zfs.vdev.max_pending="6" vfs.zfs.txg.write_limit_override=1073741824 I've tried to copy the most important users to another machine but thats gonna take a lot of time. There's 160 users (websites + mailboxes) on it. Any help would be much appreciated! Thanks in advance, Martijn -- Hostage Keizersgracht 316 1016 EZ Amsterdam tel: +31 (0)20 4632 303 http://www.hostage.nl From owner-freebsd-fs@FreeBSD.ORG Thu Apr 30 14:19:10 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DC30695 for ; Thu, 30 Apr 2015 14:19:10 +0000 (UTC) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3641F1F46 for ; Thu, 30 Apr 2015 14:19:10 +0000 (UTC) Received: by wgen6 with SMTP id n6so64054671wge.3 for ; Thu, 30 Apr 2015 07:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=JAGmK8KPPYfCnQzUmEYPmNwlM0gCoRlqUMVnpfz5uWQ=; b=uudVFtuzNcJZOMT2SU8mFHRunlRIHVvP14tVzLoOGm/WmUS2nGMm6txckk7VLeXDHT odAIIoO8NQDDypwL9PrF/u2VOkPHpUHxSbTY13FXoErcMjlYeQFdWWKc/92cek6nOiqK ZeKVd6zaBTddCUjOq42+PjaSN9dOJiFqZK4Em9TdooB0By/bpAXQMGxlCPnaGw0rNmwc lcq22G42ByU2B2W1teVORJCwt4jC+C3ztMDCXnxPKtWjRHKqyonOmWfokAwEDjUT8IYP 4HlCOBHuKxNDbDHw10lpzYv2zdAuCO/T0HF0nFrJe8hcwW7WiUz9wgE+p8xO33VHbF2f 9kyA== X-Received: by 10.194.60.4 with SMTP id d4mr9253576wjr.72.1430403548662; Thu, 30 Apr 2015 07:19:08 -0700 (PDT) Received: from robert-notebook (ukc1-fw-1-v133-dip1.oracle.co.uk. [144.24.19.5]) by mx.google.com with ESMTPSA id di9sm2670737wib.16.2015.04.30.07.19.07 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Apr 2015 07:19:08 -0700 (PDT) Date: Thu, 30 Apr 2015 16:19:02 +0200 From: Robert David To: Martijn Cc: freebsd-fs@freebsd.org Subject: Re: ZFS stuck on write Message-ID: <20150430161902.4868094c@robert-notebook> In-Reply-To: <20150430134659.GA4950@kobol.office.hostage.nl> References: <20150430134659.GA4950@kobol.office.hostage.nl> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.27; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2015 14:19:10 -0000 Hi Martin, so few information provided to suggest anything. What about the pool size, free space? Some exceeding quotas? Features enabled (compress,dedup)? Zil, l2arc? Regards, Robert. On Thu, 30 Apr 2015 15:47:00 +0200 Martijn wrote: > Hi, > > I've been trying to get an important production machine stable again since > yesterday afternoon, but to no avail (so far). > > It seems ZFS is the problem on this box. Situation is as follows: > > - it used to be fbsd 8.3. after reading about deadlocks which have been fixed > in the meantime, i upgrade to 10.1, no change, did zpool upgrade, no change > and did zfs upgrade -a also no change... > > - Its a machine on which each user has a separate zfs filesystem with refquota > set. It also did periodic zfs snapshots, every hour (48), day (14) week (8) > month (24), which is way too much but at the time of setting it up i thought > it couldn't hurt. > > - After some usage the machine gets stuck when trying to write a file. The > process just stops and can't be killed. After some time the whole machine > used to become unresponsive in 8.3, but nice 10.1 i can still reach it, > although processes attempting to write get stuck for ever. > > - nothing scary shows in dmesg > > What can i do? The machine has 24GB of registered ECC ram (17GB free), its a > RAID-Z pool with 4 sata hdd's on a LSI SAS3442E-R (1068 chip) in IT mode. > > loader.conf: > > vfs.zfs.arc_max=8G > vfs.zfs.txg.timeout="5" > vfs.zfs.prefetch_disable="1" > vfs.zfs.vdev.min_pending="3" > vfs.zfs.vdev.max_pending="6" > vfs.zfs.txg.write_limit_override=1073741824 > > I've tried to copy the most important users to another machine but thats gonna > take a lot of time. There's 160 users (websites + mailboxes) on it. > > Any help would be much appreciated! > > Thanks in advance, > > Martijn From owner-freebsd-fs@FreeBSD.ORG Thu Apr 30 14:30:56 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0106F9AB for ; Thu, 30 Apr 2015 14:30:55 +0000 (UTC) Received: from smtp.hostage.nl (smtp.hostage.nl [109.72.93.221]) by mx1.freebsd.org (Postfix) with ESMTP id 8B6B6112F for ; Thu, 30 Apr 2015 14:30:55 +0000 (UTC) Date: Thu, 30 Apr 2015 16:30:18 +0200 From: Martijn To: Robert David Cc: freebsd-fs@freebsd.org Subject: Re: ZFS stuck on write Message-ID: <20150430143017.GA5573@kobol.office.hostage.nl> References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150430161902.4868094c@robert-notebook> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2015 14:30:56 -0000 Sorry.... No features like compress or dedup, The only devs are the storage hdd's NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gpt/diskil ONLINE 0 0 0 gpt/diskir ONLINE 0 0 0 gpt/diskol ONLINE 0 0 0 gpt/diskor ONLINE 0 0 0 # sysctl -a | grep l2arc vfs.zfs.l2arc_write_max: 8388608 vfs.zfs.l2arc_write_boost: 8388608 vfs.zfs.l2arc_headroom: 2 vfs.zfs.l2arc_feed_secs: 1 vfs.zfs.l2arc_feed_min_ms: 200 vfs.zfs.l2arc_noprefetch: 1 vfs.zfs.l2arc_feed_again: 1 vfs.zfs.l2arc_norw: 1 an example stuck proc: PID TID COMM TDNAME KSTACK 4232 100606 bsdtar - mi_switch+0xe1 sleepq_wait+0x3a sleeplk+0x15d __lockmgr_args+0xc9e vop_stdlock+0x3c VOP_LOCK1_APV+0xab _vn_lock+0x43 extattr_list_vp+0x3c sys_extattr_list_fd+0xa4 amd64_syscall+0x351 Xfast_syscall+0xfb # procstat -kk 4 PID TID COMM TDNAME KSTACK 4 100092 zfskern arc_reclaim_thre mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b arc_reclaim_thread+0x301 fork_exit+0x9a fork_trampoline+0xe 4 100093 zfskern l2arc_feed_threa mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b l2arc_feed_thread+0x16f fork_exit+0x9a fork_trampoline+0xe 4 100396 zfskern trim zroot mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b trim_thread+0x9e fork_exit+0x9a fork_trampoline+0xe 4 100406 zfskern txg_thread_enter mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x2bb fork_exit+0x9a fork_trampoline+0xe 4 100407 zfskern txg_thread_enter mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d txg_sync_thread+0x2eb fork_exit+0x9a fork_trampoline+0xe If you need more i'm happy to provide. Could it be hardware related? If the reg ecc memory is broken, wouldn't it show on the console? Thanks, Martijn. Once upon a 30 Apr 2015, Robert David hit keys in the following order: > Hi Martin, > > so few information provided to suggest anything. > > What about the pool size, free space? Some exceeding quotas? Features enabled > (compress,dedup)? Zil, l2arc? > > Regards, > Robert. > > On Thu, 30 Apr 2015 15:47:00 +0200 > Martijn wrote: > > > Hi, > > > > I've been trying to get an important production machine stable again since > > yesterday afternoon, but to no avail (so far). > > > > It seems ZFS is the problem on this box. Situation is as follows: > > > > - it used to be fbsd 8.3. after reading about deadlocks which have been fixed > > in the meantime, i upgrade to 10.1, no change, did zpool upgrade, no change > > and did zfs upgrade -a also no change... > > > > - Its a machine on which each user has a separate zfs filesystem with refquota > > set. It also did periodic zfs snapshots, every hour (48), day (14) week (8) > > month (24), which is way too much but at the time of setting it up i thought > > it couldn't hurt. > > > > - After some usage the machine gets stuck when trying to write a file. The > > process just stops and can't be killed. After some time the whole machine > > used to become unresponsive in 8.3, but nice 10.1 i can still reach it, > > although processes attempting to write get stuck for ever. > > > > - nothing scary shows in dmesg > > > > What can i do? The machine has 24GB of registered ECC ram (17GB free), its a > > RAID-Z pool with 4 sata hdd's on a LSI SAS3442E-R (1068 chip) in IT mode. > > > > loader.conf: > > > > vfs.zfs.arc_max=8G > > vfs.zfs.txg.timeout="5" > > vfs.zfs.prefetch_disable="1" > > vfs.zfs.vdev.min_pending="3" > > vfs.zfs.vdev.max_pending="6" > > vfs.zfs.txg.write_limit_override=1073741824 > > > > I've tried to copy the most important users to another machine but thats gonna > > take a lot of time. There's 160 users (websites + mailboxes) on it. > > > > Any help would be much appreciated! > > > > Thanks in advance, > > > > Martijn > -- Hostage Keizersgracht 316 1016 EZ Amsterdam tel: +31 (0)20 4632 303 http://www.hostage.nl From owner-freebsd-fs@FreeBSD.ORG Fri May 1 03:56:38 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D3E63FD for ; Fri, 1 May 2015 03:56:38 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 093E117A0 for ; Fri, 1 May 2015 03:56:38 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t413ubRY006202 for ; Fri, 1 May 2015 03:56:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199775] ZFS hangs while removing large file Date: Fri, 01 May 2015 03:56:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 03:56:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199775 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri May 1 03:57:31 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAF6B54E for ; Fri, 1 May 2015 03:57:31 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A679A17C0 for ; Fri, 1 May 2015 03:57:31 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t413vVhN006888 for ; Fri, 1 May 2015 03:57:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199804] ZFS: i/o error - all block copies unavailable Date: Fri, 01 May 2015 03:57:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 03:57:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199804 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri May 1 09:32:20 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DB1BFFD for ; Fri, 1 May 2015 09:32:20 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37D051050 for ; Fri, 1 May 2015 09:32:20 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t419WKKU020829 for ; Fri, 1 May 2015 09:32:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199775] ZFS hangs while removing large file Date: Fri, 01 May 2015 09:32:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: smh@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 09:32:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199775 Steven Hartland changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |smh@FreeBSD.org --- Comment #3 from Steven Hartland --- What does gstat -d -p look like at this time? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri May 1 13:10:06 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E35E2A3 for ; Fri, 1 May 2015 13:10:06 +0000 (UTC) Received: from smtp.hostage.nl (smtp.hostage.nl [109.72.93.221]) by mx1.freebsd.org (Postfix) with ESMTP id AF10A155F for ; Fri, 1 May 2015 13:10:06 +0000 (UTC) Date: Fri, 1 May 2015 15:09:24 +0200 From: Martijn To: Robert David Cc: freebsd-fs@freebsd.org Subject: Re: ZFS stuck on write Message-ID: <20150501130924.GA12031@kobol.datajust.com> References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> <20150430143017.GA5573@kobol.office.hostage.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150430143017.GA5573@kobol.office.hostage.nl> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 13:10:07 -0000 Hi, I've been able to borrow a brand new server from my supplier, put the HDD's in and all my problems are gone. I'm gonna look into this with more detail, because i still want to pin-point the exact problem and if its really due to broken hardware. Is my assumption true that the boost in memory (the new machine has 32GB instead of 24GB) also could be the cause that everything works again? If i remember correctly a lot sysctl settings have their defaults based on the total amount of ram. I'm still wondering if there's some kind of setting(s) that would help if there's just too many filesystems+snapshots around. Or is that not something that could have been the cause of these problems? thanks, Martijn Lina. -- Hostage Keizersgracht 316 1016 EZ Amsterdam tel: +31 (0)20 4632 303 http://www.hostage.nl Once upon a 30 Apr 2015, Martijn hit keys in the following order: > Sorry.... > > No features like compress or dedup, The only devs are the storage hdd's > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > gpt/diskil ONLINE 0 0 0 > gpt/diskir ONLINE 0 0 0 > gpt/diskol ONLINE 0 0 0 > gpt/diskor ONLINE 0 0 0 > > # sysctl -a | grep l2arc > vfs.zfs.l2arc_write_max: 8388608 > vfs.zfs.l2arc_write_boost: 8388608 > vfs.zfs.l2arc_headroom: 2 > vfs.zfs.l2arc_feed_secs: 1 > vfs.zfs.l2arc_feed_min_ms: 200 > vfs.zfs.l2arc_noprefetch: 1 > vfs.zfs.l2arc_feed_again: 1 > vfs.zfs.l2arc_norw: 1 > > an example stuck proc: > > PID TID COMM TDNAME KSTACK > 4232 100606 bsdtar - mi_switch+0xe1 sleepq_wait+0x3a sleeplk+0x15d __lockmgr_args+0xc9e vop_stdlock+0x3c VOP_LOCK1_APV+0xab _vn_lock+0x43 extattr_list_vp+0x3c sys_extattr_list_fd+0xa4 amd64_syscall+0x351 Xfast_syscall+0xfb > > # procstat -kk 4 > PID TID COMM TDNAME KSTACK > 4 100092 zfskern arc_reclaim_thre mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b arc_reclaim_thread+0x301 fork_exit+0x9a fork_trampoline+0xe > 4 100093 zfskern l2arc_feed_threa mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b l2arc_feed_thread+0x16f fork_exit+0x9a fork_trampoline+0xe > 4 100396 zfskern trim zroot mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b trim_thread+0x9e fork_exit+0x9a fork_trampoline+0xe > 4 100406 zfskern txg_thread_enter mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x2bb fork_exit+0x9a fork_trampoline+0xe > 4 100407 zfskern txg_thread_enter mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d txg_sync_thread+0x2eb fork_exit+0x9a fork_trampoline+0xe > > > If you need more i'm happy to provide. > > Could it be hardware related? If the reg ecc memory is broken, wouldn't it show on the console? > > Thanks, > Martijn. > > Once upon a 30 Apr 2015, Robert David hit keys in the following order: > > Hi Martin, > > > > so few information provided to suggest anything. > > > > What about the pool size, free space? Some exceeding quotas? Features enabled > > (compress,dedup)? Zil, l2arc? > > > > Regards, > > Robert. > > > > On Thu, 30 Apr 2015 15:47:00 +0200 > > Martijn wrote: > > > > > Hi, > > > > > > I've been trying to get an important production machine stable again since > > > yesterday afternoon, but to no avail (so far). > > > > > > It seems ZFS is the problem on this box. Situation is as follows: > > > > > > - it used to be fbsd 8.3. after reading about deadlocks which have been fixed > > > in the meantime, i upgrade to 10.1, no change, did zpool upgrade, no change > > > and did zfs upgrade -a also no change... > > > > > > - Its a machine on which each user has a separate zfs filesystem with refquota > > > set. It also did periodic zfs snapshots, every hour (48), day (14) week (8) > > > month (24), which is way too much but at the time of setting it up i thought > > > it couldn't hurt. > > > > > > - After some usage the machine gets stuck when trying to write a file. The > > > process just stops and can't be killed. After some time the whole machine > > > used to become unresponsive in 8.3, but nice 10.1 i can still reach it, > > > although processes attempting to write get stuck for ever. > > > > > > - nothing scary shows in dmesg > > > > > > What can i do? The machine has 24GB of registered ECC ram (17GB free), its a > > > RAID-Z pool with 4 sata hdd's on a LSI SAS3442E-R (1068 chip) in IT mode. > > > > > > loader.conf: > > > > > > vfs.zfs.arc_max=8G > > > vfs.zfs.txg.timeout="5" > > > vfs.zfs.prefetch_disable="1" > > > vfs.zfs.vdev.min_pending="3" > > > vfs.zfs.vdev.max_pending="6" > > > vfs.zfs.txg.write_limit_override=1073741824 > > > > > > I've tried to copy the most important users to another machine but thats gonna > > > take a lot of time. There's 160 users (websites + mailboxes) on it. > > > > > > Any help would be much appreciated! > > > > > > Thanks in advance, > > > > > > Martijn > > > > -- > Hostage > Keizersgracht 316 > 1016 EZ Amsterdam > tel: +31 (0)20 4632 303 > http://www.hostage.nl From owner-freebsd-fs@FreeBSD.ORG Fri May 1 14:58:50 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99EDD927 for ; Fri, 1 May 2015 14:58:50 +0000 (UTC) Received: from mail-ig0-x232.google.com (mail-ig0-x232.google.com [IPv6:2607:f8b0:4001:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A99E105D for ; Fri, 1 May 2015 14:58:50 +0000 (UTC) Received: by iget9 with SMTP id t9so29321783ige.1 for ; Fri, 01 May 2015 07:58:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BaOd5UqlMIv8O7wQWWmhl0ZIMs7xMQy9AI+sH05zxkg=; b=I09DNQdZ0Y11mdsOtsoLvX3HpiFkGqpHwLpS1hF5G6OYJqPfbEYxMNTPMvJuqceffH 4gtE32xQwW3GB/2j7bpBeyV6ZDLnqJJ81z7S8lQtmeoRMYJe5AYtgGHRE6BNfxYOL9WH rJfKJQasmDMbion6/GHonGkW7m5WGVWoePGFidaKP+kyBNV59u9KxNLNRZ1raR7sEgR3 ZraR2o2KWAkA/GPOqBBdxbqQjYtHcy+eIx0xjaoQTbBM9TC37j86mj0wQNadH1StGG2N c5LHZ29Q8yvt8FZLxhzMCI0JJTif9mkaJJD3V5jIVbNAftCJponDxr5+RSiHRwW7ofWE qNOg== MIME-Version: 1.0 X-Received: by 10.107.9.141 with SMTP id 13mr12842214ioj.71.1430492329892; Fri, 01 May 2015 07:58:49 -0700 (PDT) Received: by 10.107.142.74 with HTTP; Fri, 1 May 2015 07:58:49 -0700 (PDT) In-Reply-To: <20150501130924.GA12031@kobol.datajust.com> References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> <20150430143017.GA5573@kobol.office.hostage.nl> <20150501130924.GA12031@kobol.datajust.com> Date: Fri, 1 May 2015 15:58:49 +0100 Message-ID: Subject: Re: ZFS stuck on write From: Tom Evans To: Martijn Cc: FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 14:58:50 -0000 On Fri, May 1, 2015 at 2:09 PM, Martijn wrote: > Hi, > > I've been able to borrow a brand new server from my supplier, put the HDD's in > and all my problems are gone. > > I'm gonna look into this with more detail, because i still want to pin-point > the exact problem and if its really due to broken hardware. > > Is my assumption true that the boost in memory (the new machine has 32GB > instead of 24GB) also could be the cause that everything works again? If i > remember correctly a lot sysctl settings have their defaults based on the total > amount of ram. I'm still wondering if there's some kind of setting(s) that > would help if there's just too many filesystems+snapshots around. Or is that > not something that could have been the cause of these problems? > Stick "hw.physmem=24G" in /boot/loader.conf and reboot. Do you have problems on the new server? Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Fri May 1 15:00:49 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8E019A4 for ; Fri, 1 May 2015 15:00:49 +0000 (UTC) Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com [209.85.212.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65DB0106C for ; Fri, 1 May 2015 15:00:48 +0000 (UTC) Received: by wizk4 with SMTP id k4so55048646wiz.1 for ; Fri, 01 May 2015 08:00:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=XFSVgHs7bgejmhEZ2kEGPmF67rju15+wzd1Hnxa2e18=; b=DgWvny5ideWF/CvJVLwMY4ku7E+F8QDY9JfGVuQ9wKWUGirdlsCRgX1ligQKJxt8xx CYcdM+KvnE6/Sp9h1cHEXj/VYV4d48fwWAQHRJqCgo/qxEWnykuYzFW66E2tUxlF9Tcb AFGkk/idaS2bsKvF9T0Ihs4jqxGVxCgwUvgtm52Ay9o/5dMDn8Ef6aRBkVxrx6Q7r3Vw 2GZA+Ln7lNR/TSTG0+RX44nJwCMaBeWXA2Bk8PSwPu039mXP3SjSqBKUuE/eM53/lqXe UCFT3EiKdv2lLXJdroptIchIpBZVIwyNbcdM3PIq/bghig0UL99rFqg5Dv7LRDjIsLqy NWMw== X-Gm-Message-State: ALoCoQmInMlU/04BJxMW7AQ7IQUjv385T/yXKtwInE0Y/IWI8lMFp5eg1zMVkzMguKt7RjwZJtiL X-Received: by 10.180.93.193 with SMTP id cw1mr15535356wib.12.1430492439606; Fri, 01 May 2015 08:00:39 -0700 (PDT) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id es5sm7902688wjc.30.2015.05.01.08.00.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2015 08:00:38 -0700 (PDT) Message-ID: <55439512.9020201@multiplay.co.uk> Date: Fri, 01 May 2015 16:00:34 +0100 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS stuck on write References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> <20150430143017.GA5573@kobol.office.hostage.nl> <20150501130924.GA12031@kobol.datajust.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 15:00:49 -0000 On 01/05/2015 15:58, Tom Evans wrote: > On Fri, May 1, 2015 at 2:09 PM, Martijn wrote: >> Hi, >> >> I've been able to borrow a brand new server from my supplier, put the HDD's in >> and all my problems are gone. >> >> I'm gonna look into this with more detail, because i still want to pin-point >> the exact problem and if its really due to broken hardware. >> >> Is my assumption true that the boost in memory (the new machine has 32GB >> instead of 24GB) also could be the cause that everything works again? If i >> remember correctly a lot sysctl settings have their defaults based on the total >> amount of ram. I'm still wondering if there's some kind of setting(s) that >> would help if there's just too many filesystems+snapshots around. Or is that >> not something that could have been the cause of these problems? >> > Stick "hw.physmem=24G" in /boot/loader.conf and reboot. Do you have > problems on the new server? > You don't have swap on ZFS do you? Regards Steve From owner-freebsd-fs@FreeBSD.ORG Fri May 1 16:27:25 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DBA3603 for ; Fri, 1 May 2015 16:27:25 +0000 (UTC) Received: from smtp.hostage.nl (smtp.hostage.nl [109.72.93.221]) by mx1.freebsd.org (Postfix) with ESMTP id 186F11DE5 for ; Fri, 1 May 2015 16:27:24 +0000 (UTC) Date: Fri, 1 May 2015 18:26:37 +0200 From: Martijn To: Tom Evans Cc: FreeBSD FS Subject: Re: ZFS stuck on write Message-ID: <20150501162635.GA19681@kobol.datajust.com> References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> <20150430143017.GA5573@kobol.office.hostage.nl> <20150501130924.GA12031@kobol.datajust.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 16:27:25 -0000 Wow can you do that? Nice one, thanks! I'll try it tonight. Once upon a 01 May 2015, Tom Evans hit keys in the following order: > On Fri, May 1, 2015 at 2:09 PM, Martijn wrote: > > I'm gonna look into this with more detail, because i still want to pin-point > > the exact problem and if its really due to broken hardware. > > > > Is my assumption true that the boost in memory (the new machine has 32GB > > instead of 24GB) also could be the cause that everything works again? If i > > remember correctly a lot sysctl settings have their defaults based on the total > > amount of ram. I'm still wondering if there's some kind of setting(s) that > > would help if there's just too many filesystems+snapshots around. Or is that > > not something that could have been the cause of these problems? > > > > Stick "hw.physmem=24G" in /boot/loader.conf and reboot. Do you have > problems on the new server? > > Cheers > > Tom -- Hostage Keizersgracht 316 1016 EZ Amsterdam tel: +31 (0)20 4632 303 http://www.hostage.nl From owner-freebsd-fs@FreeBSD.ORG Fri May 1 17:20:29 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 462B653E for ; Fri, 1 May 2015 17:20:29 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7865F1331 for ; Fri, 1 May 2015 17:20:27 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA19958; Fri, 01 May 2015 20:20:19 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1YoEbr-000KJF-Hc; Fri, 01 May 2015 20:20:19 +0300 Message-ID: <5543B599.6000503@FreeBSD.org> Date: Fri, 01 May 2015 20:19:21 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Martijn , freebsd-fs@FreeBSD.org Subject: Re: ZFS stuck on write References: <20150430134659.GA4950@kobol.office.hostage.nl> In-Reply-To: <20150430134659.GA4950@kobol.office.hostage.nl> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 17:20:29 -0000 On 30/04/2015 16:47, Martijn wrote: > - After some usage the machine gets stuck when trying to write a file. The > process just stops and can't be killed. After some time the whole machine > used to become unresponsive in 8.3, but nice 10.1 i can still reach it, > although processes attempting to write get stuck for ever. > > - nothing scary shows in dmesg > > What can i do? https://wiki.freebsd.org/AvgZfsDeadlockDebug -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri May 1 22:44:21 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C7F1CDB for ; Fri, 1 May 2015 22:44:21 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E06A71725 for ; Fri, 1 May 2015 22:44:20 +0000 (UTC) Received: by pdbqa5 with SMTP id qa5so104679031pdb.1 for ; Fri, 01 May 2015 15:44:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=N9esgJjf3zHXiYbbrUQcC0SgXMT+8UCg8W9B8yjg5y8=; b=0Kj0HjuwgbK3hJg/j4OEtRXfAXCUQ97MDUlxpzRDJAoxJnjlaDIgDhuqT5zr5f7DZo xRPo1zYsGwRxZKxsQLYrcPN5iVyjW3zHY91m/Ei4HicGIrn17JbezL45Z8G3ZtLGKqpL Ly1LCaJDsHK2UqiPGd0W5EE7skcHv7144de4f3FPYDJ7yo8pavBM/64q5+eRtTOoetx8 trmy0Q9BSHx8RCtMS3MDNAmtQd8oTAqJh0AKTUaYYPr6zFxE5wlYNFLQxDTu75rQkYXZ HRq/elkymXAeYPvPS5WGEzor/BSL5nC96XJzgpoDQqFmprNv+W+71LEZvHIEJZL6ARyS ekOw== X-Received: by 10.66.249.168 with SMTP id yv8mr21914487pac.49.1430520260146; Fri, 01 May 2015 15:44:20 -0700 (PDT) Received: from [192.168.0.3] (c-50-186-4-10.hsd1.or.comcast.net. [50.186.4.10]) by mx.google.com with ESMTPSA id sm7sm5811320pac.45.2015.05.01.15.44.18 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2015 15:44:18 -0700 (PDT) Message-ID: <554401C1.9020801@gmail.com> Date: Fri, 01 May 2015 15:44:17 -0700 From: Lacey Powers User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: vfs.zfs.write_to_degraded missing in 10.1? References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 22:44:21 -0000 Hello Gabor, It looks like this was an upstream behavior that was changed. Here's the bug from Illumos: https://www.illumos.org/issues/4976 And it was pulled in in r269010in FreeBSD https://svnweb.freebsd.org/base?view=revision&revision=269010 I'm not entirely sure about what the changes mean, but it seems like it will always try for the behavior of the vfs.zfs.write_to_degraded sysctl, except in instances where the vdev is non-redundant and top-level.It might help to ask for further clarification on an Illumos or OpenZFS mailing list, or in the IRC channels on freenode for each, #illumos or #openzfs. Hope that helps. =) Regards, Lacey On 04/29/2015 14:40, Gabor Radnai wrote: > Hi, > > I run 10.1-stable (r281854), is it possible I missed the announcement of > removing vfs.zfs.write_to_degraded sysctl variable? > > The Handbook's Advanced Topics 20.6 is mentioning this variable as "to > allow writing to degraded vdevs to prevent the amount of free space across > the vdevs from becoming unbalanced, which will reduce read and write > performance." > > How can this be achieved currently? > > Thanks. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to"freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Sat May 2 09:44:18 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5682B5C2 for ; Sat, 2 May 2015 09:44:18 +0000 (UTC) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 12C6D15FE for ; Sat, 2 May 2015 09:44:18 +0000 (UTC) Received: from m.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id A41701E9EB3; Sat, 2 May 2015 11:44:14 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id E3AFD63347; Sat, 2 May 2015 11:38:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([IPv6:2a01:170:1110:8001::25:1]) by amavis.cksoft.de (amavis.cksoft.de [IPv6:2a01:170:1110:8001::25:a1]) (amavisd-new, port 10041) with ESMTP id xh2ZTr0JZwTm; Sat, 2 May 2015 11:44:02 +0200 (CEST) Received: from pohjola.cksoft.de (unknown [IPv6:2a01:170:1110:8001:fc39:e03:b5a9:1d44]) by m.cksoft.de (Postfix) with ESMTP id 1138363345; Sat, 2 May 2015 11:38:19 +0200 (CEST) Received: by pohjola.cksoft.de (Postfix, from userid 1000) id 4489DD7997; Sat, 2 May 2015 11:44:13 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by pohjola.cksoft.de (Postfix) with ESMTP id 3EB21D788C; Sat, 2 May 2015 11:44:13 +0200 (CEST) Date: Sat, 2 May 2015 11:44:13 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@pohjola.cksoft.de Reply-To: Christian Kratzer To: Martijn cc: Robert David , freebsd-fs@freebsd.org Subject: Re: ZFS stuck on write In-Reply-To: <20150430143017.GA5573@kobol.office.hostage.nl> Message-ID: References: <20150430134659.GA4950@kobol.office.hostage.nl> <20150430161902.4868094c@robert-notebook> <20150430143017.GA5573@kobol.office.hostage.nl> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 May 2015 09:44:18 -0000 hi, On Thu, 30 Apr 2015, Martijn wrote: > Sorry.... > Could it be hardware related? If the reg ecc memory is broken, wouldn't it show on the console? I had this recently with one of my servers. Any attempts to write to the pool resulted in the writes stalling inifinitely on zio_wait. The FreeBSD 10 system eventually locked up propably because it was unable to flush any IO to the pool. The root pool was on a separate ssd mirror on a PCIe card and was unaffacted. Reflashing the IT firmware on the HBA did not help. Swapping the LSI HBA fully resolved the issue. I wasted lots of time before I finally swapped the controller partly because I did not have spares flying around. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-fs@FreeBSD.ORG Sat May 2 13:09:02 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5AB8544 for ; Sat, 2 May 2015 13:09:02 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90120182F for ; Sat, 2 May 2015 13:09:02 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t42D92vE000229 for ; Sat, 2 May 2015 13:09:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 197170] [nfs] Kernel panic when receiving invalid RPC requests Date: Sat, 02 May 2015 13:09:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: rmacklem@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 May 2015 13:09:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197170 Rick Macklem changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rmacklem@FreeBSD.org Assignee|freebsd-fs@FreeBSD.org |rmacklem@FreeBSD.org Status|New |Closed Resolution|--- |FIXED --- Comment #1 from Rick Macklem --- Patch has been committed and MFC'd. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat May 2 21:03:54 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8AF9D23 for ; Sat, 2 May 2015 21:03:54 +0000 (UTC) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7668815D7 for ; Sat, 2 May 2015 21:03:54 +0000 (UTC) Received: by qkhg7 with SMTP id g7so67273028qkh.2 for ; Sat, 02 May 2015 14:03:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=79ksxNbyraLl3/qbRPEAp929EvJI6Wcf7bXbi0uSZN0=; b=QWpzYroqXbotZKAKms1zkculexn1e/I0Thv0TG2jMo8ZvpzA/eB6DMAEQsjFPoYGoa EZs4iAGyghYDHLTVRbRRQc/J5jXlPravnmoJx6Giims3wlR6l+jJbdkb8oD3T5jZ+ewP auzlHfoVrrtyqOwlHsiiSxrYUh2xErNNPyPPAlSOxZtK7WdrmuBo5VOW9/x1CUOApowo jg/uPwjPzqBWA1vmPKe5v/V4/yteuGFRaP4qp0/+wTlFqby1p4kduo+qgzODBIJ/dE+F XFIEYhDJakHaUQAz7YwrtC/CWQjHgjFTJm4QID+6E9ozRGyv0Rc9cGg38TT0ZbtXSc03 Uw3w== X-Received: by 10.55.15.129 with SMTP id 1mr32090541qkp.29.1430600633477; Sat, 02 May 2015 14:03:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.96.118 with HTTP; Sat, 2 May 2015 14:03:33 -0700 (PDT) From: Gabor Radnai Date: Sat, 2 May 2015 23:03:33 +0200 Message-ID: Subject: Re: vfs.zfs.write_to_degraded missing in 10.1? To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 May 2015 21:03:54 -0000 Hi Lacey, Thanks for prompt answer. Quick test showed that behavior indeed changed, ie. by default a degraded redundant pool is writable. (Nevertheless Handbook is out-of-date this regards and thus misleading. Hope it will catch at some point.) Thanks again. Best regards, Gabor