From owner-freebsd-fs@FreeBSD.ORG Sun Aug 24 00:06:42 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5CC8D45; Sun, 24 Aug 2014 00:06:42 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 91D2A3C5E; Sun, 24 Aug 2014 00:06:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=OiOoOSnfDRy6fDsH3GUInNDF+V5EnI3M7GVeybSPbl0=; b=NRKJg0CcXQqb3dEdHO/zgtw4hGutbUZAOx/w0NbKhKLtppWUSuszTssmoCn8TiHZdXZlTvDNvoCQCYTXRg+w+yrWzeBoWN09j57zXfaYX0B1gdZ1qSLwluIL+3H3CXvaTW7VL6R9Afylahni3MjIPXmuCO2FKG49aut4xejYiRY=; Received: from localhost.lerctr.org ([127.0.0.1]:36570 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XLLKR-000MY1-9G; Sat, 23 Aug 2014 19:06:40 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sat, 23 Aug 2014 19:06:39 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sat, 23 Aug 2014 19:06:39 -0500 From: Larry Rosenman To: Steven Hartland Subject: Re: [Bug 187594] [zfs] [patch] ZFS ARC behavior problem and fix In-Reply-To: References: Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.2 X-Spam-Score: -3.6 (---) X-LERCTR-Spam-Score: -3.6 (---) X-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.668 X-LERCTR-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.668 Cc: freebsd-fs@freebsd.org, bugzilla-noreply@freebsd.org, owner-freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 00:06:42 -0000 On 2014-08-23 09:13, Steven Hartland wrote: > ----- Original Message ----- From: > > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594 >> >> --- Comment #25 from fullermd@over-yonder.net --- >> Having run it for a few months on a number of boxes now, my general >> impression >> is that it seems like it goes a little _too_ far (with default options >> anyway; >> I haven't tried any tuning) toward making the ARC give up its lunch >> money to >> anybody who looks threateningly at it. It feels like it should be a >> bit more >> aggressive, and historically was and did fine. >> >> However, it's still _much_ nicer than the unpatched case, where the >> rest of the >> system starves and hides out in the swap space. So from here, while >> perhaps >> imperfect and in need of some tuning work, it's still a significant >> improvement >> on the prior state, so landing it sounds just fine to me. > > The attached updated patch, which has been cleaned up and hammered hard > at > the event here I'll look to commit to head soon if there are no > objections. > > Regards > Steve > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" and a full buildworld buildkernel garners" cc -O2 -pipe -I/usr/src/cddl/usr.bin/zinject/../../../sys/cddl/compat/opensolaris -I/usr/src/cddl/usr.bin/zinject/../../compat/opensolaris/include -I/usr/src/cddl/usr.bin/zinject/../../compat/opensolaris/lib/libumem -I/usr/src/cddl/usr.bin/zinject/../../contrib/opensolaris/lib/libzfs/common -I/usr/src/cddl/usr.bin/zinject/../../contrib/opensolaris/lib/libzfs_core/common -I/usr/src/cddl/usr.bin/zinject/../../contrib/opensolaris/lib/libzpool/common -I/usr/src/cddl/usr.bin/zinject/../../contrib/opensolaris/lib/libnvpair -I/usr/src/cddl/usr.bin/zinject/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs -I/usr/src/cddl/usr.bin/zinject/../../../sys/cddl/contrib/opensolaris/uts/common/sys -I/usr/src/cddl/usr.bin/zinject/../../../sys/cddl/contrib/opensolaris/uts/common -I/usr/src/cddl/usr.bin/zinject/../../../sys/cddl/contrib/opensolaris/common/zfs/ -I/usr/src/cddl/usr.bin/zinject/../../contrib/opensolaris/head -I/usr/src/cddl/usr.bin/zinject/../../lib/libumem -DNEED_SOLARIS_BOOLEAN -std=gnu89 -Qunused-arguments -fstack-protector -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses -Wno-unknown-pragmas -o zinject zinject.o translate.o -lgeom -lm -lnvpair -lumem -luutil -lzfs_core -lzfs -lzpool /usr/obj/usr/src/tmp/usr/lib/libzpool.so: undefined reference to `kmem_free_target_size' cc: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 Stop. make[5]: stopped in /usr/src/cddl/usr.bin/zinject *** Error code 1 Stop. make[4]: stopped in /usr/src/cddl/usr.bin *** Error code 1 Stop. make[3]: stopped in /usr/src/cddl *** Error code 1 Stop. make[2]: stopped in /usr/src *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sun Aug 24 09:10:30 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA6E03B4; Sun, 24 Aug 2014 09:10:30 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id AC1AA3832; Sun, 24 Aug 2014 09:10:30 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 4C80020E7088A; Sun, 24 Aug 2014 09:10:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id B991B20E70885; Sun, 24 Aug 2014 09:10:26 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Larry Rosenman" References: Subject: Re: [Bug 187594] [zfs] [patch] ZFS ARC behavior problem and fix Date: Sun, 24 Aug 2014 10:10:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, bugzilla-noreply@freebsd.org, owner-freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 09:10:31 -0000 ----- Original Message ----- From: "Larry Rosenman" snip... > -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare > -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion > -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses > -Wno-unknown-pragmas -o zinject zinject.o > translate.o -lgeom -lm -lnvpair -lumem -luutil -lzfs_core -lzfs -lzpool > /usr/obj/usr/src/tmp/usr/lib/libzpool.so: undefined reference to > `kmem_free_target_size' > cc: error: linker command failed with exit code 1 (use -v to see > invocation) > *** Error code 1 Fixed buildworld, was only a kernel patch ;-) New patch attached to the bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594 Regards Steve From owner-freebsd-fs@FreeBSD.ORG Sun Aug 24 15:07:24 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB877E50 for ; Sun, 24 Aug 2014 15:07:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2C863628 for ; Sun, 24 Aug 2014 15:07:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7OF7Ogl037943 for ; Sun, 24 Aug 2014 15:07:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191510] [zfs] ZFS doesn't use all available memory Date: Sun, 24 Aug 2014 15:07:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: smh@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: smh@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 15:07:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191510 Steven Hartland changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-fs@FreeBSD.org |smh@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 00:55:19 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EECD3EFF for ; Mon, 25 Aug 2014 00:55:19 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9FB3391E for ; Mon, 25 Aug 2014 00:55:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=tC6idVw7bn0NCb49tPZ976+tOHBsbeO4bn3FgJPfMSw=; b=koFeDPsdf528bBPQ3WBOr2c6SvhKFMdUCtkwoyef+6/vtdA9oD4tsQCSnoMamXsVe6Rq0BUotihYFlvAsj/goWquRdAB6vVLeBOf8BsmIID5zPjFbHX+9K8Wl5IrkAG5UkcMuf7da42XNCxjRqKxY7OhjBh2VFnK2imF8KvTfOM=; Received: from localhost.lerctr.org ([127.0.0.1]:14465 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XLiZ1-00051O-Ti; Sun, 24 Aug 2014 19:55:17 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 24 Aug 2014 19:55:15 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 24 Aug 2014 19:55:15 -0500 From: Larry Rosenman To: Steven Hartland Subject: Re: [Bug 187594] [zfs] [patch] ZFS ARC behavior problem and fix In-Reply-To: References: Message-ID: <1e0806c88b6d2061a661dc870071e5ad@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.2 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 00:55:20 -0000 On 2014-08-24 04:10, Steven Hartland wrote: > ----- Original Message ----- From: "Larry Rosenman" > snip... >> -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable >> -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality >> -Wno-unused-function -Wno-enum-conversion -Wno-switch >> -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses >> -Wno-unknown-pragmas -o zinject zinject.o translate.o -lgeom -lm >> -lnvpair -lumem -luutil -lzfs_core -lzfs -lzpool >> /usr/obj/usr/src/tmp/usr/lib/libzpool.so: undefined reference to >> `kmem_free_target_size' >> cc: error: linker command failed with exit code 1 (use -v to see >> invocation) >> *** Error code 1 > > Fixed buildworld, was only a kernel patch ;-) > > New patch attached to the bug report: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594 > > Regards > Steve thanks. Any chance of a version for head? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 08:00:11 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35C493D5 for ; Mon, 25 Aug 2014 08:00:11 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 163003C12 for ; Mon, 25 Aug 2014 08:00:11 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7P80AZQ040513 for ; Mon, 25 Aug 2014 08:00:10 GMT (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201408250800.s7P80AZQ040513@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 25 Aug 2014 08:00:10 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 08:00:11 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (5 bugs) Bug 133174: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133174 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [msdosfs] [patch] msdosfs must support multibyte international characters in file names Bug 136470: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=136470 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] Cannot mount / in read-only, over NFS Bug 139651: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139651 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] mount(8): read-only remount of NFS volume does not work Bug 144447: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144447 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [zfs] sharenfs fsunshare() & fsshare_main() non functional Bug 155411: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=155411 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [regression] [8.2-release] [tmpfs]: mount: tmpfs : No space left on device From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 10:39:42 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DB65FD2 for ; Mon, 25 Aug 2014 10:39:42 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id C3DCF3B26 for ; Mon, 25 Aug 2014 10:39:41 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 67C8720E7088B; Mon, 25 Aug 2014 10:39:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id B9A4720E70885; Mon, 25 Aug 2014 10:39:28 +0000 (UTC) Message-ID: <86CBBB168E6E4C3F9D0E917A47143355@multiplay.co.uk> From: "Steven Hartland" To: "Larry Rosenman" References: <1e0806c88b6d2061a661dc870071e5ad@thebighonker.lerctr.org> Subject: Re: [Bug 187594] [zfs] [patch] ZFS ARC behavior problem and fix Date: Mon, 25 Aug 2014 11:39:28 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 10:39:42 -0000 ----- Original Message ----- From: "Larry Rosenman" > On 2014-08-24 04:10, Steven Hartland wrote: >> ----- Original Message ----- From: "Larry Rosenman" >> snip... >>> -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare >>> -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion >>> -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses >>> -Wno-unknown-pragmas -o zinject zinject.o >>> translate.o -lgeom -lm -lnvpair -lumem -luutil -lzfs_core -lzfs -lzpool >>> /usr/obj/usr/src/tmp/usr/lib/libzpool.so: undefined reference to >>> `kmem_free_target_size' >>> cc: error: linker command failed with exit code 1 (use -v to see >>> invocation) >>> *** Error code 1 >> >> Fixed buildworld, was only a kernel patch ;-) >> >> New patch attached to the bug report: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594 >> >> Regards >> Steve > thanks. > > Any chance of a version for head? Sure its attached to the bug report now. Regards Steve From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 17:25:08 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6AA0771; Mon, 25 Aug 2014 17:25:08 +0000 (UTC) Received: from caida.org (rommie.caida.org [192.172.226.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A7E113458; Mon, 25 Aug 2014 17:25:08 +0000 (UTC) Message-ID: <53FB716D.1060007@caida.org> Date: Mon, 25 Aug 2014 10:25:01 -0700 From: Daniel Andersen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: Bryan Drewery , freebsd-fs@freebsd.org Subject: Re: Process enters unkillable state and somewhat wedges zfs References: <53F25402.1020907@caida.org> <53F4E3C0.6000406@FreeBSD.org> In-Reply-To: <53F4E3C0.6000406@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 17:25:08 -0000 On 08/20/2014 11:06 AM, Bryan Drewery wrote: > On 8/18/2014 2:29 PM, Daniel Andersen wrote: >> We are currently experiencing a strange problem that sort of locks up one of our zfs pools. This is on a FreeBSD 10 >> machine. Let me give a rough layout of our system to better describe what is happening: > > When it happens get the output of 'procstat -kka|grep zfs' please. > I'd read something about that while attempting to debug the last few times, so I happen to have that output, I believe. Here are the last two examples ( the latter having had more wedged processes ): 0 100687 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 0 101055 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 38 100306 zfskern arc_reclaim_thre mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe 38 100307 zfskern l2arc_feed_threa mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe 38 100672 zfskern trim tank mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe 38 100688 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b fork_exit+0x9a fork_trampoline+0xe 38 100689 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe 38 101054 zfskern trim work mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe 38 101056 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b fork_exit+0x9a fork_trampoline+0xe 38 101057 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe 16656 101610 ls - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb and 0 100688 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 0 101056 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 38 100306 zfskern arc_reclaim_thre mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe 38 100307 zfskern l2arc_feed_threa mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe 38 100673 zfskern trim tank mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe 38 100689 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b fork_exit+0x9a fork_trampoline+0xe 38 100690 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d zio_wait+0x5b vdev_uberblock_sync_list+0xad vdev_config_sync+0x118 spa_sync+0x827 txg_sync_thread+0x375 fork_exit+0x9a fork_trampoline+0xe 38 101055 zfskern trim work mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe 38 101057 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b fork_exit+0x9a fork_trampoline+0xe 38 101058 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe 2638 101160 nfsd nfsd: service mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_vget+0xfe nullfs_vget+0x54 nfsrvd_readdirplus+0x905 nfsrvd_dorpc+0x773 nfssvc_program+0x4f6 svc_run_internal+0x1f9 svc_thread_start+0xb fork_exit+0x9a fork_trampoline+0xe 3891 101641 collectd - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb 11808 101802 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb 13900 101899 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb 17961 102536 df - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb 38964 102279 ls - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb 42315 102539 df - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb 42335 101782 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 21:15:05 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D683E61B for ; Mon, 25 Aug 2014 21:15:05 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63DA33D2A for ; Mon, 25 Aug 2014 21:15:05 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id u56so13925954wes.0 for ; Mon, 25 Aug 2014 14:15:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=g7rM7PtsUwXDKRKh/9HHjhueUHLix0VPwH4i6d6SUpU=; b=itIq6Ps1drI+HEDuADzYBeyu5anelxnbdo7PnT0krf9ssr95K8f3TaLiI/RMuqzj8K 4sHYyK8T5Ny/9Zh++YqlsPspB9Fp3LdfClQWuftqk9S+z67easYUQayzHjbpzx/6YhwQ 5BRAneq2AA0fdcJ/fyPuir6jKeFpneUjrE+8qIXfFue88mgGfANUpnbw+Mlg2mnV0K3f N0g4U+f5nT80BLB+0SLbxYhwiEobJ8rQ+bbsxJxgkjUPp67se+mdgtKDc5jKtISWdC/5 cjqZSxN3iGbCZs3vGIpdYy8c+djMVCCgV5vCCYammxrxzM9YtAixnDUvDZbDxIUCUdmd 3YrQ== X-Received: by 10.180.106.8 with SMTP id gq8mr17650082wib.56.1409001302949; Mon, 25 Aug 2014 14:15:02 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id p10sm2521958wjo.22.2014.08.25.14.15.01 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Aug 2014 14:15:01 -0700 (PDT) Sender: Baptiste Daroussin Date: Mon, 25 Aug 2014 23:14:59 +0200 From: Baptiste Daroussin To: Jordan Hubbard Subject: Re: FreeBSD support being added to GlusterFS Message-ID: <20140825211459.GB65120@ivaldir.etoilebsd.net> References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="V0207lvV8h4k8FAm" Content-Disposition: inline In-Reply-To: <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 21:15:06 -0000 --V0207lvV8h4k8FAm Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 16, 2014 at 07:57:02PM -0700, Jordan Hubbard wrote: > Thanks! Just to make sure I understand for our own ports repo in FreeNAS= -land: >=20 > 1. You have updated the tarball but Baptiste is still maintaining the por= t, correct? >=20 > If Baptiste would like someone else to take it over, I can probably coax = one of our resident committers to do it as well. We frankly have not been = able to do any testing of glusterfs in FreeNAS yet since glusterd just catc= hes a signal 10 whenever any glusterfs operation is performed. Since this = is also really early BETA software, I figured the FreeBSD port was still a = bit green and I'd simply wait for it to get a bit more mature. Now that yo= u say you=E2=80=99ve gotten a regression suite up and running, I=E2=80=99m = guessing the time to really push on it in FreeNAS 9.3-ALPHA is probably soo= n? >=20 I do not plan to maintain the port I'm just trying to help :) I would prefe= r one of your resident committer you has FreeNAS vendor will probably have more feedbacks from glusterfs user and probably have more opportunities to test = it. I have no usage for glusterfs but deeply support having glusterfs is FreeBS= D :) > 2. That said, it also sounds like updating the glusterfs port is going to= have to wait for your cmockery2 port to get accepted and committed to the = tree, since otherwise it won=E2=80=99t build - is that a correct assessment? >=20 > Cheers, >=20 > - Jordan --V0207lvV8h4k8FAm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlP7p1MACgkQ8kTtMUmk6ExJpwCeOMqX9CzDepUJrQJGzCQ43kb9 oWsAn1RtmEl+ImOuNB0cem5CYAXK7OS/ =JwWV -----END PGP SIGNATURE----- --V0207lvV8h4k8FAm-- From owner-freebsd-fs@FreeBSD.ORG Mon Aug 25 22:23:46 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC94E993; Mon, 25 Aug 2014 22:23:46 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E401344F; Mon, 25 Aug 2014 22:23:46 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 343087E7C9; Mon, 25 Aug 2014 15:23:46 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 84276-04; Mon, 25 Aug 2014 15:23:46 -0700 (PDT) Received: from [10.2.0.80] (unknown [10.2.0.80]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id C4AD87E7BC; Mon, 25 Aug 2014 15:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1409005425; bh=44pVo0aurDyTU6gg3FxUYYwpwscPieleLONBkjfF88w=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=j1ClsZc6jDkWSqcAHSNbJErqlxs9pYdoIAUtKc4Mu2DSZ45zvGleTlD3SCaSOqFGp kvs3N1/Q6PX4nyF/62JS68YJXOfqv1xllemaeH/npPYXDWYQOcD6Hv0scoC4U7TxQz sbsS7Oara6yquj/n/ZHJ9/EW8JBBRNaWlvM1Ywjw= Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1973.6\)) Subject: Re: FreeBSD support being added to GlusterFS From: Jordan Hubbard In-Reply-To: <20140825211459.GB65120@ivaldir.etoilebsd.net> Date: Mon, 25 Aug 2014 15:24:43 -0700 Message-Id: References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> <20140825211459.GB65120@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1973.6) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 22:23:46 -0000 > On Aug 25, 2014, at 2:14 PM, Baptiste Daroussin = wrote: >=20 > I do not plan to maintain the port I'm just trying to help :) I would = prefer one > of your resident committer you has FreeNAS vendor will probably have = more > feedbacks from glusterfs user and probably have more opportunities to = test it. > I have no usage for glusterfs but deeply support having glusterfs is = FreeBSD :) Fair enough! I will attempt to make those arrangements (just as soon = as everyone gets back from VMWorld and/or China :-) ). In the = meantime, here=E2=80=99s a patch to bring the port up to date: I=E2=80=99ve already committed it to the FreeNAS repo and will be = building a nightly with this version of glusterfs in it shortly! Thanks, - Jordan From owner-freebsd-fs@FreeBSD.ORG Tue Aug 26 00:09:50 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 191503DD for ; Tue, 26 Aug 2014 00:09:50 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 00C213CCD for ; Tue, 26 Aug 2014 00:09:50 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7Q09nHS016017 for ; Tue, 26 Aug 2014 00:09:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 144234] [zfs] Cannot boot machine with recent gptzfsboot code [regression] Date: Tue, 26 Aug 2014 00:09:50 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: elliot.robinson@argiopetech.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Aug 2014 00:09:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144234 --- Comment #4 from Elliot Robinson --- I have attempted building sys/boot from commit 199714 under 9.3 (doesn't build under 10 with clang). It complains about the ZFS version being too new, so this is no longer an immediately viable fix under FreeBSD 10. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 27 18:39:24 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1286FA60 for ; Wed, 27 Aug 2014 18:39:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED41638D1 for ; Wed, 27 Aug 2014 18:39:23 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7RIdNPQ091440 for ; Wed, 27 Aug 2014 18:39:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 158802] amd(8) ICMP storm and unkillable process. Date: Wed, 27 Aug 2014 18:39:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Aug 2014 18:39:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=158802 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Issue Resolved CC| |jhb@FreeBSD.org Resolution|--- |Unable to Reproduce --- Comment #4 from John Baldwin --- Submitter reports issue no longer occurs with 9.2 and 10.0. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 27 19:14:42 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FF8BD3F for ; Wed, 27 Aug 2014 19:14:42 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 166F83D79 for ; Wed, 27 Aug 2014 19:14:42 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7RJEf2E016915 for ; Wed, 27 Aug 2014 19:14:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 144234] [zfs] Cannot boot machine with recent gptzfsboot code [regression] Date: Wed, 27 Aug 2014 19:14:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Aug 2014 19:14:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144234 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhb@FreeBSD.org --- Comment #5 from John Baldwin --- Created attachment 146387 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=146387&action=edit extra_debug_info.patch Please try this patch. It will not fix the problem, but it will provide more details about the request that fails. Error code 1 from the BIOS is roughly equivalent to EINVAL. This will hopefully dump enough details that we can figure out the issue. Given that the reporter claims the issue cropped up when we started letting gptboot use memory above 1MB, my guess is that something in ZFS is doing I/O using a buffer allocated via malloc(). Note that the biosdisk.c code in the loader has more complicated logic to use bounce buffers to ensure that all I/O is done in a lower memory range. Probably this needs to be ported to drv.c for gptboot/gptzfsboot. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 27 20:10:38 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57663BA0 for ; Wed, 27 Aug 2014 20:10:38 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3ED283668 for ; Wed, 27 Aug 2014 20:10:38 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7RKAcNo052881 for ; Wed, 27 Aug 2014 20:10:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 144234] [zfs] Cannot boot machine with recent gptzfsboot code [regression] Date: Wed, 27 Aug 2014 20:10:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Aug 2014 20:10:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144234 --- Comment #6 from John Baldwin --- Created attachment 146392 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=146392&action=edit bounce_buffer_1.patch Here is a first cut at a bounce buffer patch for gptboot/gptzfsboot. I have not tested this at all (though it does compile). This might actually fix the issue however as it should use a bounce buffer for any I/O request to a destination buffer above 1MB. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 27 20:43:08 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF76FFC3; Wed, 27 Aug 2014 20:43:08 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 93AD73A33; Wed, 27 Aug 2014 20:43:08 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 46FCCB948; Wed, 27 Aug 2014 16:43:07 -0400 (EDT) From: John Baldwin To: freebsd-fs@freebsd.org Subject: Re: Process enters unkillable state and somewhat wedges zfs Date: Wed, 27 Aug 2014 16:39:09 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <53F25402.1020907@caida.org> <53F4E3C0.6000406@FreeBSD.org> <53FB716D.1060007@caida.org> In-Reply-To: <53FB716D.1060007@caida.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201408271639.09352.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 27 Aug 2014 16:43:07 -0400 (EDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Aug 2014 20:43:09 -0000 On Monday, August 25, 2014 1:25:01 pm Daniel Andersen wrote: > On 08/20/2014 11:06 AM, Bryan Drewery wrote: > > On 8/18/2014 2:29 PM, Daniel Andersen wrote: > >> We are currently experiencing a strange problem that sort of locks up one of our zfs pools. This is on a FreeBSD 10 > >> machine. Let me give a rough layout of our system to better describe what is happening: > > > > When it happens get the output of 'procstat -kka|grep zfs' please. > > > > I'd read something about that while attempting to debug the last few times, so I happen to have that output, I believe. > Here are the last two examples ( the latter having had more wedged processes ): > > 0 100687 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 > fork_exit+0x9a fork_trampoline+0xe > 0 101055 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 > fork_exit+0x9a fork_trampoline+0xe > 38 100306 zfskern arc_reclaim_thre mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe > 38 100307 zfskern l2arc_feed_threa mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe > 38 100672 zfskern trim tank mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe > 38 100688 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b > fork_exit+0x9a fork_trampoline+0xe > 38 100689 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe > 38 101054 zfskern trim work mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe > 38 101056 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b > fork_exit+0x9a fork_trampoline+0xe > 38 101057 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe > 16656 101610 ls - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 > sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb > > > and > > 0 100688 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 > fork_exit+0x9a fork_trampoline+0xe > 0 101056 kernel zfs_vn_rele_task mi_switch+0xde sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 > fork_exit+0x9a fork_trampoline+0xe > 38 100306 zfskern arc_reclaim_thre mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe > 38 100307 zfskern l2arc_feed_threa mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe > 38 100673 zfskern trim tank mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe > 38 100689 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b > fork_exit+0x9a fork_trampoline+0xe > 38 100690 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d zio_wait+0x5b > vdev_uberblock_sync_list+0xad vdev_config_sync+0x118 spa_sync+0x827 txg_sync_thread+0x375 fork_exit+0x9a > fork_trampoline+0xe > 38 101055 zfskern trim work mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe > 38 101057 zfskern txg_thread_enter mi_switch+0xde sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b > fork_exit+0x9a fork_trampoline+0xe > 38 101058 zfskern txg_thread_enter mi_switch+0xde sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d > txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe > 2638 101160 nfsd nfsd: service mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_vget+0xfe nullfs_vget+0x54 nfsrvd_readdirplus+0x905 > nfsrvd_dorpc+0x773 nfssvc_program+0x4f6 svc_run_internal+0x1f9 svc_thread_start+0xb fork_exit+0x9a fork_trampoline+0xe > 3891 101641 collectd - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f > sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb > 11808 101802 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 > sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb > 13900 101899 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 > sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb > 17961 102536 df - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f > sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb > 38964 102279 ls - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 > sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb > 42315 102539 df - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statfs+0x6f > sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb > 42335 101782 sendsize - mi_switch+0xde sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 > vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 namei+0x504 kern_statat_vnhook+0xa5 > sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb These are all blocked in "zfs" then. (For future reference, the 'mwchan' field that you see as 'STATE' in top or via 'ps O mwchan' is more detailed than the 'D' state.) To diagnose this further, you would need to see which thread holds the ZFS vnode lock these threads need. I have some gdb scripts you can use to do that at www.freebsd.org/~jhb/gdb/. You would want to download 'gdb6*' files from there and then do this as root: # cd /path/to/gdb/files # kgdb (kgdb) source gdb6 (kgdb) sleepchain 42335 Where '42335' is the pid of some process stuck in "zfs". -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Wed Aug 27 21:24:54 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE6E32E6; Wed, 27 Aug 2014 21:24:54 +0000 (UTC) Received: from caida.org (rommie.caida.org [192.172.226.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D77E83E94; Wed, 27 Aug 2014 21:24:54 +0000 (UTC) Message-ID: <53FE4C9F.7030406@caida.org> Date: Wed, 27 Aug 2014 14:24:47 -0700 From: Daniel Andersen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: John Baldwin , freebsd-fs@freebsd.org Subject: Re: Process enters unkillable state and somewhat wedges zfs References: <53F25402.1020907@caida.org> <53F4E3C0.6000406@FreeBSD.org> <53FB716D.1060007@caida.org> <201408271639.09352.jhb@freebsd.org> In-Reply-To: <201408271639.09352.jhb@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Aug 2014 21:24:55 -0000 On 08/27/2014 01:39 PM, John Baldwin wrote: > On Monday, August 25, 2014 1:25:01 pm Daniel Andersen wrote: >> On 08/20/2014 11:06 AM, Bryan Drewery wrote: >>> On 8/18/2014 2:29 PM, Daniel Andersen wrote: >>>> We are currently experiencing a strange problem that sort of locks up one > of our zfs pools. This is on a FreeBSD 10 >>>> machine. Let me give a rough layout of our system to better describe > what is happening: >>> >>> When it happens get the output of 'procstat -kka|grep zfs' please. >>> >> >> I'd read something about that while attempting to debug the last few times, > so I happen to have that output, I believe. >> Here are the last two examples ( the latter having had more wedged processes > ): >> >> 0 100687 kernel zfs_vn_rele_task mi_switch+0xde > sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 >> fork_exit+0x9a fork_trampoline+0xe >> 0 101055 kernel zfs_vn_rele_task mi_switch+0xde > sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 >> fork_exit+0x9a fork_trampoline+0xe >> 38 100306 zfskern arc_reclaim_thre mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe >> 38 100307 zfskern l2arc_feed_threa mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe >> 38 100672 zfskern trim tank mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe >> 38 100688 zfskern txg_thread_enter mi_switch+0xde > sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b >> fork_exit+0x9a fork_trampoline+0xe >> 38 100689 zfskern txg_thread_enter mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe >> 38 101054 zfskern trim work mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe >> 38 101056 zfskern txg_thread_enter mi_switch+0xde > sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b >> fork_exit+0x9a fork_trampoline+0xe >> 38 101057 zfskern txg_thread_enter mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe >> 16656 101610 ls - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statat_vnhook+0xa5 >> sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb >> >> >> and >> >> 0 100688 kernel zfs_vn_rele_task mi_switch+0xde > sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 >> fork_exit+0x9a fork_trampoline+0xe >> 0 101056 kernel zfs_vn_rele_task mi_switch+0xde > sleepq_wait+0x3a _sleep+0x26f taskqueue_thread_loop+0xd5 >> fork_exit+0x9a fork_trampoline+0xe >> 38 100306 zfskern arc_reclaim_thre mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> arc_reclaim_thread+0x302 fork_exit+0x9a fork_trampoline+0xe >> 38 100307 zfskern l2arc_feed_threa mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> l2arc_feed_thread+0xb29 fork_exit+0x9a fork_trampoline+0xe >> 38 100673 zfskern trim tank mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe >> 38 100689 zfskern txg_thread_enter mi_switch+0xde > sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b >> fork_exit+0x9a fork_trampoline+0xe >> 38 100690 zfskern txg_thread_enter mi_switch+0xde > sleepq_wait+0x3a _cv_wait+0x16d zio_wait+0x5b >> vdev_uberblock_sync_list+0xad vdev_config_sync+0x118 spa_sync+0x827 > txg_sync_thread+0x375 fork_exit+0x9a >> fork_trampoline+0xe >> 38 101055 zfskern trim work mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> trim_thread+0x9b fork_exit+0x9a fork_trampoline+0xe >> 38 101057 zfskern txg_thread_enter mi_switch+0xde > sleepq_wait+0x3a _cv_wait+0x16d txg_quiesce_thread+0x30b >> fork_exit+0x9a fork_trampoline+0xe >> 38 101058 zfskern txg_thread_enter mi_switch+0xde > sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18d >> txg_sync_thread+0x1dd fork_exit+0x9a fork_trampoline+0xe >> 2638 101160 nfsd nfsd: service mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_vget+0xfe > nullfs_vget+0x54 nfsrvd_readdirplus+0x905 >> nfsrvd_dorpc+0x773 nfssvc_program+0x4f6 svc_run_internal+0x1f9 > svc_thread_start+0xb fork_exit+0x9a fork_trampoline+0xe >> 3891 101641 collectd - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statfs+0x6f >> sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb >> 11808 101802 sendsize - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statat_vnhook+0xa5 >> sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb >> 13900 101899 sendsize - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statat_vnhook+0xa5 >> sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb >> 17961 102536 df - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statfs+0x6f >> sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb >> 38964 102279 ls - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statat_vnhook+0xa5 >> sys_lstat+0x30 amd64_syscall+0x357 Xfast_syscall+0xfb >> 42315 102539 df - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statfs+0x6f >> sys_statfs+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb >> 42335 101782 sendsize - mi_switch+0xde > sleepq_wait+0x3a sleeplk+0x11c __lockmgr_args+0x950 >> vop_stdlock+0x3c VOP_LOCK1_APV+0x9d _vn_lock+0x43 zfs_root+0x8f lookup+0x7f0 > namei+0x504 kern_statat_vnhook+0xa5 >> sys_stat+0x2d amd64_syscall+0x357 Xfast_syscall+0xfb > > These are all blocked in "zfs" then. (For future reference, the 'mwchan' > field that you see as 'STATE' in top or via 'ps O mwchan' is more detailed > than the 'D' state.) > > To diagnose this further, you would need to see which thread holds the > ZFS vnode lock these threads need. I have some gdb scripts you can use to > do that at www.freebsd.org/~jhb/gdb/. You would want to download 'gdb6*' > files from there and then do this as root: > > # cd /path/to/gdb/files > # kgdb > (kgdb) source gdb6 > (kgdb) sleepchain 42335 > > Where '42335' is the pid of some process stuck in "zfs". > I will keep this in mind the next time the machine wedges. Another data point: the second procstat output I sent was the most recent. All the processes listed there were after the fact. The process that started the entire problem ( this time ) was sudo, and it only has this one entry in procstat: 38003 102797 sudo - Of note, this does not appear to be blocked on zfs in anyway. 'ps' showed it in 'R' state instead of 'D' ( I will be sure to use mwchan in the future. ) It appeared to be pegging an entire CPU core at 100% usage, as well, and was only single threaded. Processes that try to access /tank after that wedge as shown above, while processes that access the same filesystem through the /data/ nullfs mount do not. nfsd appears to be an exception, as we only export nfs mounts via a /tank/foo -> /export/foo nullfs mount. It's possible nfsd is doing something else there, though, I guess. :/ Many thanks, Dan From owner-freebsd-fs@FreeBSD.ORG Thu Aug 28 03:20:54 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9A2DB0B for ; Thu, 28 Aug 2014 03:20:54 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B102912D5 for ; Thu, 28 Aug 2014 03:20:54 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7S3Ks7U028654 for ; Thu, 28 Aug 2014 03:20:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 144234] [zfs] Cannot boot machine with recent gptzfsboot code [regression] Date: Thu, 28 Aug 2014 03:20:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: elliot.robinson@argiopetech.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 03:20:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144234 --- Comment #7 from Elliot Robinson --- Applied extra_debug_info.patch manually. Had to remove the conditional to get things to print, though it very well could have been a typo on my part. gptzfsboot: error 1 lba 48 drv c338c483 count 16 to 2000:0 gptzfsboot: error 1 lba 1 drv c338c483 count 1 to 2200:0 gptzfsboot: No ZFS pools located, can't boot. I couldn't get bounce_buffer_1.patch to apply against 10.0-RELEASE's source bundle. If you'll let me know which commit you're working from, I'll be happy to pull and test from there. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Thu Aug 28 18:08:02 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73EB77B2 for ; Thu, 28 Aug 2014 18:08:02 +0000 (UTC) Received: from smtp5.cc.ksu.edu (smtp-feca09d8910ed08e5e479c79aac3e744.cc.ksu.edu [129.130.255.125]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 512F41AA4 for ; Thu, 28 Aug 2014 18:08:01 +0000 (UTC) Received: from mew.cns.ksu.edu (mew.cns.ksu.edu [129.130.0.181]) by smtp5.cc.ksu.edu (8.14.3/8.14.3) with ESMTP id s7SI7vuS014337 for ; Thu, 28 Aug 2014 13:07:59 -0500 (CDT) Message-ID: <53FF6FFC.2090407@ksu.edu> Date: Thu, 28 Aug 2014 13:07:56 -0500 From: "Lawrence K. Chen, P.Eng." Organization: Kansas State University - ITS/Enterprise Server Technologies User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org Subject: Resilver ZIL & Sequential Resilvering? Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.4 at cts-virus3 X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.72 on 129.130.255.119 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 18:08:02 -0000 As I'm suffering through the process of watching ZFS resilver....I wondered if there are any updates to ZFS coming down that help in these areas? The first is Resilvering a ZIL scan: resilver in progress since Thu Aug 28 08:35:38 2014 996G scanned out of 1.15T at 75.7M/s, 0h41m to go 0 resilvered, 84.33% done ----------- scan: resilver in progress since Thu Aug 28 08:37:12 2014 3.80T scanned out of 4.92T at 298M/s, 1h5m to go 0 resilvered, 77.24% done ----------- scan: resilver in progress since Thu Aug 28 08:37:49 2014 7.83T scanned out of 9.41T at 616M/s, 0h44m to go 0 resilvered, 83.23% done Replaced one of the SSDs for my mirrored ZIL this morning. Why does it have to scan all the storage in the main pool to resilver the ZIL? Why does it have to resilver the ZIL at all? Granted it goes a whole lot faster than if I were replacing a disk in one of the pools (which I recently did, twice, for the top pool.... when it started it had estimates of like 400+ hours, though reality was ~60 hours. Which brings me to the second item.... sequential resilvering? We were way behind on updates for our 7420, and I happened to spot this as a new feature. The gist is that all the random i/o, especially at the beginning makes resilvering slow...so this enhancement splits the resilvering into two steps, in the first step it scans all the blocks that need to be copied and sorts them into LBA order.... In the meantime...more slow resilvering is on my horizon. Since replaced one SSD with a larger SSD, I'm going to want replace the other. Plus I'm thinking of migrating to a new root pool, to see if it'll rid me of the "ZFS: i/o error - all block copies unavailable" messages during boot....and make some layout changes. Plus need to see about getting the first pool expand to its new size....someday I'll need to figure something out for the other two pools (there's one drive about to go in the second pool...) All the harddrives were 512 byte sectors, but only the first pool (a pair of ST31500341AS drives) was made with ashift=12. So, when one drive reported imminent failure with the sudden relocation of 3000+ sectors....when grew to 4000+ the next day, and got replaced the next...on a Sunday. But, it wasn't that big a problem as I had a pair of 3TB WD Reds that were supposed to go somewhere else. Second pool is raidz of similar 1.5TB drives, third pool is raidz of Hitachi 2TB drives....so far its only been the 1.5TB drives that keep leaving me. I suspect at the time, I expected the drives in the first pool to fail and that I would be upgrading to bigger 4K drives, while the other two pools....well...the pool of 1.5TB was temporary, only its not anymore....plus I still have a few extras from other failed arrays. And, the 2TB pool was probably because I knew it was going to have lots of tiny files, etc. The overhead of 4K versus 512 is pretty huge.... Originally created my /poudriere space on the second pool....a ports directory was about 450MB. Moved things over to the first pool....the same directories are now about 840MB. I guess things could've been worse.... Had noticed similar things when I changed the page size of an sqlite file from 1k to 4k.... -- Who: Lawrence K. Chen, P.Eng. - W0LKC - Sr. Unix Systems Administrator For: Enterprise Server Technologies (EST) -- & SafeZone Ally From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 13:05:06 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FE14AFF for ; Fri, 29 Aug 2014 13:05:06 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39ECA1A60 for ; Fri, 29 Aug 2014 13:05:05 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1XNLrN-0008Ot-4o for freebsd-fs@freebsd.org; Fri, 29 Aug 2014 15:04:57 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Aug 2014 15:04:57 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Aug 2014 15:04:57 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Subject: lockf(1) and NFS Date: Fri, 29 Aug 2014 15:04:22 +0200 Lines: 47 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7dC5RRDARlPrAJvalGUKMh7CMnWCFA4Fn" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 13:05:06 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7dC5RRDARlPrAJvalGUKMh7CMnWCFA4Fn Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, I had some fun troubleshooting NFS locking and among other things, found that lockf(1) doesn't really work on NFSv4 mounts. Googling around (so correct me if I'm wrong), it looks like this is because NFS quietly translates the old-style locks into POSIX range locks, and those cannot be acquired exclusively if the file is opened read-only. I've tested the following patch and it works. Any objections to committing it? --- a/lockf.c Fri Aug 29 14:58:10 2014 +0200 +++ b/lockf.c Fri Aug 29 14:59:12 2014 +0200 @@ -169,7 +169,7 @@ { int fd; - if ((fd =3D open(name, flags|O_RDONLY|O_EXLOCK|flags, 0666)) =3D=3D -1)= { + if ((fd =3D open(name, flags|O_RDWR|O_EXLOCK|flags, 0666)) =3D=3D -1) {= if (errno =3D=3D EAGAIN || errno =3D=3D EINTR) return (-1); err(EX_CANTCREAT, "cannot open %s", name); --7dC5RRDARlPrAJvalGUKMh7CMnWCFA4Fn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iKYEARECAGYFAlQAemRfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDYxNDE4MkQ3ODMwNDAwMDJFRUIzNDhFNUZE MDhENTA2M0RGRjFEMkMACgkQ/QjVBj3/HSwlVgCeOAIZdpnVbarFltjwPm9SPeH7 VakAnAmzt1PF8iLCMfoTRgoOx9OVzn3V =Ii34 -----END PGP SIGNATURE----- --7dC5RRDARlPrAJvalGUKMh7CMnWCFA4Fn-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 19:45:36 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83E86A11; Fri, 29 Aug 2014 19:45:36 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A1B31CF7; Fri, 29 Aug 2014 19:45:36 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 17C6EB960; Fri, 29 Aug 2014 15:45:35 -0400 (EDT) From: John Baldwin To: Daniel Andersen Subject: Re: Process enters unkillable state and somewhat wedges zfs Date: Fri, 29 Aug 2014 14:24:07 -0400 Message-ID: <5842681.mjgMD2kESs@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <53FE4C9F.7030406@caida.org> References: <53F25402.1020907@caida.org> <201408271639.09352.jhb@freebsd.org> <53FE4C9F.7030406@caida.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 29 Aug 2014 15:45:35 -0400 (EDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 19:45:36 -0000 On Wednesday, August 27, 2014 02:24:47 PM Daniel Andersen wrote: > On 08/27/2014 01:39 PM, John Baldwin wrote: > > These are all blocked in "zfs" then. (For future reference, the 'mwchan' > > field that you see as 'STATE' in top or via 'ps O mwchan' is more detailed > > than the 'D' state.) > > > > To diagnose this further, you would need to see which thread holds the > > ZFS vnode lock these threads need. I have some gdb scripts you can use to > > do that at www.freebsd.org/~jhb/gdb/. You would want to download 'gdb6*' > > files from there and then do this as root: > > > > # cd /path/to/gdb/files > > # kgdb > > (kgdb) source gdb6 > > (kgdb) sleepchain 42335 > > > > Where '42335' is the pid of some process stuck in "zfs". > > I will keep this in mind the next time the machine wedges. Another data > point: the second procstat output I sent was the most recent. All the > processes listed there were after the fact. The process that started the > entire problem ( this time ) was sudo, and it only has this one entry in > procstat: > > 38003 102797 sudo - > > Of note, this does not appear to be blocked on zfs in anyway. 'ps' showed > it in 'R' state instead of 'D' ( I will be sure to use mwchan in the > future. ) It appeared to be pegging an entire CPU core at 100% usage, as > well, and was only single threaded. Well, if it is spinning in some sort of loop in the kernel while holding a ZFS vnode lock that could be blocking all the other threads. In that case, you don't need to do what I asked for above. Instead, we need to find out what that thread is doing. There are two ways of doing this. One is to force a panic via 'sysctl debug.kdb.panic=1' and then use kgdb on the crashdump to determine what the running thread is doing. Another option is to break into the DDB debugger on the console (note that you will need to build a custom kernel with DDB if you are on stable) and request a stack trace of the running process via 'tr '. Ideally you can do this over a serial console so you can just cut and paste the output of the trace into a mail. Over a video console you can either transcribe it by hand or take photos. -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 20:59:29 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D525F7; Fri, 29 Aug 2014 20:59:29 +0000 (UTC) Received: from sasl.smtp.pobox.com (sasl.smtp.pobox.com [208.72.237.25]) by mx1.freebsd.org (Postfix) with ESMTP id 5295D17C3; Fri, 29 Aug 2014 20:59:28 +0000 (UTC) Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl0.pobox.com (Postfix) with ESMTP id E6E5635A7F; Fri, 29 Aug 2014 16:59:21 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date :message-id:from:to:cc:subject:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=sasl; bh=MjXcB/CmCZPrw7WCzhFUyvRtngM=; b=ljqvb+tJShEAn0oTItmzrd7k3Eml pWcI5vVJ1BrUCNJY1qVKbYDljz7C35IUAihHY1Es5o2KoCjsPxANKYTHWoaoY4fK KRkVXtXEft5zKp5IqAwX4Sd1CrNVJ617mziXQKjfbm0epS3ovaU1m7nDtErX7o2z UaWiEa3ONMows34= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:message-id :from:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=sasl; b=EUiyOR usLjt2HfqgupiWdun/AHlP6I6OQI+RX5YIT6OiVD4gK46er21bQy/GMPBQkj6b2r ycQcvHV39CNmHNn5O4KP3QXPhFODLKv7gXAiHLlwK1ugKLShJQhQVS7/72lpvX94 WTWfhLWFpxAQH+AvosJAbpwCOpgJyGcFQaDOs= Received: from pb-sasl0.int.icgroup.com (unknown [127.0.0.1]) by pb-sasl0.pobox.com (Postfix) with ESMTP id DEC1135A7E; Fri, 29 Aug 2014 16:59:21 -0400 (EDT) Received: from bmach.nederware.nl (unknown [27.252.230.206]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl0.pobox.com (Postfix) with ESMTPSA id AFEB235A7A; Fri, 29 Aug 2014 16:59:13 -0400 (EDT) Received: from quadrio.nederware.nl (quadrio.nederware.nl [192.168.33.13]) by bmach.nederware.nl (Postfix) with ESMTP id 548D941C04; Sat, 30 Aug 2014 08:59:11 +1200 (NZST) Received: from quadrio.nederware.nl (localhost [127.0.0.1]) by quadrio.nederware.nl (Postfix) with ESMTP id 46752835B4C3; Sat, 30 Aug 2014 08:59:11 +1200 (NZST) Date: Sat, 30 Aug 2014 08:59:11 +1200 Message-ID: <87mwant3kw.wl%berend@pobox.com> From: Berend de Boer To: Ivan Voras Subject: Re: lockf(1) and NFS In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/24.3 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: Xplain Technology Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Sat_Aug_30_08:59:10_2014-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit X-Pobox-Relay-ID: 54E17552-2FBF-11E4-B639-7FB96395E023-48001098!pb-sasl0.pobox.com Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 20:59:29 -0000 --pgp-sign-Multipart_Sat_Aug_30_08:59:10_2014-1 Content-Type: text/plain; charset=US-ASCII >>>>> "Ivan" == Ivan Voras writes: Ivan> I've tested the following patch and it works. Any Ivan> objections to committing it? That may fix some of the problems I have with NFSv4 (so still using NFSv3)! -- All the best, Berend de Boer --pgp-sign-Multipart_Sat_Aug_30_08:59:10_2014-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit Content-Description: OpenPGP Digital Signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABCAAGBQJUAOmfAAoJEKOfeD48G3g5wRIP/21f4EGt9ClSSPd5M5uM+d27 nGwpCy0tmenOqX1GRKDxO0YIsyNcWE8cH71doDpfRcYkX6hdvaln890JrphyMFJb rFglt3ud8NdJuFwy9C0IEx/lGdmIQs73xe28QbbG1zg6rTLzlHlNIt4Q3jsV2Xpk X5AhOxaCwC43c0WIt9Ob4Pc2h8TTe30wvx7EJVpRy+/vsQb2OgezN2CBrbFRo3gu 8N7LFuGvR74VgplkF0TLQR8XuCyGoJw4YLQX2rIC3wWGd7V92YHmzNcY2+uw8TLj ubUEgyk9jW9jY9jHGkpqQp5b1w/wxmP5hATXSsT+YKrc8H5xomy2NmmzNMNARSaR yHgkrxspDs2WI3I+aUpN4SMVEcp0iGD9CUpx2zchP/NtCsrmEKOpwgLEQCNTNxvV VaGWwTTtC97lEYY8iKdcig8eVle7Zql9ETtvv6VVav9pFzulHQJBEeY3J8c+4Zsv kVgEvBpO+TgrLjYImhwqmB1Gh4/RGVwUjBWClRKeYhZffoa36U+w8MbNr6BWxOJh 3QZkTtdHne4btmUoQ1QOsYC1iSslV2krpzdag06VUMFS04tA5AWYEVWJPZ6oi/8j FOd/+MFlKYLjp+pTdvRv6Vd1ZQyXGK7suC46ikeE6S/9TMAT1acOM5TrshO8zXqL Uh/UC+MuN6gZw3N78dSG =iglq -----END PGP SIGNATURE----- --pgp-sign-Multipart_Sat_Aug_30_08:59:10_2014-1-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 21:05:41 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5BBE2BE for ; Fri, 29 Aug 2014 21:05:41 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5FE8A1897 for ; Fri, 29 Aug 2014 21:05:41 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id b17so3297278lan.8 for ; Fri, 29 Aug 2014 14:05:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=MJ2WGWIKAcZsvD33u8mdbP6ylqdAWGA/BDMPRgBxmXE=; b=vkXnRi2CgIzo2Tw/C6yjhMtOOoGdbMCOkDQ7seDDBIqtwRqnv9MgAnfTj9U8yT5pXc MVvgrU2gor33VRvsCYFs68RcVKqj4nRBrid+RRonFh01pErtuECPbO5tEDA0aF5H0Nkc q5J36TjxsEICwxhmprBsROqEKGF72u7Co26Ct0001JB0OgT8o5OQp+TZY7zP0WxNCTHN 7ZL14kOD+CZrQlcRtJdnzieFIt4FjJWTXaCHd9CFdPxjrtVvcCOeCcHrVYtEeIGWuwtk PjNgg+C5U9q2cnIgHlo4b8CnNc1B8wlnODslRPJIl9YWW60iQu16A07BzAJuqCZCG8Wa RkWA== X-Received: by 10.152.5.66 with SMTP id q2mr13596758laq.11.1409346339223; Fri, 29 Aug 2014 14:05:39 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.25.149.205 with HTTP; Fri, 29 Aug 2014 14:04:59 -0700 (PDT) In-Reply-To: <87mwant3kw.wl%berend@pobox.com> References: <87mwant3kw.wl%berend@pobox.com> From: Ivan Voras Date: Fri, 29 Aug 2014 23:04:59 +0200 X-Google-Sender-Auth: hwNh3VoPvf_2CEaRDTW7gCymISQ Message-ID: Subject: Re: lockf(1) and NFS To: Berend de Boer Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 21:05:41 -0000 On 29 August 2014 22:59, Berend de Boer wrote: >>>>>> "Ivan" == Ivan Voras writes: > > Ivan> I've tested the following patch and it works. Any > Ivan> objections to committing it? > > That may fix some of the problems I have with NFSv4 (so still using NFSv3)! Could be... but are you sure? This is a patch to the lockf utility, not to libc or the kernel...? I do think that if lockf has this problem there has to be other code which also has it, but I've found that Linux's NFSv4 implementation does exactly the same thing so most open-source programs should already be fixed because Linux requires it also. From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 22:09:11 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDE605D6; Fri, 29 Aug 2014 22:09:10 +0000 (UTC) Received: from sasl.smtp.pobox.com (sasl.smtp.pobox.com [208.72.237.25]) by mx1.freebsd.org (Postfix) with ESMTP id A98E71F2C; Fri, 29 Aug 2014 22:09:10 +0000 (UTC) Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl0.pobox.com (Postfix) with ESMTP id 8317735E8A; Fri, 29 Aug 2014 18:09:08 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date :message-id:from:to:cc:subject:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=sasl; bh=7jje0k1m5pj03CZa1cu+eexhO2U=; b=uVkLvOktj2DhKNpk5RTi/sTFZiLv Rju3UW3dc9beU7blrWoqw9qlF0+q3+RhmKPBwooHZUfAkyoqeenov8XUr+FXLoZE SEhoHi8hfN0liwXEJZ+WAB0Hg5Dw5ISg6F6aP4sT52BTzo3s5qemnxEU31Xyy8r0 TUn0e1WiH1n4LzQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:message-id :from:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=sasl; b=QmyloP 3a4490NBeMJoV83LoxjbFvPaaErCyM7eJfi7UPG/kZfzIuIQi+rEhoDYerCDQLFS xUvvsXJGtaM6JTA/Xf/EXmIf1SdRWhyacAwiNdg0R1AMU6HzRO/iRpQ171dW1new ZiXzA24Vr4kCRBqVABYLa2ZEBQuTGJjh+oT7A= Received: from pb-sasl0.int.icgroup.com (unknown [127.0.0.1]) by pb-sasl0.pobox.com (Postfix) with ESMTP id E414535E89; Fri, 29 Aug 2014 18:09:07 -0400 (EDT) Received: from bmach.nederware.nl (unknown [27.252.230.206]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl0.pobox.com (Postfix) with ESMTPSA id 2529835E84; Fri, 29 Aug 2014 18:08:58 -0400 (EDT) Received: from quadrio.nederware.nl (quadrio.nederware.nl [192.168.33.13]) by bmach.nederware.nl (Postfix) with ESMTP id C0A5641C5F; Sat, 30 Aug 2014 10:08:55 +1200 (NZST) Received: from quadrio.nederware.nl (localhost [127.0.0.1]) by quadrio.nederware.nl (Postfix) with ESMTP id 9659C835B4C3; Sat, 30 Aug 2014 10:08:55 +1200 (NZST) Date: Sat, 30 Aug 2014 10:08:55 +1200 Message-ID: <87k35rt0co.wl%berend@pobox.com> From: Berend de Boer To: Ivan Voras Subject: Re: lockf(1) and NFS In-Reply-To: References: <87mwant3kw.wl%berend@pobox.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/24.3 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: Xplain Technology Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Sat_Aug_30_10:08:55_2014-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit X-Pobox-Relay-ID: 12FF69D2-2FC9-11E4-B282-7FB96395E023-48001098!pb-sasl0.pobox.com Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 22:09:11 -0000 --pgp-sign-Multipart_Sat_Aug_30_10:08:55_2014-1 Content-Type: text/plain; charset=US-ASCII >>>>> "Ivan" == Ivan Voras writes: Ivan> Could be... but are you sure? This is a patch to the lockf Ivan> utility, not to libc or the kernel...? My mistake, thought this was lockf(3)! -- All the best, Berend de Boer --pgp-sign-Multipart_Sat_Aug_30_10:08:55_2014-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit Content-Description: OpenPGP Digital Signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABCAAGBQJUAPn3AAoJEKOfeD48G3g5AikQAMQUOGrSN1YOjPSRgjqJ+Y6X fokKblg/q0QJPpkXCyZMoiDlKvx4Dn73tvstF9ex8uVWJVBgHEKQe+QKrmJTbJkw QY8cu587EUqsDyJQd+8oTj3/HKX6BXuVL1sQLNHDpR8bEPsElaOJlZB/LJ5q/hAq vKNJRpypBKURTsSBpGAUL4UlQRP8wd2E8gmYU4If00d24D1uSWNODzP0QvHzzGu2 L5lTqffNFC+AnsEV7EQaAsGLry+btYm4RljqSKyVpEcTKw+cr415QYWq6Os9VSNK qbvDoygcubQ+dtKL9P9LF/5fcYyf98G5M42OA/V/PvpZz3ycnPIYCtr2OqrRsMUq cB0hCd//cT+ca9zJLAGFobVWIyQld1o8IuVBk7FchL0KV/mRaDkrml/WqtyAX8qS iLTskHNsWTuxtEULcX2Ip8P7krryQQB+qYDjX8quNc38n6QIELvZD2NocNVIMU38 JX2AvFwPDlGKG3a4RFU4yh9Xi4p6CwNfnedB2KwT87yt58GvO43Q4pc3U+geSzCp 5zZSfoPnrW1rvRhebt6TtDTvfNZW3Ofxd1Kl7tuPpZehexG17pAiC6IKZbE5BNK1 UqCTcv9IkM4HyvSZ1KTcwMViH8PCo7A1t+k34IzsCoTKLw6RGrwi7KxLdV+Oxzz4 u23PgMeX+MUasMeHooqI =ZgvJ -----END PGP SIGNATURE----- --pgp-sign-Multipart_Sat_Aug_30_10:08:55_2014-1-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 29 23:54:59 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5AF88FEA; Fri, 29 Aug 2014 23:54:59 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 0CA601C37; Fri, 29 Aug 2014 23:54:58 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqMEAKISAVSDaFve/2dsb2JhbABbhDuCeMklgx8BgSl3hAMBAQQBI1YFFg4KAgINGQJLAQ0GiE0Ip2OVHwEXgSyNbDQHgnmBUwWodYkFg3whgXeBBwEBAQ X-IronPort-AV: E=Sophos;i="5.04,427,1406606400"; d="scan'208";a="150641379" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 29 Aug 2014 19:54:37 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 735A5B3F41; Fri, 29 Aug 2014 19:54:37 -0400 (EDT) Date: Fri, 29 Aug 2014 19:54:37 -0400 (EDT) From: Rick Macklem To: Ivan Voras Message-ID: <1912666252.30566061.1409356477461.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: lockf(1) and NFS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Aug 2014 23:54:59 -0000 Ivan Voras wrote: > Hi, > > I had some fun troubleshooting NFS locking and among other things, > found > that lockf(1) doesn't really work on NFSv4 mounts. Googling around > (so > correct me if I'm wrong), it looks like this is because NFS quietly > translates the old-style locks into POSIX range locks, and those > cannot > be acquired exclusively if the file is opened read-only. > Yes, the NFSv4 protocol only supports POSIX byte range locks. The only alternative to translating flock(2) locks to POSIX locks is to not support flock(2) locks at all. To be honest, NFSv4 does have Windows style Open locks, which would implement something close to O_EXLOCK, but not flock(2). Since they couldn't be flock(2) compatible, I didn't do this. > I've tested the following patch and it works. > Any objections to committing it? > What happens if the file "name" exists and the process only has read access to it for a file on a local fs? (I'm wondering if the patch breaks that case and causes a POLA?) Alternately you could add a new command option for this case and document the need to use it for NFSv4 mounts, maybe? Have fun with it, rick > --- a/lockf.c Fri Aug 29 14:58:10 2014 +0200 > +++ b/lockf.c Fri Aug 29 14:59:12 2014 +0200 > @@ -169,7 +169,7 @@ > { > int fd; > > - if ((fd = open(name, flags|O_RDONLY|O_EXLOCK|flags, 0666)) == -1) { > + if ((fd = open(name, flags|O_RDWR|O_EXLOCK|flags, 0666)) == -1) { > if (errno == EAGAIN || errno == EINTR) > return (-1); > err(EX_CANTCREAT, "cannot open %s", name); > > From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 00:53:04 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4482A31; Sat, 30 Aug 2014 00:53:04 +0000 (UTC) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C6E3011F1; Sat, 30 Aug 2014 00:53:04 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s7U0peLr073400; Fri, 29 Aug 2014 17:51:44 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201408300051.s7U0peLr073400@gw.catspoiler.org> Date: Fri, 29 Aug 2014 17:51:40 -0700 (PDT) From: Don Lewis Subject: Re: lockf(1) and NFS To: rmacklem@uoguelph.ca In-Reply-To: <1912666252.30566061.1409356477461.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-fs@FreeBSD.org, ivoras@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 00:53:05 -0000 On 29 Aug, Rick Macklem wrote: > Ivan Voras wrote: >> Hi, >> >> I had some fun troubleshooting NFS locking and among other things, >> found >> that lockf(1) doesn't really work on NFSv4 mounts. Googling around >> (so >> correct me if I'm wrong), it looks like this is because NFS quietly >> translates the old-style locks into POSIX range locks, and those >> cannot >> be acquired exclusively if the file is opened read-only. >> > Yes, the NFSv4 protocol only supports POSIX byte range locks. > The only alternative to translating flock(2) locks to POSIX locks is > to not support flock(2) locks at all. As I recall, on SunOS, flock(2) locks were local to the machine. Looks like that was changed later on ... "Locks obtained through the flock() mechanism under SunOS 4.1 were known only within the system on which they were placed. This is no longer true." That was actually a feature because you could use flock() to coordinate between local processes, while avoiding any NFS lock manager bugs, even on diskless machines. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 11:47:01 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17E8ACB2; Sat, 30 Aug 2014 11:47:01 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id B259F1198; Sat, 30 Aug 2014 11:47:00 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqgEAFO5AVSDaFve/2dsb2JhbABbg2BXBIJ4xHuHTAGBJ3eEBAEFI1YbDgoCAg0ZAlkGiFUNpn6UfgEXgSyNbTQHgnmBUwWodYkFg30hLwEBgUaBBwEBAQ X-IronPort-AV: E=Sophos;i="5.04,431,1406606400"; d="scan'208";a="150686617" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 30 Aug 2014 07:46:59 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2CEFDB3EA2; Sat, 30 Aug 2014 07:46:59 -0400 (EDT) Date: Sat, 30 Aug 2014 07:46:59 -0400 (EDT) From: Rick Macklem To: Don Lewis Message-ID: <1588968883.30624902.1409399219119.JavaMail.root@uoguelph.ca> In-Reply-To: <201408300051.s7U0peLr073400@gw.catspoiler.org> Subject: Re: lockf(1) and NFS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@FreeBSD.org, ivoras@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 11:47:01 -0000 Don Lewis wrote: > On 29 Aug, Rick Macklem wrote: > > Ivan Voras wrote: > >> Hi, > >> > >> I had some fun troubleshooting NFS locking and among other things, > >> found > >> that lockf(1) doesn't really work on NFSv4 mounts. Googling around > >> (so > >> correct me if I'm wrong), it looks like this is because NFS > >> quietly > >> translates the old-style locks into POSIX range locks, and those > >> cannot > >> be acquired exclusively if the file is opened read-only. > >> > > Yes, the NFSv4 protocol only supports POSIX byte range locks. > > The only alternative to translating flock(2) locks to POSIX locks > > is > > to not support flock(2) locks at all. > > As I recall, on SunOS, flock(2) locks were local to the machine. > Looks > like that was changed later on ... > > "Locks obtained through the flock() mechanism under SunOS 4.1 > were known only within the system on which they were placed. > This is no longer true." > > > > That was actually a feature because you could use flock() to > coordinate > between local processes, while avoiding any NFS lock manager bugs, > even > on diskless machines. > > > Doing locking locally within the client is still possible with the "nolockd" option. However, when this mount option is used all file locking, including POSIX byte range locks, are done locally. Linux has a similar mount option (called "nolock" I think?), but I don't know if Solaris has any mount option. I'm pretty sure Ivan knows about this, so I suspect he needs lockf(1) to work across multiple clients for his app., but don't know for sure? rick From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 17:30:15 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C78D19A; Sat, 30 Aug 2014 17:30:15 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D2B911235; Sat, 30 Aug 2014 17:30:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=fzwQKIgrH/eSRvCqRw2o75vdLLqIKQDDOaHNpKeX+x4=; b=uh2DmiAmC5ex91qnIkvm/kJYhW4eolicJxwm9bnjY19B8EoZVa/LsC497HlDPqhrk1LMtVgX5/PKb0NLQ18kzmQP7Hd9gTNiNmoOOdHrwVlVKnbxh45XruJZC4xeh/YP2kA2Rh7EzgZ1k5CC2G0x5vO5faFAtBpnYN3u1Vl5rMI=; Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]:39214 helo=borg.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XNmTZ-000MZz-RU; Sat, 30 Aug 2014 12:30:13 -0500 Date: Sat, 30 Aug 2014 12:29:58 -0500 From: Larry Rosenman To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Solaris Assert: How to fix? Message-ID: <20140830172958.GA99694@borg.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 17:30:15 -0000 I'm getting: borg.lerctr.org dumped core - see /var/crash/vmcore.0 Fri Aug 29 16:16:35 CDT 2014 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r270811: Fri Aug 29 09:21:58 CDT 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #0 r270811: Fri Aug 29 09:21:58 CDT 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 VT: running with driver "vga". info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 VT-x: HLT,PAUSE TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65627250688 (62587 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 random: initialized module_register_init: MOD_LOAD (vesa, 0xffffffff80e94c10, 0) error 19 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33001600 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000010CF6000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff800118de000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0x0 iic1: on iicbus1 iicbus2: on iicbb2 addr 0x0 iic2: on iicbus2 iicbus3: on iicbb3 addr 0x0 iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 VT: Replacing driver "vga" with new "fb". info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 coretemp1: on cpu1 est1: on cpu1 coretemp2: on cpu2 est2: on cpu2 coretemp3: on cpu3 est3: on cpu3 coretemp4: on cpu4 est4: on cpu4 coretemp5: on cpu5 est5: on cpu5 coretemp6: on cpu6 est6: on cpu6 coretemp7: on cpu7 est7: on cpu7 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x444 offMax=0x690 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 ugen0.1: at usbus0 uhub0: on usbus0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub1: on usbus3 ugen2.1: at usbus2 uhub2: on usbus2 ugen1.1: at usbus1 uhub3: on usbus1 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present SMP: AP CPU #4 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #7 Launched! Root mount waiting for: usbus3 uhub1: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 <118>Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. <118>Setting hostid: 0xf53a926e. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 cpuid = 5 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100be1d040 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100be1d0f0 vpanic() at vpanic+0x189/frame 0xfffffe100be1d170 panic() at panic+0x43/frame 0xfffffe100be1d1d0 assfail3() at assfail3+0x2f/frame 0xfffffe100be1d1f0 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x4e0/frame 0xfffffe100be1d240 zap_deref_leaf() at zap_deref_leaf+0xd3/frame 0xfffffe100be1d290 fzap_cursor_retrieve() at fzap_cursor_retrieve+0xf1/frame 0xfffffe100be1d300 zap_cursor_retrieve() at zap_cursor_retrieve+0x20d/frame 0xfffffe100be1d390 ddt_zap_walk() at ddt_zap_walk+0x53/frame 0xfffffe100be1d640 ddt_walk() at ddt_walk+0x79/frame 0xfffffe100be1d670 dsl_scan_sync() at dsl_scan_sync+0x8e7/frame 0xfffffe100be1d9d0 spa_sync() at spa_sync+0x5c1/frame 0xfffffe100be1dac0 txg_sync_thread() at txg_sync_thread+0x24d/frame 0xfffffe100be1dbb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100be1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100be1dbf0 --- trap 0, rip = 0, rsp = 0xfffffe100be1dcb0, rbp = 0 --- Uptime: 14s Dumping 2340 out of 64456 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/linux.ko.symbols...done. Loaded symbols for /boot/kernel/linux.ko.symbols Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel/if_lagg.ko.symbols Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel/snd_spicds.ko.symbols Reading symbols from /boot/kernel/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel/coretemp.ko.symbols Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel/ichsmb.ko.symbols Reading symbols from /boot/kernel/smbus.ko.symbols...done. Loaded symbols for /boot/kernel/smbus.ko.symbols Reading symbols from /boot/kernel/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel/ichwd.ko.symbols Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel/cpuctl.ko.symbols Reading symbols from /boot/kernel/crypto.ko.symbols...done. Loaded symbols for /boot/kernel/crypto.ko.symbols Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel/cryptodev.ko.symbols Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel/dtraceall.ko.symbols Reading symbols from /boot/kernel/profile.ko.symbols...done. Loaded symbols for /boot/kernel/profile.ko.symbols Reading symbols from /boot/kernel/cyclic.ko.symbols...done. Loaded symbols for /boot/kernel/cyclic.ko.symbols Reading symbols from /boot/kernel/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel/dtrace.ko.symbols Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel/systrace.ko.symbols...done. Loaded symbols for /boot/kernel/systrace.ko.symbols Reading symbols from /boot/kernel/sdt.ko.symbols...done. Loaded symbols for /boot/kernel/sdt.ko.symbols Reading symbols from /boot/kernel/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel/lockstat.ko.symbols Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel/fasttrap.ko.symbols Reading symbols from /boot/kernel/fbt.ko.symbols...done. Loaded symbols for /boot/kernel/fbt.ko.symbols Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel/dtnfscl.ko.symbols Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi.ko.symbols Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkms.ko.symbols Reading symbols from /boot/kernel/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel/iicbb.ko.symbols Reading symbols from /boot/kernel/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel/iicbus.ko.symbols Reading symbols from /boot/kernel/iic.ko.symbols...done. Loaded symbols for /boot/kernel/iic.ko.symbols Reading symbols from /boot/kernel/drm2.ko.symbols...done. Loaded symbols for /boot/kernel/drm2.ko.symbols Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. Loaded symbols for /boot/kernel/fdescfs.ko.symbols #0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0xffffffff80a15267 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0xffffffff80a15808 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:746 #3 0xffffffff80a15853 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:675 #4 0xffffffff8032f0af in assfail3 (a=, lv=, op=, rv=, f=, l=) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91 #5 0xffffffff803c99a0 in zap_get_leaf_byblk (zap=, blkid=, tx=, lt=, lp=0xfffffe100be1d4d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:534 #6 0xffffffff803c7053 in zap_deref_leaf (zap=0xfffff80024d3ff00, h=9377620324092215296, tx=0x0, lt=RW_READER, lp=0xfffffe100be1d4d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:585 #7 0xffffffff803c8e11 in fzap_cursor_retrieve (zap=0xfffff80024d3ff00, zc=0xfffffe100be1d4c8, za=0xfffffe100be1d3b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:1181 #8 0xffffffff803cf40d in zap_cursor_retrieve (zc=0xfffffe100be1d4c8, za=0xfffffe100be1d3b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1290 #9 0xffffffff8035ca53 in ddt_zap_walk (os=0xfffff80024948000, object=156, dde=0xfffffe100be1d818, walk=0xfffff8001181fad8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c:117 #10 0xffffffff8035c709 in ddt_walk (spa=0xfffffe00110cd000, ddb=0xfffff8001181fac0, dde=0xfffffe100be1d818) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:218 #11 0xffffffff8038c977 in dsl_scan_sync (dp=0xfffff80024095000, tx=0xfffff800276b7600) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c:1194 #12 0xffffffff803a7a91 in spa_sync (spa=0xfffffe00110cd000, txg=17880931) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:6610 #13 0xffffffff803b332d in txg_sync_thread (arg=0xfffff80024095000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:517 #14 0xffffffff809e3694 in fork_exit ( callout=0xffffffff803b30e0 , arg=0xfffff80024095000, frame=0xfffffe100be1dc00) at /usr/src/sys/kern/kern_fork.c:977 #15 0xffffffff80e07fce in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #16 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) What can I do to fix the data on the disk? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 18:02:40 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 386CF874; Sat, 30 Aug 2014 18:02:40 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07CE41738; Sat, 30 Aug 2014 18:02:40 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id ft15so2763910pdb.19 for ; Sat, 30 Aug 2014 11:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=2mPm4EHVMBdQE0JVMfbRIc57CyEHM1VwQL0eftI61t0=; b=nMUi0c0OmXSLXdPEBtEIUWmh2BUPTCOIyErKtGBo41qBjLYaz+UyylZYmf/YVYsQS/ a74YJNofEO8zlC4MMTbWdjngO9eWRJs9d6UamxoLbzPQCzUCqV4PQWeHNtqPNaYljktC t7Od5Ju9tjuC+Hh/i6eaSnw2a6n2E344bCuWzAeaYY9mVadAUqDQLGnTGWAIRu0txREz VBRi5cBPkezPEPCYLTd0Q5B7Dm4MXGvLHHYqLpWZDPA+CX5DuXHKu1Aqk7ke2UFwSv+n JSA+13vKF/j1H9KDc52v72TVAZfXdHG38UXZ/z/Vt8gII33/021zovMLNv+Li6MAcQ/F rqvA== X-Received: by 10.66.236.38 with SMTP id ur6mr26015489pac.49.1409421759552; Sat, 30 Aug 2014 11:02:39 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:d9df:e63e:f2c6:ea6d? ([2601:8:ab80:7d6:d9df:e63e:f2c6:ea6d]) by mx.google.com with ESMTPSA id cw5sm3425200pbc.9.2014.08.30.11.02.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 30 Aug 2014 11:02:38 -0700 (PDT) References: <20140830172958.GA99694@borg.lerctr.org> Mime-Version: 1.0 (1.0) In-Reply-To: <20140830172958.GA99694@borg.lerctr.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <2EBFA952-C7EE-49D5-B9D3-301FE690983B@gmail.com> X-Mailer: iPhone Mail (11D257) From: Garrett Cooper Subject: Re: Solaris Assert: How to fix? Date: Sat, 30 Aug 2014 11:02:37 -0700 To: Larry Rosenman Cc: "freebsd-fs@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 18:02:40 -0000 > On Aug 30, 2014, at 10:29, Larry Rosenman wrote: >=20 > I'm getting: Hi Larry, Please file a bug on bugs.freebsd.org so this can be tracked and resolve= d appropriately. Please be sure to include relevant details like the last ke= rnel that booted properly, the kernel that fails to boot, and your zpool sta= tus output. If you can provide the point in time where it worked and failed,= someone can potentially bisect the change to help resolve the issue. Thanks! -Garrett= From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 21:03:42 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 055BB6FD for ; Sat, 30 Aug 2014 21:03:42 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E09C919A2 for ; Sat, 30 Aug 2014 21:03:41 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UL3flh077941 for ; Sat, 30 Aug 2014 21:03:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 21:03:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 21:03:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 --- Comment #3 from Larry Rosenman --- I'm STILL seeing these, and have commented out: =================================================================== --- zap.c (revision 270765) +++ zap.c (working copy) @@ -476,7 +476,9 @@ * chain. There should be no chained leafs (as we have removed * support for them). */ +#if 0 /*LER: to see what else blows up */ ASSERT0(l->l_phys->l_hdr.lh_pad1); +#endif /* * There should be more hash entries than there can be @@ -531,7 +533,9 @@ ASSERT3U(l->l_blkid, ==, blkid); ASSERT3P(l->l_dbuf, ==, db); ASSERT3P(l->l_phys, ==, l->l_dbuf->db_data); +#if 0 /* LER */ ASSERT3U(l->l_phys->l_hdr.lh_block_type, ==, ZBT_LEAF); +#endif ASSERT3U(l->l_phys->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); *lp = l; borg.lerctr.org /usr/src.old/sys/cddl/contrib/opensolaris/uts/common/fs/zfs $ Can I PLEASE get someone to look at these, and tell me what I need to do to fix the on-disk image? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 21:04:45 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D2B4777 for ; Sat, 30 Aug 2014 21:04:45 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4448019AE for ; Sat, 30 Aug 2014 21:04:45 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UL4jnH092471 for ; Sat, 30 Aug 2014 21:04:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 21:04:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 21:04:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 --- Comment #4 from Larry Rosenman --- Created attachment 146572 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=146572&action=edit Latest panic core.txt. I **DO** have the vmcore available -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 21:06:01 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CCB2819; Sat, 30 Aug 2014 21:06:01 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 29CB919C5; Sat, 30 Aug 2014 21:06:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=fRAkvgNTMkmRyFCAy/MjQwRYSPzWcmEf6Mg8NUM2Yq0=; b=hkl8o5QiUyy/4m0U7mULsfMfkD11IRaVkTcKq4VEjHfVoMV2ddfmeor8bduxMaiwGZ9CfP9ciFv0ylW7ukjG78/eiwI3NcogcS97n5f6hpcknHQj3j8KmUW9WicMEA0Prpksl3rjyIEWC4UAd8vrPccnN4d625CM/jZjI/LXM30=; Received: from localhost.lerctr.org ([127.0.0.1]:62372 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XNpqQ-000OVd-CO; Sat, 30 Aug 2014 16:05:59 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sat, 30 Aug 2014 16:05:58 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sat, 30 Aug 2014 16:05:58 -0500 From: Larry Rosenman To: Garrett Cooper Subject: Re: Solaris Assert: How to =?UTF-8?Q?fix=3F?= In-Reply-To: <2EBFA952-C7EE-49D5-B9D3-301FE690983B@gmail.com> References: <20140830172958.GA99694@borg.lerctr.org> <2EBFA952-C7EE-49D5-B9D3-301FE690983B@gmail.com> Message-ID: <3598806df85fd2c298724adcec8a2fb5@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.2 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=0.001 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 21:06:01 -0000 On 2014-08-30 13:02, Garrett Cooper wrote: >> On Aug 30, 2014, at 10:29, Larry Rosenman wrote: >> >> I'm getting: > > Hi Larry, > Please file a bug on bugs.freebsd.org so this can be tracked and > resolved appropriately. Please be sure to include relevant details > like the last kernel that booted properly, the kernel that fails to > boot, and your zpool status output. If you can provide the point in > time where it worked and failed, someone can potentially bisect the > change to help resolve the issue. > Thanks! > -Garrett https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 Which I filed LONG ago, and has seen NO action on it WHATSOEVER. Can SOMEONE please look at it? I added the latest core.txt to it. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 21:27:34 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40E439C; Sat, 30 Aug 2014 21:27:34 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 949B51BE6; Sat, 30 Aug 2014 21:27:33 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id s7so4223015lbd.35 for ; Sat, 30 Aug 2014 14:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=JFSbSryLXBshj7XFrtDC3MlB1qGQD7GOudn70bZOsEg=; b=t3JEqQ2rltDT1jHqAgi3K5hI0yq7wSiQoHqjFHEVc85rGXhEPauF46yiLCPXjANWi8 dWrAu0MgQy+dXF4DDgws4/MY34HhxlUrrU/vsd2Gxet9497R47CrrvrxYuIiyWxgUAjF LL4Ik39emCgEUcCgPVJ1i2sERScI6CWFz0WoCEgtcCPx8bdcce+PHRqQarO6tmpsm4Jd Qt5qlmKKbgonjxsRWvPOmx22HuHkindxE40a5fEF6XIvzZo1g8rhZcaY2ujz14+kMmyD NicTyWe93kNBDNiBYYRMrudquzS6WU7YewnZ96AHr3EBsfNE3lPP0ezUk4i5DzA2LwBv eHLg== X-Received: by 10.153.5.38 with SMTP id cj6mr19267007lad.34.1409434051067; Sat, 30 Aug 2014 14:27:31 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.25.149.205 with HTTP; Sat, 30 Aug 2014 14:26:50 -0700 (PDT) In-Reply-To: <1588968883.30624902.1409399219119.JavaMail.root@uoguelph.ca> References: <201408300051.s7U0peLr073400@gw.catspoiler.org> <1588968883.30624902.1409399219119.JavaMail.root@uoguelph.ca> From: Ivan Voras Date: Sat, 30 Aug 2014 23:26:50 +0200 X-Google-Sender-Auth: iLr3yNoCAbxFlJnVDIZvkyj6KE0 Message-ID: Subject: Re: lockf(1) and NFS To: Rick Macklem Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs , Don Lewis X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 21:27:34 -0000 On 30 August 2014 13:46, Rick Macklem wrote: > Doing locking locally within the client is still possible with the "nolockd" > option. However, when this mount option is used all file locking, > including POSIX byte range locks, are done locally. > > Linux has a similar mount option (called "nolock" I think?), but I > don't know if Solaris has any mount option. > > I'm pretty sure Ivan knows about this, so I suspect he needs lockf(1) > to work across multiple clients for his app., but don't know for sure? Yes, I know about the mount options; actually, I found later that the cause of my problem was not in lockf(1), so I don't actually *need* this patch (or at least I don't need it *yet*). I wanted to have proper locking working across machines. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 22:14:26 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3658DA43 for ; Sat, 30 Aug 2014 22:14:26 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 184551FD1 for ; Sat, 30 Aug 2014 22:14:26 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UMEPWb034590 for ; Sat, 30 Aug 2014 22:14:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 22:14:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: peter@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 22:14:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 Peter Wemm changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |peter@FreeBSD.org --- Comment #5 from Peter Wemm --- If you look at the whole comment for one of the assertions you disabled: /* * lhr_pad was previously used for the next leaf in the leaf * chain. There should be no chained leafs (as we have removed * support for them). */ ASSERT0(l->l_phys->l_hdr.lh_pad1); Or looking at the annotation: 168404 pjd /* 168404 pjd * lhr_pad was previously used for the next leaf in the leaf 168404 pjd * chain. There should be no chained leafs (as we have removed 168404 pjd * support for them). 168404 pjd */ 240415 mm ASSERT0(l->l_phys->l_hdr.lh_pad1); One of those is the initial commit: ------------------------------------------------------------------------ r168404 | pjd | 2007-04-05 18:09:06 -0700 (Thu, 05 Apr 2007) | 11 lines Please welcome ZFS - The last word in file systems. ZFS file system was ported from OpenSolaris operating system. The code in under CDDL license. ------------------------------------------------------------------------ The second is: ------------------------------------------------------------------------ r240415 | mm | 2012-09-12 11:05:43 -0700 (Wed, 12 Sep 2012) | 21 lines Merge recent zfs vendor changes, sync code and adjust userland DEBUG. ------------------------------------------------------------------------ The change is just a syntax one: - ASSERT3U(l->l_phys->l_hdr.lh_pad1, ==, 0); + ASSERT0(l->l_phys->l_hdr.lh_pad1); So.. ever since ZFS existed in FreeBSD, that field should be zero, on disk. You are tripping those asserts because it isn't the case on your machine. This suggests to me that you've either got a very old file system that pre-dates freebsd, or you've got some evil on-disk corruption. I suspect the latter. If this was my machine, I'd be contemplating a backup/restore. I see from the disk sizes you may have a lot of data, but that's what I would be doing. I also see the stack traces come from the zfs de-dupe code. There are a great number of people who have a very dim view of that code, even the authors apologized for it. If it were my machine, I would be turning off dedup immediately, if not sooner. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 22:17:36 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8276BB9 for ; Sat, 30 Aug 2014 22:17:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 902321FF0 for ; Sat, 30 Aug 2014 22:17:36 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UMHaMC037101 for ; Sat, 30 Aug 2014 22:17:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 22:17:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 22:17:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 --- Comment #6 from Larry Rosenman --- This pool has been around a LONG time -- I'm wondering where I can borrow a Terabyte of disk to backup what I care about and rebuild it one more time... (this pool started on FreeBSD 8(IIRC) on 400G disk, and got upgraded....) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 22:51:16 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1550852A for ; Sat, 30 Aug 2014 22:51:16 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F14361340 for ; Sat, 30 Aug 2014 22:51:15 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UMpFNI099018 for ; Sat, 30 Aug 2014 22:51:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 22:51:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: peter@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 22:51:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 --- Comment #7 from Peter Wemm --- Out of curiosity, what does "zpool status -D poolname" show? http://serverfault.com/questions/533877/how-large-is-my-zfs-dedupe-table-at-the-moment You might be burning a significant chunk of ARC metadata space to hold the dedupe table in memory. If I had to guess at this point, I would guess that there are a couple of corrupt records in the on-disk dedup table for one or more of your data sets. Since this is an old pool, there's been plenty of opportunities over the years for this to accumulate while zfs support matured. >From your comment, it sounds like you have 1TB of data spread over a pool made of 2TB disks? Is that deduplicated? Is one of the drives a spare? (if so, can you take the spare offline and make a pool on it to facilitate a dump/restore?) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Aug 30 23:49:58 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0AD7C8B for ; Sat, 30 Aug 2014 23:49:58 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE6E51988 for ; Sat, 30 Aug 2014 23:49:58 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s7UNnw7K053053 for ; Sat, 30 Aug 2014 23:49:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 186112] [zfs] [panic] ZFS Panic/Solaris Assert/zap.c:479 Date: Sat, 30 Aug 2014 23:49:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 23:49:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186112 --- Comment #8 from Larry Rosenman --- borg.lerctr.org /home/ler $ zpool status -D zroot pool: zroot state: ONLINE scan: scrub repaired 0 in 5h8m with 0 errors on Sat Aug 30 13:53:49 2014 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gptid/71e84d97-043a-11e2-8cd6-003048f2299c ONLINE 0 0 0 gptid/3540b18c-0db2-11e2-8ef2-003048f2299c ONLINE 0 0 0 gptid/b0a645ed-0db9-11e2-8ef2-003048f2299c ONLINE 0 0 0 gptid/dc133fc1-0dc0-11e2-8ef2-003048f2299c ONLINE 0 0 0 gptid/21bef448-0dc8-11e2-8ef2-003048f2299c ONLINE 0 0 0 gptid/46033e1c-0432-11e2-8cd6-003048f2299c ONLINE 0 0 0 errors: No known data errors dedup: DDT entries 1415219, size 2676 on disk, 540 in core bucket allocated referenced ______ ______________________________ ______________________________ refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE ------ ------ ----- ----- ----- ------ ----- ----- ----- 1 1.26M 116G 66.3G 69.2G 1.26M 116G 66.3G 69.2G 2 86.1K 6.84G 6.04G 6.20G 174K 13.8G 12.1G 12.4G 4 3.15K 253M 243M 249M 14.6K 1.00G 985M 1019M 8 196 2.32M 823K 1.78M 1.96K 24.5M 8.42M 18.4M 16 109 1.23M 1024K 1.52M 2.50K 31.9M 26.2M 38.2M 32 33 330K 318K 498K 1.37K 18.1M 17.5M 24.9M 64 3 1.50K 1.50K 19.2K 249 124K 124K 1.55M 128 3 28K 28K 38.3K 470 4.96M 4.96M 6.40M 256 4 162K 162K 173K 1.33K 64.3M 64.3M 67.6M Total 1.35M 123G 72.6G 75.7G 1.45M 131G 79.5G 82.8G borg.lerctr.org /home/ler $ I'm actually using 500G on a 6 disk raid-Z1 pool of 2TB disks. That's the "logical Used" (approx). I went out and bought a 4TB NAS to use for this, and then for other things. -- You are receiving this mail because: You are the assignee for the bug.