From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 10:27:23 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D02BE10657C8 for ; Sun, 25 Dec 2011 10:27:23 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 558178FC14 for ; Sun, 25 Dec 2011 10:27:23 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id pBPARJff045147 for ; Sun, 25 Dec 2011 10:27:19 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.3 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id pBPARJPL045146 for freebsd-fs@freebsd.org; Sun, 25 Dec 2011 11:27:19 +0100 (CET) (envelope-from marco) Date: Sun, 25 Dec 2011 11:27:19 +0100 From: Marco van Tol To: FreeBSD FS Message-ID: <20111225102719.GA44906@tolstoy.tols.org> Mail-Followup-To: FreeBSD FS Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tolstoy.tols.org Subject: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 10:27:23 -0000 Hi there, I was experimenting was zfs send and receive, and noticed when including all properties in the transfer the mountpoint happily comes along as well. Ofcourse. This could give issues when receiving the filesysyem not into its own zpool, because mountpoint settings can collide and give some surprises on reboots. But then I found out that if you change a setting on the receiving side, it will not get overwritten by future receives, which is good. :-) Then I started wondering how to figure out received properties on the receiving side after overriding them. This email from 2009 seems relevant. http://mail.opensolaris.org/pipermail/opensolaris-arc/2009-September/018315.html In that mail there is mention about "zfs get -s received", which is accepted on the commandline, but doesn't display the received property separatedly from what is currently in effect. Also there is mention of "zfs inherit -S", which I have tested to work, and which is great. It would be good to include it in the freebsd zfs manual page, as it is a very usefull zfs inherit flag. So, long story short: - Is there a way to see received properties on a filesystem after they have been locally overrided on FreeBSD? - Can the -S flag to inherit be included in the manual page? Thanks, and merry christmas! Marco System: FreeBSD 8.2-STABLE #13: Sat Dec 24 01:54:44 CET 2011 ZFS Pool Version: 28 ZFS filesystem version: 4 on the received FS, 5 for the rest -- Als de redding het hoogst is, is de nood nabij! From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 12:54:27 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22277106566B for ; Sun, 25 Dec 2011 12:54:27 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A52528FC08 for ; Sun, 25 Dec 2011 12:54:26 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Renaq-0004Fa-Ph for freebsd-fs@freebsd.org; Sun, 25 Dec 2011 13:54:24 +0100 Received: from dyn1243-120.vpn.ic.ac.uk ([129.31.243.120]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 25 Dec 2011 13:54:24 +0100 Received: from johannes by dyn1243-120.vpn.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 25 Dec 2011 13:54:24 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Sun, 25 Dec 2011 12:54:11 +0000 Lines: 44 Message-ID: References: <20111225102719.GA44906@tolstoy.tols.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: dyn1243-120.vpn.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 In-Reply-To: <20111225102719.GA44906@tolstoy.tols.org> Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 12:54:27 -0000 On 25/12/2011 10:27, Marco van Tol wrote: > Hi there, > > I was experimenting was zfs send and receive, and noticed when including > all properties in the transfer the mountpoint happily comes along as > well. Ofcourse. This could give issues when receiving the filesysyem > not into its own zpool, because mountpoint settings can collide and give > some surprises on reboots. Check out zfs receive -u, it doesn't mount the receiving filesystem. zpool import -N doesn't mount your importing backup-pool. > But then I found out that if you change a > setting on the receiving side, it will not get overwritten by future > receives, which is good. :-) > > Then I started wondering how to figure out received properties on the > receiving side after overriding them. This email from 2009 seems > relevant. > http://mail.opensolaris.org/pipermail/opensolaris-arc/2009-September/018315.html > > In that mail there is mention about "zfs get -s received", which > is accepted on the commandline, but doesn't display the received > property separatedly from what is currently in effect. > > Also there is mention of "zfs inherit -S", which I have tested to work, > and which is great. It would be good to include it in the freebsd zfs > manual page, as it is a very usefull zfs inherit flag. > > So, long story short: > - Is there a way to see received properties on a filesystem after they > have been locally overrided on FreeBSD? > - Can the -S flag to inherit be included in the manual page? > > Thanks, and merry christmas! > > Marco > > System: FreeBSD 8.2-STABLE #13: Sat Dec 24 01:54:44 CET 2011 > ZFS Pool Version: 28 > ZFS filesystem version: 4 on the received FS, 5 for the rest > From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 13:30:48 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F2A31065677 for ; Sun, 25 Dec 2011 13:30:48 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A6E8A8FC21 for ; Sun, 25 Dec 2011 13:30:47 +0000 (UTC) Received: by lahl5 with SMTP id l5so5823326lah.13 for ; Sun, 25 Dec 2011 05:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kY4FeoLYzNLa2IBqS+NYj8ff+d3XCsCx1pRgiNGzwmI=; b=ep/liLfbLNJEQ3hYPxpPvbiAnSlutJ6rSAPmh+iIhUByQ077LVBTyIvEk69xxrABIj PLMy3/S46wOlMsZzVaLYXieWgSREgCZAl5x98kNquHVx0jopCq7Ct6Z/cj5YGbCLmLoi GSG4ihhtOA04wAcDiGOIGQ1dPuxcVGGfyhjmw= MIME-Version: 1.0 Received: by 10.152.122.100 with SMTP id lr4mr2865092lab.47.1324818133741; Sun, 25 Dec 2011 05:02:13 -0800 (PST) Received: by 10.152.20.202 with HTTP; Sun, 25 Dec 2011 05:02:13 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 14:02:13 +0100 Message-ID: From: claudiu vasadi To: Johannes Totz Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 13:30:48 -0000 On Sun, Dec 25, 2011 at 1:54 PM, Johannes Totz wrote: Check out zfs receive -u, it doesn't mount the receiving filesystem. zpool > import -N doesn't mount your importing backup-pool. > > True, it doesn't, but upon reboot, since the received datasets keep their properties, they will be mounted (and this has the risk of potentially overwriting an exiting mountpoint). -- Best regards, Claudiu Vasadi From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 19:50:13 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56B34106566B for ; Sun, 25 Dec 2011 19:50:13 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0E7148FC0C for ; Sun, 25 Dec 2011 19:50:12 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Reu5C-00051I-QB for freebsd-fs@freebsd.org; Sun, 25 Dec 2011 20:50:11 +0100 Received: from dyn1247-107.vpn.ic.ac.uk ([129.31.247.107]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 25 Dec 2011 20:50:10 +0100 Received: from johannes by dyn1247-107.vpn.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 25 Dec 2011 20:50:10 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Sun, 25 Dec 2011 19:49:57 +0000 Lines: 15 Message-ID: References: <20111225102719.GA44906@tolstoy.tols.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: dyn1247-107.vpn.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 In-Reply-To: Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 19:50:13 -0000 On 25/12/2011 13:02, claudiu vasadi wrote: > On Sun, Dec 25, 2011 at 1:54 PM, Johannes Totz wrote: > > Check out zfs receive -u, it doesn't mount the receiving filesystem. zpool >> import -N doesn't mount your importing backup-pool. >> >> > True, it doesn't, but upon reboot, since the received datasets keep their > properties, they will be mounted (and this has the risk of potentially > overwriting an exiting mountpoint). Good point! Haven't thought about reboot, I always export my backup pool when I'm done with send-receive. Could be worked around, I guess, by disabling cache file entry and manually importing it from some random script... From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 20:22:12 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 766501065672 for ; Sun, 25 Dec 2011 20:22:12 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 312D88FC0A for ; Sun, 25 Dec 2011 20:22:11 +0000 (UTC) Received: by yenl9 with SMTP id l9so7514241yen.13 for ; Sun, 25 Dec 2011 12:22:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=wh2hqnAZqJU4Ds1yMzDHpjNROWT/mOl4/lc9LH8M9M8=; b=d3L9rswscHx1K78X/RSRuIIY9Bc/45fgJCx0O9aVtD+N9BpvFWs+fNWXTohzJAI5qU IUU+YwtKyNOsh3a5hi1P5XB4xjnb9wJQWbe1nOqw1cmCp4cNns0fWUj5ecK3rGeoqK6T WTuPznNN1P6XKindIGBzwiDPPotdN85/SNveU= MIME-Version: 1.0 Received: by 10.236.131.97 with SMTP id l61mr30675941yhi.52.1324844531514; Sun, 25 Dec 2011 12:22:11 -0800 (PST) Sender: artemb@gmail.com Received: by 10.147.52.27 with HTTP; Sun, 25 Dec 2011 12:22:11 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 12:22:11 -0800 X-Google-Sender-Auth: RUj5ctRhG2UYnrjr7SqJAvjZt3U Message-ID: From: Artem Belevich To: Johannes Totz Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 20:22:12 -0000 On Sun, Dec 25, 2011 at 11:49 AM, Johannes Totz wrote: > On 25/12/2011 13:02, claudiu vasadi wrote: >> >> On Sun, Dec 25, 2011 at 1:54 PM, Johannes Totz =A0wrot= e: >> >> Check out zfs receive -u, it doesn't mount the receiving filesystem. zpo= ol >>> >>> import -N doesn't mount your importing backup-pool. >>> >>> >> True, it doesn't, but upon reboot, since the received datasets keep thei= r >> properties, they will be mounted (and this has the risk of potentially >> overwriting an exiting mountpoint). > > > Good point! Haven't thought about reboot, I always export my backup pool > when I'm done with send-receive. > Could be worked around, I guess, by disabling cache file entry and manual= ly > importing it from some random script... Perhaps "zpool import/create -R /foo" (or "-o altroot=3Dfoo, cachefile=3Dnone") is what you're looking for. It effectively gives your pool its own name space under /foo and it does not record any info about it in zpool cache, so on reboot the pool should remain unimported. --Artem From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 22:06:52 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F12B5106566C; Sun, 25 Dec 2011 22:06:52 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4408C8FC0A; Sun, 25 Dec 2011 22:06:51 +0000 (UTC) Received: by lahl5 with SMTP id l5so5893463lah.13 for ; Sun, 25 Dec 2011 14:06:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lbPv2fGjxbnc0Ix58sWQO07nrmt7oTEq6x9GafBEA3g=; b=bwn8HeuiQ4yicbfT7rnRFmZP61eEhU/3ZKauwbArPkpzCYkjndfvdX/M66rTvbk9wS zra3BWct3MW4aV5D0Ij1URbfi83VMdgFFHixAcuxhckMQHBTTBEZwlN1c4qUnNO6QBnL kCs6lrKGRVaB7wuszvURYfZ8JoiMA6twLaPoI= MIME-Version: 1.0 Received: by 10.152.110.169 with SMTP id ib9mr9321878lab.33.1324850810981; Sun, 25 Dec 2011 14:06:50 -0800 (PST) Received: by 10.152.20.202 with HTTP; Sun, 25 Dec 2011 14:06:50 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 23:06:50 +0100 Message-ID: From: claudiu vasadi To: Artem Belevich Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 22:06:53 -0000 But that leaves one the "daunting" task of manually mounting the pool before any backups are run and also before a restore is attempted. I believe there should be a cleaner solution to this. -- Best regards, Claudiu Vasadi From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 22:34:29 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0140F106564A for ; Sun, 25 Dec 2011 22:34:29 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id ADE5A8FC13 for ; Sun, 25 Dec 2011 22:34:28 +0000 (UTC) Received: by yhfq46 with SMTP id q46so8232471yhf.13 for ; Sun, 25 Dec 2011 14:34:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=4vwv7MdJSiJOsyjCL3dJo/dQCuvCcdixNfMShCbBsps=; b=jujuRZMIJUPuCehWuJY174nHWDmRyPuv1qOi9nOx1foM0sYm2eGoQ3ULZjXyvcpOy+ uA19SqlOI49H2YKCJjtDkQcRRjArtLkQyqg02EJpCjBbQwPH2AjVNbDFmNQVYjSXGbBZ eep7g0qpSjZnPzKfHNuSbx9SI4BBITIbpJawA= MIME-Version: 1.0 Received: by 10.236.174.2 with SMTP id w2mr30341360yhl.35.1324852467999; Sun, 25 Dec 2011 14:34:27 -0800 (PST) Sender: artemb@gmail.com Received: by 10.147.52.27 with HTTP; Sun, 25 Dec 2011 14:34:27 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 14:34:27 -0800 X-Google-Sender-Auth: nNx--JRAPf-OSqr3Sx6ocS3RfwE Message-ID: From: Artem Belevich To: claudiu vasadi Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 22:34:29 -0000 On Sun, Dec 25, 2011 at 2:06 PM, claudiu vasadi wrote: > > But that leaves one the "daunting" task of manually mounting the pool before > any backups are run and also before a restore is attempted. > I believe there should be a cleaner solution to this. I'm not sure what would be the ideal behavior you have in mind. If I understand you correctly, you want to have properties transferred during send/receive, but in such a way that some of them (mountpoint?) are ignored or interpreted differently. Is that so? --Artem From owner-freebsd-fs@FreeBSD.ORG Sun Dec 25 22:44:31 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A0F31065670; Sun, 25 Dec 2011 22:44:31 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A290F8FC15; Sun, 25 Dec 2011 22:44:30 +0000 (UTC) Received: by lahl5 with SMTP id l5so5897676lah.13 for ; Sun, 25 Dec 2011 14:44:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lREu/C+XsYBZXuiY8oP+LIXhEmd0sldwqTp+2QUesdo=; b=hdT9+opnicRZCuGhlAZa2NfT9Oi2gNzCVv5YiBfBkd1jGTcGVjyHdkl9AlyDwpjqaK 3GCX88c3kUTPqo7Jc+RxGdHqdK3Ij26ysUi1vLfQ/It5jW3rcyrXBZlWmJBY9z/K5/Ds OqNaM2HgC55pWNHRow1q3Sf8aBRzZVbzNyly0= MIME-Version: 1.0 Received: by 10.152.145.71 with SMTP id ss7mr16355591lab.28.1324853069293; Sun, 25 Dec 2011 14:44:29 -0800 (PST) Received: by 10.152.20.202 with HTTP; Sun, 25 Dec 2011 14:44:29 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 23:44:29 +0100 Message-ID: From: claudiu vasadi To: Artem Belevich Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 22:44:31 -0000 Correct. This is to avoid, for instance, a zfs send/receive of a recursive (-r) snapshot of the ZFS boot pool. Since all properties (mountpoints too of course) are kept, upon reboot, the receive-ing side would mount the snapshot datasets over any existing (and same) datasets. Does it make sense ? Best regards, Claudiu Vasadi From owner-freebsd-fs@FreeBSD.ORG Mon Dec 26 00:10:44 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 615FD1065673 for ; Mon, 26 Dec 2011 00:10:44 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1A6A58FC1A for ; Mon, 26 Dec 2011 00:10:43 +0000 (UTC) Received: by ggnp1 with SMTP id p1so10739893ggn.13 for ; Sun, 25 Dec 2011 16:10:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=ZkTOVoeQGgOetX3R7xCNuC798gLJ6DuiTrD9bqDWvUk=; b=gQjcLweERF6llWA75+W4RlVLEAc2xSwDQ/zF1jQqL9XToqh4lWoIDpDYibS6Hd9Ltu 3gfFqR/rmBODVcsOV1+X6nkvdYabaDMwoBAIVoXmaSYx/ZTKTakIYItGNx/nSNooQK4M 2Vo0NK5rJ2hmnY85HqBdZlhNIVQkz9fKKwHEc= MIME-Version: 1.0 Received: by 10.101.174.17 with SMTP id b17mr9313463anp.62.1324858243345; Sun, 25 Dec 2011 16:10:43 -0800 (PST) Sender: artemb@gmail.com Received: by 10.147.52.27 with HTTP; Sun, 25 Dec 2011 16:10:43 -0800 (PST) In-Reply-To: References: <20111225102719.GA44906@tolstoy.tols.org> Date: Sun, 25 Dec 2011 16:10:43 -0800 X-Google-Sender-Auth: HLgCFVYmilXxBN5pnmobe4L4A3A Message-ID: From: Artem Belevich To: claudiu vasadi Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 00:10:44 -0000 On Sun, Dec 25, 2011 at 2:44 PM, claudiu vasadi wrote: > Correct. This is to avoid, for instance, a zfs send/receive of a recursive > (-r) snapshot of the ZFS boot pool. Since all properties (mountpoints too of > course) are kept, upon reboot, the receive-ing side would mount the snapshot > datasets over any existing (and same) datasets. > > Does it make sense ? I think I see why you may want it, but I don't think you'll get that behavior from ZFS without hacking it. "zpool import -R root" is probably your best bet. Alas altroot property is not persistent and the pool would have to reimported on reboot. Another option may be to backup/restore properties separately and ignore properties during send/receive. --Artem From owner-freebsd-fs@FreeBSD.ORG Mon Dec 26 11:07:06 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 915871065677 for ; Mon, 26 Dec 2011 11:07:06 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7406C8FC16 for ; Mon, 26 Dec 2011 11:07:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBQB76ag022479 for ; Mon, 26 Dec 2011 11:07:06 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBQB75Gg022477 for freebsd-fs@FreeBSD.org; Mon, 26 Dec 2011 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 26 Dec 2011 11:07:05 GMT Message-Id: <201112261107.pBQB75Gg022477@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 11:07:06 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/162083 fs [zfs] [panic] zfs unmount -f pool o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo o kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161674 fs [ufs] snapshot on journaled ufs doesn't work o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161511 fs [unionfs] Filesystem deadlocks when using multiple uni o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs Random UFS root filesystem corruption with SU+J [regre o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159971 fs [ffs] [panic] panic with soft updates journaling durin o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158711 fs [ffs] [panic] panic in ffs_blkfree and ffs_valloc o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157722 fs [geli] unable to newfs a geli encrypted partition o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs f kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153847 fs [nfs] [panic] Kernel panic from incorrect m_free in nf o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files f sparc/123566 fs [zfs] zpool import issue: EOVERFLOW o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] [iconv] mount_msdosfs: msdosfs_iconv: Operat o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 256 problems total. From owner-freebsd-fs@FreeBSD.ORG Tue Dec 27 16:37:50 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FFA61065680 for ; Tue, 27 Dec 2011 16:37:50 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A6AF28FC0A for ; Tue, 27 Dec 2011 16:37:49 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rfa27-0004W5-0Z for freebsd-fs@freebsd.org; Tue, 27 Dec 2011 17:37:47 +0100 Received: from dyn1243-120.vpn.ic.ac.uk ([129.31.243.120]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 27 Dec 2011 17:37:46 +0100 Received: from johannes by dyn1243-120.vpn.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 27 Dec 2011 17:37:46 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Tue, 27 Dec 2011 16:37:32 +0000 Lines: 36 Message-ID: References: <4EE764DA.4030206@brockmann-consult.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: dyn1243-120.vpn.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 In-Reply-To: Subject: Re: zpool failmode=continue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 16:37:50 -0000 On 13/12/2011 14:53, Johannes Totz wrote: > On 13/12/2011 14:44, Peter Maloney wrote: >> Are you using NFS or ZVOLs? > > Neither, see below. > >> My zfs hangs (all IO) if I go into the .zfs/snapshots directory over >> NFS. (planning to file a PR after I find a way to reproduce it reliably, >> but it depends on specific snapshots). My workaround is to mount >> /var/empty on top of the .zfs directory on the nfs client, and give >> nobody else access. Another workaround I thought of is to have another >> parent directory in the dataset, and share the 2nd level down which >> doesn't contain the .zfs directory. > > My pool is not exported to any clients. My situation is actually the > other way around, should have been more clear: the block device on which > I created the pool is a on the network. > It's kind of a crazy setup: > - sshfs to another (Linux) machine > - create big image file > - create pool from file vdev mounted via sshfs > Eventually the network drops out, zpool shows read and write errors, > fine so far. But all new io just hangs instead of failing with an error. After some observation, turns out that periodic/security/100.chksetuid makes all i/o die on the test pool. Is find doing something funny? As it does not even search around on the testpool (it's imported but not mounted) nor the sshfs (only ufs and zfs is searched) I don't have any clue as to what might go wrong... zpool status simply mentions read/write errors. I noticed this because when logging iostat to a file, i/o always stopped at 3am. But I can also trigger it by simply running 100.chksetuid. All the other stuff in daily and security is fine. Anybody has any idea what might cause it? From owner-freebsd-fs@FreeBSD.ORG Tue Dec 27 20:46:57 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24710106564A for ; Tue, 27 Dec 2011 20:46:57 +0000 (UTC) (envelope-from rolling.robot@gmail.com) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id AD1458FC08 for ; Tue, 27 Dec 2011 20:46:56 +0000 (UTC) Received: by wibhr1 with SMTP id hr1so10293632wib.13 for ; Tue, 27 Dec 2011 12:46:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; bh=KkXMqt93815CN+zaR3xCqImGubyJpTKEntb48t6Jb/8=; b=pojkKr9eS36NaejDLnosPfLfJKW9DPvN7/O3zNAyxKnJGAtPk81YMOV09KHJbBuv6S lESgrRt8Fwn/paXMhinfscEk/ylOiopjD+HTbf21J+hhcrV2vzOnJR5G6dlbPanVWKbo 3rquSYs6mgJgmeGahlF+g4LHS1QkiRW2V8X8I= Received: by 10.180.72.133 with SMTP id d5mr64209300wiv.7.1325016962336; Tue, 27 Dec 2011 12:16:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.180.3.233 with HTTP; Tue, 27 Dec 2011 12:15:41 -0800 (PST) From: Peter Mukhachev Date: Tue, 27 Dec 2011 23:15:41 +0300 Message-ID: To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: journal on raid device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 20:46:57 -0000 Hello everyone! I recently bought a via6421 bulk raid controller and I'm trying to get journalling working. I've partitioned it and set up journal with fdisk, bsdlabel and gjournal: # fdisk -I /dev/ar0 # bsdlabel -w /dev/ar0 # gjournal load # gjournal label /dev/ar0s1a # newfs -O 2 -J /dev/ar0s1a.journal # echo 'geom_journal_load="YES" ' >> /boot/loader.conf After that I can mount a filesystem and do whatever I want. But after reboot I have no /dev/ar0s1a.journal, only /dev/ar0s1a that I can mount without journalling. However, on the disks that comprize the massive, file systems with journals are visible and mountable. They are ad8s1a.journal ad9s1a.journal. # mount /dev/ad8s1a.journal on /mnt (ufs, local, read-only, gjournal) Is it a bug or I'm doing something wrong? Mukhachev Peter. PS # uname -a FreeBSD pasty.lan 8.2-STABLE-201105 FreeBSD 8.2-STABLE-201105 #0: Tue May 17 05:46:49 UTC 2011 root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 From owner-freebsd-fs@FreeBSD.ORG Tue Dec 27 21:15:01 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6382A106564A for ; Tue, 27 Dec 2011 21:15:01 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id 356FE8FC0A for ; Tue, 27 Dec 2011 21:15:00 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta02.emeryville.ca.mail.comcast.net with comcast id EM8t1i0030cQ2SLA2MEuhw; Tue, 27 Dec 2011 21:14:54 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta10.emeryville.ca.mail.comcast.net with comcast id EMBa1i00r1t3BNj8WMBbPh; Tue, 27 Dec 2011 21:11:36 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id A92FE102C19; Tue, 27 Dec 2011 13:14:58 -0800 (PST) Date: Tue, 27 Dec 2011 13:14:58 -0800 From: Jeremy Chadwick To: Johannes Totz Message-ID: <20111227211458.GA21839@icarus.home.lan> References: <4EE764DA.4030206@brockmann-consult.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: zpool failmode=continue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 21:15:01 -0000 On Tue, Dec 27, 2011 at 04:37:32PM +0000, Johannes Totz wrote: > On 13/12/2011 14:53, Johannes Totz wrote: > >On 13/12/2011 14:44, Peter Maloney wrote: > >>Are you using NFS or ZVOLs? > > > >Neither, see below. > > > >>My zfs hangs (all IO) if I go into the .zfs/snapshots directory over > >>NFS. (planning to file a PR after I find a way to reproduce it reliably, > >>but it depends on specific snapshots). My workaround is to mount > >>/var/empty on top of the .zfs directory on the nfs client, and give > >>nobody else access. Another workaround I thought of is to have another > >>parent directory in the dataset, and share the 2nd level down which > >>doesn't contain the .zfs directory. > > > >My pool is not exported to any clients. My situation is actually the > >other way around, should have been more clear: the block device on which > >I created the pool is a on the network. > >It's kind of a crazy setup: > >- sshfs to another (Linux) machine > >- create big image file > >- create pool from file vdev mounted via sshfs > >Eventually the network drops out, zpool shows read and write errors, > >fine so far. But all new io just hangs instead of failing with an error. > > After some observation, turns out that > periodic/security/100.chksetuid makes all i/o die on the test pool. > Is find doing something funny? As it does not even search around on > the testpool (it's imported but not mounted) nor the sshfs (only ufs > and zfs is searched) I don't have any clue as to what might go > wrong... > zpool status simply mentions read/write errors. > > I noticed this because when logging iostat to a file, i/o always > stopped at 3am. But I can also trigger it by simply running > 100.chksetuid. All the other stuff in daily and security is fine. > > Anybody has any idea what might cause it? This ""problem"" (note the quotes) has been brought up before. There isn't anything wrong with the periodic script; if you look at the script, you'll see that it's ""heavy"" on I/O due to all of the operations being done: find -sx $MP /dev/null -type f \ \( -perm -u+x -or -perm -g+x -or -perm -o+x \) \ \( -perm -u+s -or -perm -g+s \) -exec ls -liTd \{\} \+ | check_diff setuid - "${host} setuid diffs:" This is going to traverse the filesystem and do a couple stat(2) calls (I assume find(1) is smart enough to consolidate them into 1 or 2 at most), plus there's the -exec call on every single result (pretty sure one cannot use xargs in this case given the nature of what's being done). I can try to dig up those threads for you, but I'm sure if you search mailing lists for "100.chksetuid zfs" you'll see. ZFS tends to "bring to light" underlying issues with hardware, as it stresses the system a lot more than UFS would. For example, folks using mps(4) (I think; trying to remember which LSI driver) were having problems for a while and "fixes to make ZFS happy" were committed to the driver. You get the idea I hope. Your statement here: > zpool status simply mentions read/write errors. Acts as pretty much confirmation of this fact. You're going to need to provide a verbose description of your setup, including all storage controllers that are used/associated with ZFS, every disk involved (model string would be helpful), if possible SMART statistics for each disk (smartctl -A), if you have a heterogeneous setup (ZFS on some disks, UFS on others), make.conf, loader.conf, sysctl.conf, full "zpool status" output, uname -a, full "dmesg" output, etc.. I make no promises that there's any solution to this as well. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 20:43:38 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9176106564A for ; Thu, 29 Dec 2011 20:43:38 +0000 (UTC) (envelope-from ufs.user@yahoo.com) Received: from nm29.bullet.mail.ac4.yahoo.com (nm29.bullet.mail.ac4.yahoo.com [98.139.52.226]) by mx1.freebsd.org (Postfix) with SMTP id 733EE8FC19 for ; Thu, 29 Dec 2011 20:43:38 +0000 (UTC) Received: from [98.139.52.190] by nm29.bullet.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 20:30:59 -0000 Received: from [98.139.52.158] by tm3.bullet.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 20:30:59 -0000 Received: from [127.0.0.1] by omp1041.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 20:30:59 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 647221.19022.bm@omp1041.mail.ac4.yahoo.com Received: (qmail 70799 invoked by uid 60001); 29 Dec 2011 20:30:59 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1325190659; bh=8yFXjLCG0OTjs8gMsbC2JTFyvbDABZSbCN6TWP6ghzQ=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=lSKavB+En+66r6GwtsxuzKDjsqzOxcZy1vk3vGDj/ggku6MD0g0ACtDIOGHuHocjx+Gs/LvsKQhxeCL8jxhqvt4XOggWsz9gRcSxUw2hybvWRjWRpJ6asq/Mq0CY2Y9ADalqgHHPMCGy9xDfo9K7XQBNdmex+KWXIzXE7cFPqec= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=II1aTgCWdsUyOItgn6807ym2IP//BIU1UPNkG4hfVjm85KTu6OMxNj0lW6ansfdrflv9Ae9MS+Ryhd6xipbxdoJKAvp8G9GHPIeYsv767bAzDb9jjMrAg37KwVLy9/8z1itWjBT07oIQbAYANFjUN64aYUTcStnnM09a7+dwLwU=; X-YMail-OSG: g6i83BYVM1m0_jvSd8TmCUvvJQIHHBkyYoV46rdF5Qc0cqL H.41SuWLKXsz4D1DowU4TfUTpmYhaNl4_rdan0mT82s78sGNhhoDjTR6xjvH Wnbdnled4Dwb1._1HA5FzFpwCZA7uOxl1wTYtAK0QyTf5UodmcSwTxMzD6kC GQZXPjyQeOl.U4422aTqBCQfvBB0v17JRMc949wEJ9ab8YPwl3ph.GhagvpP ABWJqzkJmubZbeys1MouGTQXd7Mxn0_b.E8unhDgPcUq5hrjce9PshnWWQDd OtyVovzmqO9Q0OhPof3nzxAUNYWSiV5j_5sROrvJkbGLrkxkLWe6ZbJLqTMC xWNyh.lIjUNLIq3An9CkmlEWJEFXRbL3DWeC4wfURwA-- Received: from [24.179.213.102] by web140408.mail.bf1.yahoo.com via HTTP; Thu, 29 Dec 2011 12:30:59 PST X-Mailer: YahooMailWebService/0.8.115.331698 Message-ID: <1325190659.66676.YahooMailNeo@web140408.mail.bf1.yahoo.com> Date: Thu, 29 Dec 2011 12:30:59 -0800 (PST) From: UFS User To: "freebsd-fs@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Painted into a corner with i386 system and big FS ... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: UFS User List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 20:43:39 -0000 I have an older i386 system with an 8TB filesystem on it, roughly 40 millio= n inodes.=0A=0AIt appears that this is impossible to fsck - I have nudged u= p maxdsiz all the way to:=0A=0Akern.maxdsiz=3D"3072000000"=0A=0A=0Aand it's= not enough. =A0Any larger values cause the system to become unbootable.=0A= =0ASo, my plan is to boot 8.x amd64 FIXIT, and run fsck from the fixit envi= ronment.=0A=0AThe issues I foresee are:=0A=0A1. Do I need to specify a maxd= siz in the 8.x fixit environment, or does this all just autoscale=0Athese d= ays ? =A0I'm hoping my process doesn't grow past 3.5 GB, but do I need to b= urn a=0Acustom fixit to do that ? =A0Or can I drop into the boot loader of = FIXIT and set a custom load=0Avalue ?=0A=0AOr perhaps I don't care at all a= nd maxdsiz will just auto-scale to however large the fsck gets ?=0A=0A=0A2.= There is only 4 GB of physical in the system ... if I could grow to 3.072 = GB process size in=0Ai386, then all else being equal, I should be able to g= row to 3.5 in 8.x FIXIT, right ?=0A=0A=0A3. If I get larger than 4 GB in am= d64, with a 4GB physical memory, I can always swap, right ?=0ABut where wou= ld I swap to ? =A0Maybe I could mount my /var (which is clean, and is 4 GB = in size)=0Aand swap to there ?=0A=0ACan I specify a file-backed swapfile in= FIXIT ?=0A=0A=0AAnything else I am missing ?=0A=0A=0AThanks. From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 21:36:06 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08162106566B for ; Thu, 29 Dec 2011 21:36:06 +0000 (UTC) (envelope-from dpd@bitgravity.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id CABA28FC0A for ; Thu, 29 Dec 2011 21:36:05 +0000 (UTC) Received: by iadj38 with SMTP id j38so30611403iad.13 for ; Thu, 29 Dec 2011 13:36:05 -0800 (PST) Received: by 10.50.189.137 with SMTP id gi9mr7947541igc.1.1325193007250; Thu, 29 Dec 2011 13:10:07 -0800 (PST) Received: from netops-69.sfo1.bitgravity.com ([209.131.110.69]) by mx.google.com with ESMTPS id rc7sm81276408igb.0.2011.12.29.13.10.06 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 29 Dec 2011 13:10:06 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: David P Discher In-Reply-To: Date: Thu, 29 Dec 2011 13:10:05 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <87E5C7D5-2D73-4A42-B871-8B27448B0DD9@bitgravity.com> References: To: Johannes Totz X-Mailer: Apple Mail (2.1084) Cc: freebsd-fs@freebsd.org Subject: Re: fusefs broken on 8-stable? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 21:36:06 -0000 Johannes - Were you able to get a resolution for this ? I've been able to replicate this panic in fiov_adjust(), but being = called from vnlru, looks like its doing VOP_RECLAIM_APV(). I'm willing to dive deeper and debug this, however would appreciate some = guidance form the community. I'm having issues wrapping my head around = macro FUSE_DIMALLOC(), which I think is using some sort of anonymous = function call to get to fiov_adjust(). I've been battling this for a = week or two now, so I have symbols in fuse module, so I have a bit more = info. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D (kgdb) up #11 0xffffffff81147412 in fdisp_make_pid = (fdip=3D0xffffff9bce6399f0, mp=3DVariable "mp" is not available. ) at fuse_msg.c:788 788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, = fdip->finh, (kgdb) l 783 if (fdip->tick) 784 fticket_refresh(fdip->tick); 785 else 786 fdip->tick =3D fuse_ticket_fetch(data); 787 788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, = fdip->finh, 789 fdip->indata, fdip->iosize); 790 fuse_setup_ihead(fdip->finh, fdip->tick, nid, = op, fdip->iosize, pid, cred); 791 } 792 (kgdb) p fdip $4 =3D (struct fuse_dispatcher *) 0xffffff9bce6399f0 (kgdb) p *(struct fuse_dispatcher *)fdip $5 =3D {tick =3D 0x0, finh =3D 0x0, indata =3D = 0xffffff9bce639a30, iosize =3D 8,=20 nodeid =3D 0, answ_stat =3D 1356898584, answ =3D = 0xffffff0350e0a138} (kgdb) p *(struct fuse_dispatcher *)fdip->tick Cannot access memory at address 0x0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D (kgdb) up #9 0xffffffff807bd7b8 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 228 call trap Current language: auto; currently asm (kgdb) up #10 0xffffffff81146b17 in fiov_adjust (fiov=3D0x30, size=3D48) = at fuse_msg.c:96 96 { Current language: auto; currently c (kgdb) l 91 free(fiov->base, M_FUSEMSG); 92 } 93 94 void 95 fiov_adjust(struct fuse_iov *fiov, size_t size) 96 { 97 if (fiov->allocated_size < size || 98 (fuse_iov_permanent_bufsize >=3D 0 && 99 fiov->allocated_size - size > = fuse_iov_permanent_bufsize && 100 --fiov->credit < 0)) { (kgdb) p fiov $1 =3D (struct fuse_iov *) 0x30 (kgdb) p (struct fuse_iov *)fiov $2 =3D (struct fuse_iov *) 0x30 (kgdb) p *(struct fuse_iov *)fiov Cannot access memory at address 0x30 (kgdb) p size $3 =3D 48 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D (kgdb) bt #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:263 #1 0xffffffff801f6c9c in db_fncall (dummy1=3DVariable "dummy1" = is not available. ) at /usr/src/sys/ddb/db_command.c:548 #2 0xffffffff801f6fd1 in db_command = (last_cmdp=3D0xffffffff80b67a40, cmd_table=3DVariable "cmd_table" is not = available. ) at /usr/src/sys/ddb/db_command.c:445 #3 0xffffffff801f7220 in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #4 0xffffffff801f9269 in db_trap (type=3DVariable "type" is not = available. ) at /usr/src/sys/ddb/db_main.c:229 #5 0xffffffff80575ff1 in kdb_trap (type=3D12, code=3D0, = tf=3D0xffffff9bce6398a0) at /usr/src/sys/kern/subr_kdb.c:548 #6 0xffffffff807d4bad in trap_fatal (frame=3D0xffffff9bce6398a0, = eva=3DVariable "eva" is not available. ) at /usr/src/sys/amd64/amd64/trap.c:820 #7 0xffffffff807d4eea in trap_pfault (frame=3D0xffffff9bce6398a0,= usermode=3D0) at /usr/src/sys/amd64/amd64/trap.c:741 #8 0xffffffff807d52ff in trap (frame=3D0xffffff9bce6398a0) at /usr/src/sys/amd64/amd64/trap.c:478 #9 0xffffffff807bd7b8 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #10 0xffffffff81146b17 in fiov_adjust (fiov=3D0x30, size=3D48) = at fuse_msg.c:96 #11 0xffffffff81147412 in fdisp_make_pid = (fdip=3D0xffffff9bce6399f0, mp=3DVariable "mp" is not available. ) at fuse_msg.c:788 #12 0xffffffff8114a6d4 in fuse_send_forget (mp=3DVariable "mp" = is not available. ) at fuse_vnops.c:378 #13 0xffffffff8114a7d6 in fuse_recyc_backend = (vp=3D0xffffff0350e0a000,=20 td=3D0xffffff0029533000) at fuse_vnops.c:539 #14 0xffffffff808272fa in VOP_RECLAIM_APV = (vop=3D0xffffffff811504e0,=20 a=3D0xffffff9bce639a90) at vnode_if.c:1926 #15 0xffffffff805ccf95 in vgonel (vp=3D0xffffff0350e0a000) at = vnode_if.h:830 #16 0xffffffff805cf9c4 in vnlru_free (count=3D28) at /usr/src/sys/kern/vfs_subr.c:975 #17 0xffffffff805d030f in vnlru_proc () at = /usr/src/sys/kern/vfs_subr.c:834 #18 0xffffffff805211bd in fork_exit (callout=3D0xffffffff805cfec0 = ,=20 arg=3D0x0, frame=3D0xffffff9bce639c50) at = /usr/src/sys/kern/kern_fork.c:876 #19 0xffffffff807bdcfe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:602 =09 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Fatal trap 12: page fault while in kernel mode^M cpuid =3D 7; apic id =3D 15^M fault virtual address =3D 0x40^M fault code =3D supervisor read data, page not = present^M instruction pointer =3D 0x20:0xffffffff81146b17^M stack pointer =3D 0x28:0xffffff9bce639950^M frame pointer =3D 0x28:0xffffff9bce639980^M code segment =3D base 0x0, limit 0xfffff, type 0x1b^M =3D DPL 0, pres = 1, long 1, def32 0, gran 1^M processor eflags =3D interrupt enabled, resume, IOPL =3D = 0^M current process =3D 18 (vnlru)^M [thread pid 18 tid 100138 ] Stopped at fiov_adjust+0x17: movq 0x10(%rdi),%rax db> [-- MARK -- Thu Dec 29 01:15:00 2011] bt Tracing pid 18 tid 100138 td 0xffffff0029533000 fiov_adjust() at fiov_adjust+0x17 fdisp_make_pid() at fdisp_make_pid+0xd2 fuse_send_forget() at fuse_send_forget+0x44 fuse_recyc_backend() at fuse_recyc_backend+0xc6 VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0x4a vgonel() at vgonel+0xf5 vnlru_free() at vnlru_free+0x294 vnlru_proc() at vnlru_proc+0x44f fork_exit() at fork_exit+0x11d fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff9bce639d00, rbp =3D 0 --- db> call doadump=20 =20 --- David P. Discher dpd@bitgravity.com * AIM: bgDavidDPD BITGRAVITY * http://www.bitgravity.com On Jun 3, 2011, at 6:49 PM, Johannes Totz wrote: > On 13/05/2011 13:56, Johannes Totz wrote: >> Heya! >>=20 >> Using encfs (built on top of fuse) gives me panics in combination = with >> rsync. Dump didn't succeed. The info below is transcribbled from a >> photograph. This is repeatable. >> Without dump this is probably not very helpful.... >>=20 >>=20 >> # uname -a >> FreeBSD XXX 8.2-STABLE FreeBSD 8.2-STABLE #1: Thu Mar 10 23:30:08 GMT >> 2011 root@XXX:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > Managed to get a dump: >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 1; apic id =3D 01 > fault virtual address =3D 0x40 > fault code =3D supervisor read data, page not present > instruction pointer =3D 0x20:0xffffffff81f2ba3c > stack pointer =3D 0x28:0xffffff813aa2f570 > frame pointer =3D 0x28:0xffffff813aa2f5a0 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 1560 (rsync) > trap number =3D 12 > panic: page fault > cpuid =3D 1 > KDB: stack backtrace: > #0 0xffffffff805cc226 at kdb_backtrace+0x5c > #1 0xffffffff8059d000 at panic+0x1b4 > #2 0xffffffff80875a75 at trap_fatal+0x394 > #3 0xffffffff80875ce5 at trap_pfault+0x252 > #4 0xffffffff808761b5 at trap+0x3f4 > #5 0xffffffff8085f384 at calltrap+0x8 > #6 0xffffffff81f2c170 at fdisp_make+0xe4 > #7 0xffffffff81f3033d at fuse_lookup+0x1dc > #8 0xffffffff808caac8 at VOP_LOOKUP_APV+0x4c > #9 0xffffffff8061b0db at lookup+0x61e > #10 0xffffffff8061bf1f at namei+0x592 > #11 0xffffffff80629df4 at kern_statat_vnhook+0x87 > #12 0xffffffff80629f98 at kern_statat+0x15 > #13 0xffffffff80629fb6 at kern_lstat+0x1c > #14 0xffffffff8062a03d at lstat+0x25 > #15 0xffffffff805d7cf4 at syscallenter+0x2d9 > #16 0xffffffff80875d45 at syscall+0x38 > #17 0xffffffff8085f662 at Xfast_syscall+0xe2 >=20 >=20 > #0 doadump () at pcpu.h:224 > #1 0xffffffff8059cd12 in boot (howto=3D260) at > /usr/src/sys/kern/kern_shutdown.c:419 > #2 0xffffffff8059d03b in panic (fmt=3DVariable "fmt" is not = available.) > at /usr/src/sys/kern/kern_shutdown.c:592 > #3 0xffffffff80875a75 in trap_fatal (frame=3D0xffffff813aa2f4c0, = eva=3D64) > at /usr/src/sys/amd64/amd64/trap.c:809 > #4 0xffffffff80875ce5 in trap_pfault (frame=3D0xffffff813aa2f4c0, > usermode=3D0) at /usr/src/sys/amd64/amd64/trap.c:725 > #5 0xffffffff808761b5 in trap (frame=3D0xffffff813aa2f4c0) at > /usr/src/sys/amd64/amd64/trap.c:475 > #6 0xffffffff8085f384 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:228 > #7 0xffffffff81f2ba3c in fiov_adjust () from = /usr/local/modules/fuse.ko > #8 0xffffffff81f2c170 in fdisp_make () from = /usr/local/modules/fuse.ko > #9 0xffffffff81f3033d in fuse_lookup () from = /usr/local/modules/fuse.ko > #10 0xffffffff808caac8 in VOP_LOOKUP_APV (vop=3D0x0, a=3D0x30) at = vnode_if.c:123 > #11 0xffffffff8061b0db in lookup (ndp=3D0x618) at vnode_if.h:54 > #12 0xffffffff8061bf1f in namei (ndp=3D0xffffff813aa2f8e0) at > /usr/src/sys/kern/vfs_lookup.c:269 > #13 0xffffffff80629df4 in kern_statat_vnhook (td=3D0xffffff00039f2460, > flag=3DVariable "flag" is not available.) at > /usr/src/sys/kern/vfs_syscalls.c:2346 > #14 0xffffffff80629f98 in kern_statat (td=3DVariable "td" is not > available.) at /usr/src/sys/kern/vfs_syscalls.c:2327 > #15 0xffffffff80629fb6 in kern_lstat (td=3DVariable "td" is not > available.) at /usr/src/sys/kern/vfs_syscalls.c:2400 > #16 0xffffffff8062a03d in lstat (td=3DVariable "td" is not available.) = at > /usr/src/sys/kern/vfs_syscalls.c:2390 > #17 0xffffffff805d7cf4 in syscallenter (td=3D0xffffff00039f2460, > sa=3D0xffffff813aa2fbc0) at /usr/src/sys/kern/subr_trap.c:315 > #18 0xffffffff80875d45 in syscall (frame=3D0xffffff813aa2fc50) at > /usr/src/sys/amd64/amd64/trap.c:914 > #19 0xffffffff8085f662 in Xfast_syscall () at > /usr/src/sys/amd64/amd64/exception.S:381 > #20 0x000000080097a5dc in ?? () >=20 > At #10 a=3D0x30 looks dodgy and fault address is 0x40, but I have no = idea > what that means... >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 21:41:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 923D31065672 for ; Thu, 29 Dec 2011 21:41:02 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.westchester.pa.mail.comcast.net (qmta14.westchester.pa.mail.comcast.net [76.96.59.212]) by mx1.freebsd.org (Postfix) with ESMTP id 3E5F68FC0A for ; Thu, 29 Dec 2011 21:41:01 +0000 (UTC) Received: from omta05.westchester.pa.mail.comcast.net ([76.96.62.43]) by qmta14.westchester.pa.mail.comcast.net with comcast id F9gN1i0040vyq2s5E9h20Z; Thu, 29 Dec 2011 21:41:02 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta05.westchester.pa.mail.comcast.net with comcast id F9h11i00Z1t3BNj3R9h1W0; Thu, 29 Dec 2011 21:41:02 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DF09A102C1E; Thu, 29 Dec 2011 13:40:59 -0800 (PST) Date: Thu, 29 Dec 2011 13:40:59 -0800 From: Jeremy Chadwick To: UFS User Message-ID: <20111229214059.GA4452@icarus.home.lan> References: <1325190659.66676.YahooMailNeo@web140408.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1325190659.66676.YahooMailNeo@web140408.mail.bf1.yahoo.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" Subject: Re: Painted into a corner with i386 system and big FS ... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 21:41:02 -0000 On Thu, Dec 29, 2011 at 12:30:59PM -0800, UFS User wrote: > I have an older i386 system with an 8TB filesystem on it, roughly 40 million inodes. I'm sorry to hear that. Your Subject line is more or less correct here; you did this to yourself. Don't do that! > It appears that this is impossible to fsck - I have nudged up maxdsiz all the way to: > kern.maxdsiz="3072000000" > > and it's not enough. ?Any larger values cause the system to become unbootable. > So, my plan is to boot 8.x amd64 FIXIT, and run fsck from the fixit environment. > The issues I foresee are: > > 1. Do I need to specify a maxdsiz in the 8.x fixit environment, or does this all just autoscale > these days ? It does not autoscale. You will need to set it manually at the loader prompt by entering 'set kern.maxdsiz="3072000000"' then 'boot'. > I'm hoping my process doesn't grow past 3.5 GB, but do I need to burn a > custom fixit to do that ? ?Or can I drop into the boot loader of FIXIT and set a custom load > value ? See above. > Or perhaps I don't care at all and maxdsiz will just auto-scale to however large the fsck gets ? Unknown; you will need to try, or someone else here can chime in with a more accurate answer. > 2. There is only 4 GB of physical in the system ... if I could grow to 3.072 GB process size in > i386, then all else being equal, I should be able to grow to 3.5 in 8.x FIXIT, right ? The 3.072GB value you came up with is related to i386; I can point you to documentation that explains where the remaining 1GB goes (you can thank x86 architecture for that). On some systems it's as low as 2.8GB, as it depends greatly on northbridge/MCH and devices on the system. The 3.5GB value you came up with is equally as amusing. How much you actually get greatly varies on hardware details and so on. Your motherboard manufacturer should be able to tell you how much you'll get, as long as you provide them a full list of any PCI/PCI-X/PCIe devices you have installed in slots, and any system features you have enabled (AHCI, on-board SCSI, etc.). > 3. If I get larger than 4 GB in amd64, with a 4GB physical memory, I can always swap, right ? Yes. Prepare for the system to be extremely slow given massive I/O, however. You should be able to add swap to the system in real-time by executing "swapon /dev/something" (e.g. swapon /dev/ada0s1b). > But where would I swap to ? Your swap partition, assuming you have one. If not, you will need to create one, or use a physically separate disk to act as swap. Please do not do something like "use a USB memory stick" (the I/O speed will be pathetically slow), or "use mdconfig" (you will shoot yourself in the foot doing this). > Maybe I could mount my /var (which is clean, and is 4 GB in size) > and swap to there ? How would you do that? /var is a UFS/UFS2 filesystem; swap is a dedicated slice or partition. > Can I specify a file-backed swapfile in FIXIT ? man swapon(8). The answer is no, AFAIK, and "fixit" has nothing to do with it. There is nothing different about "fixit" compared to a standard FreeBSD system/installation. My recommendation: get yourself access to an amd64 system that has a large amount of memory on it (16-32GB if possible) and can be used for this task, physically attach the disk to that system, and fsck it. Once you're done, do something about the mess on your filesystem. Upgrading to amd64 won't solve the problem either -- you've simply abused the filesystem to the point where you're hurting from the abuse. ZFS would handle this situation better, but I am in no way shape or form going to tell you "go with ZFS going forward" because there are many other nuances involved with that. So let's not discuss that here. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 22:06:40 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E3B9106566B for ; Thu, 29 Dec 2011 22:06:40 +0000 (UTC) (envelope-from dpd@bitgravity.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3BFD98FC14 for ; Thu, 29 Dec 2011 22:06:39 +0000 (UTC) Received: by iadj38 with SMTP id j38so30655794iad.13 for ; Thu, 29 Dec 2011 14:06:39 -0800 (PST) Received: by 10.42.151.65 with SMTP id d1mr38020906icw.18.1325196398262; Thu, 29 Dec 2011 14:06:38 -0800 (PST) Received: from netops-69.sfo1.bitgravity.com ([209.131.110.69]) by mx.google.com with ESMTPS id nw8sm60476958igc.7.2011.12.29.14.06.37 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 29 Dec 2011 14:06:37 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: David P Discher In-Reply-To: <20111229215125.GA8539@reks> Date: Thu, 29 Dec 2011 14:06:36 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87E5C7D5-2D73-4A42-B871-8B27448B0DD9@bitgravity.com> <20111229215125.GA8539@reks> To: Gleb Kurtsou X-Mailer: Apple Mail (2.1084) Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: fusefs broken on 8-stable? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 22:06:40 -0000 Is this a straight dump for p4 ? =20 Earlier this week, I checked out //depot/projects/soc2011/ilya_fuse/... = out of p4. But only used the kmod, not the libs. Unfortunately, it was = 10x slower than the 0.3.9 version currently in the ports tree. I never = got it to the same point ... after a few days, had only written about = 6-10GB of data. =20 My panic happens after about 500-600GB of data has been written via the = fuse.=20 However, my testing is easy (except for the waiting) ... I'll gladly = give it a try.=20 --- David P. Discher dpd@bitgravity.com * AIM: bgDavidDPD BITGRAVITY * http://www.bitgravity.com On Dec 29, 2011, at 1:51 PM, Gleb Kurtsou wrote: >=20 > There is alternative fuse port at = https://github.com/glk/fuse-freebsd-ports >=20 > Could you try it and let us know if it works. From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 22:14:00 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07FE3106566C for ; Thu, 29 Dec 2011 22:14:00 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7C8258FC08 for ; Thu, 29 Dec 2011 22:13:59 +0000 (UTC) Received: by eekc50 with SMTP id c50so16032145eek.13 for ; Thu, 29 Dec 2011 14:13:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=V9XWH8q0jiaE10QbAYEu94nQDZ5BleK/2xjSv70QrQQ=; b=dU6XOXJsOs59+FG88+GUPyExzfo8evnTuPOs/69ic+Zjz7MN+o47HkKOoXBXGEAbeA nnOxP0gS3TCW7MdLOJ95S7wwFAJ+Yhcg8Q5Z1EK+D1u6f7LTZ+YDpr0PoIhlhOg8dyOP XAo2VnPAqsNTI87NYXzZVoXx0o1cz+2rKV1vo= Received: by 10.213.31.207 with SMTP id z15mr2943195ebc.144.1325195477105; Thu, 29 Dec 2011 13:51:17 -0800 (PST) Received: from localhost ([78.157.92.5]) by mx.google.com with ESMTPS id b49sm108230968eec.9.2011.12.29.13.51.15 (version=SSLv3 cipher=OTHER); Thu, 29 Dec 2011 13:51:16 -0800 (PST) Date: Thu, 29 Dec 2011 23:51:25 +0200 From: Gleb Kurtsou To: David P Discher Message-ID: <20111229215125.GA8539@reks> References: <87E5C7D5-2D73-4A42-B871-8B27448B0DD9@bitgravity.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87E5C7D5-2D73-4A42-B871-8B27448B0DD9@bitgravity.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: fusefs broken on 8-stable? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 22:14:00 -0000 On (29/12/2011 13:10), David P Discher wrote: > Johannes - Were you able to get a resolution for this ? > > I've been able to replicate this panic in fiov_adjust(), but being called from vnlru, looks like its doing VOP_RECLAIM_APV(). > > I'm willing to dive deeper and debug this, however would appreciate some guidance form the community. I'm having issues wrapping my head around macro FUSE_DIMALLOC(), which I think is using some sort of anonymous function call to get to fiov_adjust(). I've been battling this for a week or two now, so I have symbols in fuse module, so I have a bit more info. There is alternative fuse port at https://github.com/glk/fuse-freebsd-ports Could you try it and let us know if it works. > > ======================================== > > (kgdb) up > #11 0xffffffff81147412 in fdisp_make_pid (fdip=0xffffff9bce6399f0, mp=Variable "mp" is not available. > ) > at fuse_msg.c:788 > 788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, fdip->finh, > (kgdb) l > 783 if (fdip->tick) > 784 fticket_refresh(fdip->tick); > 785 else > 786 fdip->tick = fuse_ticket_fetch(data); > 787 > 788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, fdip->finh, > 789 fdip->indata, fdip->iosize); > 790 fuse_setup_ihead(fdip->finh, fdip->tick, nid, op, fdip->iosize, pid, cred); > 791 } > 792 > > (kgdb) p fdip > $4 = (struct fuse_dispatcher *) 0xffffff9bce6399f0 > (kgdb) p *(struct fuse_dispatcher *)fdip > $5 = {tick = 0x0, finh = 0x0, indata = 0xffffff9bce639a30, iosize = 8, > nodeid = 0, answ_stat = 1356898584, answ = 0xffffff0350e0a138} > (kgdb) p *(struct fuse_dispatcher *)fdip->tick > Cannot access memory at address 0x0 > > ========================================== > > (kgdb) up > #9 0xffffffff807bd7b8 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:228 > 228 call trap > Current language: auto; currently asm > (kgdb) up > #10 0xffffffff81146b17 in fiov_adjust (fiov=0x30, size=48) at fuse_msg.c:96 > 96 { > Current language: auto; currently c > (kgdb) l > 91 free(fiov->base, M_FUSEMSG); > 92 } > 93 > 94 void > 95 fiov_adjust(struct fuse_iov *fiov, size_t size) > 96 { > 97 if (fiov->allocated_size < size || > 98 (fuse_iov_permanent_bufsize >= 0 && > 99 fiov->allocated_size - size > fuse_iov_permanent_bufsize && > 100 --fiov->credit < 0)) { > > > (kgdb) p fiov > $1 = (struct fuse_iov *) 0x30 > (kgdb) p (struct fuse_iov *)fiov > $2 = (struct fuse_iov *) 0x30 > (kgdb) p *(struct fuse_iov *)fiov > Cannot access memory at address 0x30 > (kgdb) p size > $3 = 48 > > ========================================== > > (kgdb) bt > #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:263 > #1 0xffffffff801f6c9c in db_fncall (dummy1=Variable "dummy1" is not available. > ) > at /usr/src/sys/ddb/db_command.c:548 > #2 0xffffffff801f6fd1 in db_command (last_cmdp=0xffffffff80b67a40, cmd_table=Variable "cmd_table" is not available. > ) > at /usr/src/sys/ddb/db_command.c:445 > #3 0xffffffff801f7220 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:498 > #4 0xffffffff801f9269 in db_trap (type=Variable "type" is not available. > ) at /usr/src/sys/ddb/db_main.c:229 > #5 0xffffffff80575ff1 in kdb_trap (type=12, code=0, tf=0xffffff9bce6398a0) > at /usr/src/sys/kern/subr_kdb.c:548 > #6 0xffffffff807d4bad in trap_fatal (frame=0xffffff9bce6398a0, eva=Variable "eva" is not available. > ) > at /usr/src/sys/amd64/amd64/trap.c:820 > #7 0xffffffff807d4eea in trap_pfault (frame=0xffffff9bce6398a0, usermode=0) > at /usr/src/sys/amd64/amd64/trap.c:741 > #8 0xffffffff807d52ff in trap (frame=0xffffff9bce6398a0) > at /usr/src/sys/amd64/amd64/trap.c:478 > #9 0xffffffff807bd7b8 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:228 > #10 0xffffffff81146b17 in fiov_adjust (fiov=0x30, size=48) at fuse_msg.c:96 > #11 0xffffffff81147412 in fdisp_make_pid (fdip=0xffffff9bce6399f0, mp=Variable "mp" is not available. > ) > at fuse_msg.c:788 > #12 0xffffffff8114a6d4 in fuse_send_forget (mp=Variable "mp" is not available. > ) at fuse_vnops.c:378 > #13 0xffffffff8114a7d6 in fuse_recyc_backend (vp=0xffffff0350e0a000, > td=0xffffff0029533000) at fuse_vnops.c:539 > #14 0xffffffff808272fa in VOP_RECLAIM_APV (vop=0xffffffff811504e0, > a=0xffffff9bce639a90) at vnode_if.c:1926 > #15 0xffffffff805ccf95 in vgonel (vp=0xffffff0350e0a000) at vnode_if.h:830 > #16 0xffffffff805cf9c4 in vnlru_free (count=28) > at /usr/src/sys/kern/vfs_subr.c:975 > #17 0xffffffff805d030f in vnlru_proc () at /usr/src/sys/kern/vfs_subr.c:834 > #18 0xffffffff805211bd in fork_exit (callout=0xffffffff805cfec0 , > arg=0x0, frame=0xffffff9bce639c50) at /usr/src/sys/kern/kern_fork.c:876 > #19 0xffffffff807bdcfe in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:602 > > ========================================== > > > > Fatal trap 12: page fault while in kernel mode^M > cpuid = 7; apic id = 15^M > fault virtual address = 0x40^M > fault code = supervisor read data, page not present^M > instruction pointer = 0x20:0xffffffff81146b17^M > stack pointer = 0x28:0xffffff9bce639950^M > frame pointer = 0x28:0xffffff9bce639980^M > code segment = base 0x0, limit 0xfffff, type 0x1b^M > = DPL 0, pres 1, long 1, def32 0, gran 1^M > processor eflags = interrupt enabled, resume, IOPL = 0^M > current process = 18 (vnlru)^M > [thread pid 18 tid 100138 ] > Stopped at fiov_adjust+0x17: movq 0x10(%rdi),%rax > db> [-- MARK -- Thu Dec 29 01:15:00 2011] > bt > Tracing pid 18 tid 100138 td 0xffffff0029533000 > fiov_adjust() at fiov_adjust+0x17 > fdisp_make_pid() at fdisp_make_pid+0xd2 > fuse_send_forget() at fuse_send_forget+0x44 > fuse_recyc_backend() at fuse_recyc_backend+0xc6 > VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0x4a > vgonel() at vgonel+0xf5 > vnlru_free() at vnlru_free+0x294 > vnlru_proc() at vnlru_proc+0x44f > fork_exit() at fork_exit+0x11d > fork_trampoline() at fork_trampoline+0xe > --- trap 0, rip = 0, rsp = 0xffffff9bce639d00, rbp = 0 --- > db> call doadump > > > > --- > David P. Discher > dpd@bitgravity.com * AIM: bgDavidDPD > BITGRAVITY * http://www.bitgravity.com > > On Jun 3, 2011, at 6:49 PM, Johannes Totz wrote: > > > On 13/05/2011 13:56, Johannes Totz wrote: > >> Heya! > >> > >> Using encfs (built on top of fuse) gives me panics in combination with > >> rsync. Dump didn't succeed. The info below is transcribbled from a > >> photograph. This is repeatable. > >> Without dump this is probably not very helpful.... > >> > >> > >> # uname -a > >> FreeBSD XXX 8.2-STABLE FreeBSD 8.2-STABLE #1: Thu Mar 10 23:30:08 GMT > >> 2011 root@XXX:/usr/obj/usr/src/sys/GENERIC amd64 > > > > Managed to get a dump: > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 1; apic id = 01 > > fault virtual address = 0x40 > > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff81f2ba3c > > stack pointer = 0x28:0xffffff813aa2f570 > > frame pointer = 0x28:0xffffff813aa2f5a0 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = interrupt enabled, resume, IOPL = 0 > > current process = 1560 (rsync) > > trap number = 12 > > panic: page fault > > cpuid = 1 > > KDB: stack backtrace: > > #0 0xffffffff805cc226 at kdb_backtrace+0x5c > > #1 0xffffffff8059d000 at panic+0x1b4 > > #2 0xffffffff80875a75 at trap_fatal+0x394 > > #3 0xffffffff80875ce5 at trap_pfault+0x252 > > #4 0xffffffff808761b5 at trap+0x3f4 > > #5 0xffffffff8085f384 at calltrap+0x8 > > #6 0xffffffff81f2c170 at fdisp_make+0xe4 > > #7 0xffffffff81f3033d at fuse_lookup+0x1dc > > #8 0xffffffff808caac8 at VOP_LOOKUP_APV+0x4c > > #9 0xffffffff8061b0db at lookup+0x61e > > #10 0xffffffff8061bf1f at namei+0x592 > > #11 0xffffffff80629df4 at kern_statat_vnhook+0x87 > > #12 0xffffffff80629f98 at kern_statat+0x15 > > #13 0xffffffff80629fb6 at kern_lstat+0x1c > > #14 0xffffffff8062a03d at lstat+0x25 > > #15 0xffffffff805d7cf4 at syscallenter+0x2d9 > > #16 0xffffffff80875d45 at syscall+0x38 > > #17 0xffffffff8085f662 at Xfast_syscall+0xe2 > > > > > > #0 doadump () at pcpu.h:224 > > #1 0xffffffff8059cd12 in boot (howto=260) at > > /usr/src/sys/kern/kern_shutdown.c:419 > > #2 0xffffffff8059d03b in panic (fmt=Variable "fmt" is not available.) > > at /usr/src/sys/kern/kern_shutdown.c:592 > > #3 0xffffffff80875a75 in trap_fatal (frame=0xffffff813aa2f4c0, eva=64) > > at /usr/src/sys/amd64/amd64/trap.c:809 > > #4 0xffffffff80875ce5 in trap_pfault (frame=0xffffff813aa2f4c0, > > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:725 > > #5 0xffffffff808761b5 in trap (frame=0xffffff813aa2f4c0) at > > /usr/src/sys/amd64/amd64/trap.c:475 > > #6 0xffffffff8085f384 in calltrap () at > > /usr/src/sys/amd64/amd64/exception.S:228 > > #7 0xffffffff81f2ba3c in fiov_adjust () from /usr/local/modules/fuse.ko > > #8 0xffffffff81f2c170 in fdisp_make () from /usr/local/modules/fuse.ko > > #9 0xffffffff81f3033d in fuse_lookup () from /usr/local/modules/fuse.ko > > #10 0xffffffff808caac8 in VOP_LOOKUP_APV (vop=0x0, a=0x30) at vnode_if.c:123 > > #11 0xffffffff8061b0db in lookup (ndp=0x618) at vnode_if.h:54 > > #12 0xffffffff8061bf1f in namei (ndp=0xffffff813aa2f8e0) at > > /usr/src/sys/kern/vfs_lookup.c:269 > > #13 0xffffffff80629df4 in kern_statat_vnhook (td=0xffffff00039f2460, > > flag=Variable "flag" is not available.) at > > /usr/src/sys/kern/vfs_syscalls.c:2346 > > #14 0xffffffff80629f98 in kern_statat (td=Variable "td" is not > > available.) at /usr/src/sys/kern/vfs_syscalls.c:2327 > > #15 0xffffffff80629fb6 in kern_lstat (td=Variable "td" is not > > available.) at /usr/src/sys/kern/vfs_syscalls.c:2400 > > #16 0xffffffff8062a03d in lstat (td=Variable "td" is not available.) at > > /usr/src/sys/kern/vfs_syscalls.c:2390 > > #17 0xffffffff805d7cf4 in syscallenter (td=0xffffff00039f2460, > > sa=0xffffff813aa2fbc0) at /usr/src/sys/kern/subr_trap.c:315 > > #18 0xffffffff80875d45 in syscall (frame=0xffffff813aa2fc50) at > > /usr/src/sys/amd64/amd64/trap.c:914 > > #19 0xffffffff8085f662 in Xfast_syscall () at > > /usr/src/sys/amd64/amd64/exception.S:381 > > #20 0x000000080097a5dc in ?? () > > > > At #10 a=0x30 looks dodgy and fault address is 0x40, but I have no idea > > what that means... > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 22:19:50 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B35BF1065670 for ; Thu, 29 Dec 2011 22:19:50 +0000 (UTC) (envelope-from ufs.user@yahoo.com) Received: from nm20.bullet.mail.ac4.yahoo.com (nm20.bullet.mail.ac4.yahoo.com [98.139.52.217]) by mx1.freebsd.org (Postfix) with SMTP id 661A58FC08 for ; Thu, 29 Dec 2011 22:19:50 +0000 (UTC) Received: from [98.139.52.188] by nm20.bullet.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 22:19:49 -0000 Received: from [98.139.52.165] by tm1.bullet.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 22:19:49 -0000 Received: from [127.0.0.1] by omp1048.mail.ac4.yahoo.com with NNFMP; 29 Dec 2011 22:19:49 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 781122.17474.bm@omp1048.mail.ac4.yahoo.com Received: (qmail 55563 invoked by uid 60001); 29 Dec 2011 22:19:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1325197189; bh=Ufiar4Gpr3bSzWhedowq10tKxK0YMb3gA5g5IQ5mV+g=; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=HiTGOrtaDBzRVEdJXnK5c5KEeXjFRM1x8B0G9pYK7WJxviUSvfJCOJ6IxKkwmAdAmhlaoAhbyLGUrtWb7Z6g2+zELIjkZ8HFDkX669s0bS7C6r0gimDuwAqmVCK5TnMpK+qil+C4UpFzl+9sOFoFcREyMSvcn3+85nRpC3A41NM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=jpkcLFxW1gK+DHfWkipHa5mM3MoIn0lw5OPp+mePm+KeUqku0pSFBvwCdWNHAMGM0JWRY+5LW6Z8e2N73MJzVLrt1KmvdIQZI24IKwE5yJc9QLtz1Af+WEoHhIFBxAME3X0bBnyjSRxRMJWV8dN1g1OLnb32iicXf3rh0GW6G8A=; X-YMail-OSG: AzpA9tcVM1nGnLc3UKQ6YdQj4fcqaM5CYYS9g.aGjOuA9ss _cOjP9J2Jj_0C0Yay1llRD9p6Pv0gJWf5oEHv7dmFX6ASXjNsDacBeCMZF8F xVgxX3tMZAQc28V1Gx_lBWyKIOr4D5Qtw9d3ubsH8n_1ZnsHy9HmMVMEtYyi Q2XGM7Mm3xf0770VBVdJFk.0aBZ5fkJLZN5K1OhcwaZIIgHtHSjXfNBkbkHz SiQdV0AU1S2kksCAY4mVygvzJ1gMpKNlrxomV7lV57EJ0LN.B7.FYdZMYMgS 4BWWO2mmrlLmrCn9A3_uVUyEm5pKuZuIZGkxF_H4Ay_kmqbOpSoqo0owRxKp uo61j2hSOlRV27h5m41FiOqfWGs2yONL5WLpumwP9HawdsedZop3EBfXvkPU H1b_n Received: from [24.179.213.102] by web140413.mail.bf1.yahoo.com via HTTP; Thu, 29 Dec 2011 14:19:49 PST X-Mailer: YahooMailWebService/0.8.115.331698 References: <1325190659.66676.YahooMailNeo@web140408.mail.bf1.yahoo.com> <20111229214059.GA4452@icarus.home.lan> <1325197128.44125.YahooMailNeo@web140420.mail.bf1.yahoo.com> Message-ID: <1325197189.54564.YahooMailNeo@web140413.mail.bf1.yahoo.com> Date: Thu, 29 Dec 2011 14:19:49 -0800 (PST) From: UFS User To: "freebsd-fs@freebsd.org" In-Reply-To: <1325197128.44125.YahooMailNeo@web140420.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: RE: Painted into a corner with i386 system and big FS ... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: UFS User List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 22:19:50 -0000 =0A=0AThanks for this response - see comments below...=0A=0A=0A=0A=0A> It a= ppears that this is impossible to fsck - I have nudged up maxdsiz all the w= ay to:=0A> kern.maxdsiz=3D"3072000000"=0A> =0A> and it's not enough. ?Any l= arger values cause the system to become unbootable.=0A> So, my plan is to b= oot 8.x amd64 FIXIT, and run fsck from the fixit environment.=0A> The issue= s I foresee are:=0A> =0A> 1. Do I need to specify a maxdsiz in the 8.x fixi= t environment, or does this all just autoscale=0A> these days ?=0A=0A=0A> I= t does not autoscale.=A0 You will need to set it manually at the loader=0A>= prompt by entering 'set kern.maxdsiz=3D"3072000000"' then 'boot'.=0A=0A=0A= =0AOk, great. =A0And it's the same old process:=0A=0AType '?' for a list of= commands, 'help' for more detailed help.=0AOK set=A0kern.maxdsiz=3D"357200= 0000"=0AOK boot=0A=0AYes ?=0A=0A=0A> 2. There is only 4 GB of physical in t= he system ... if I could grow to 3.072 GB process size in=0A> i386, then al= l else being equal, I should be able to grow to 3.5 in 8.x FIXIT, right ?= =0A=0A=0A(snip)=0A=0A=0A>The 3.5GB value you came up with is equally as amu= sing.=A0 How much you=0A>actually get greatly varies on hardware details an= d so on.=A0 Your=0A>motherboard manufacturer should be able to tell you how= much you'll get,=0A>as long as you provide them a full list of any PCI/PCI= -X/PCIe devices=0A>you have installed in slots, and any system features you= have enabled=0A>(AHCI, on-board SCSI, etc.).=0A=0A=0A=0AOh, sorry for the = confusion - the installed OS is freebsd i386, BUT the underlying system is = in fact a 64bit system.=0A=0AIt is the physical inaccessibility, and other = logistical issues that prevent us from just upgrading the entire OS, etc.= =0A=0ASo I will boot amd64 FreeBSD 8.x FIXIT, and will be running in a prop= er amd64 architecture, but the limitation is I=0Aonly have 4 GB physical ra= m.=0A=0ASo I am hoping that, all else being equal, if i386 can only see (fr= om dmesg):=0A=0Areal memory =A0=3D 3489071104 (3327 MB)=0Aavail memory =3D = 3414245376 (3256 MB)=0A=0Athen an amd64 system would see (roughly) 500 mega= bytes more memory than that, and the 3.072 GB process size I get on i386=0A= will then be (roughly) 3.572 on amd64.=0A=0AA reasonable guess ?=0A=0AI hop= e I don't have to swap, but if I do, presumably this will only grow to 3.8 = or 4.0 or 4.2 GB in process size, so that's just the last 1/8 of the job=0A= that will be slow...=0A=0AThanks again.=0A From owner-freebsd-fs@FreeBSD.ORG Thu Dec 29 23:39:29 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3102A106564A for ; Thu, 29 Dec 2011 23:39:29 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net [76.96.62.48]) by mx1.freebsd.org (Postfix) with ESMTP id CDE038FC15 for ; Thu, 29 Dec 2011 23:39:28 +0000 (UTC) Received: from omta15.westchester.pa.mail.comcast.net ([76.96.62.87]) by qmta05.westchester.pa.mail.comcast.net with comcast id FBTf1i0011swQuc55BfVS8; Thu, 29 Dec 2011 23:39:29 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta15.westchester.pa.mail.comcast.net with comcast id FBfU1i0021t3BNj3bBfUge; Thu, 29 Dec 2011 23:39:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B7773102C1E; Thu, 29 Dec 2011 15:39:26 -0800 (PST) Date: Thu, 29 Dec 2011 15:39:26 -0800 From: Jeremy Chadwick To: UFS User Message-ID: <20111229233926.GA6315@icarus.home.lan> References: <1325190659.66676.YahooMailNeo@web140408.mail.bf1.yahoo.com> <20111229214059.GA4452@icarus.home.lan> <1325197128.44125.YahooMailNeo@web140420.mail.bf1.yahoo.com> <1325197189.54564.YahooMailNeo@web140413.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1325197189.54564.YahooMailNeo@web140413.mail.bf1.yahoo.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" Subject: Re: Painted into a corner with i386 system and big FS ... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 23:39:29 -0000 On Thu, Dec 29, 2011 at 02:19:49PM -0800, UFS User wrote: > > It appears that this is impossible to fsck - I have nudged up maxdsiz all the way to: > > kern.maxdsiz="3072000000" > > > > and it's not enough. ?Any larger values cause the system to become unbootable. > > So, my plan is to boot 8.x amd64 FIXIT, and run fsck from the fixit environment. > > The issues I foresee are: > > > > 1. Do I need to specify a maxdsiz in the 8.x fixit environment, or does this all just autoscale > > these days ? > > > It does not autoscale.? You will need to set it manually at the loader > > prompt by entering 'set kern.maxdsiz="3072000000"' then 'boot'. > > Ok, great. ?And it's the same old process: > > Type '?' for a list of commands, 'help' for more detailed help. > OK set?kern.maxdsiz="3572000000" > OK boot > > Yes ? Correct. > > 2. There is only 4 GB of physical in the system ... if I could grow to 3.072 GB process size in > > i386, then all else being equal, I should be able to grow to 3.5 in 8.x FIXIT, right ? > > >The 3.5GB value you came up with is equally as amusing.? How much you > >actually get greatly varies on hardware details and so on.? Your > >motherboard manufacturer should be able to tell you how much you'll get, > >as long as you provide them a full list of any PCI/PCI-X/PCIe devices > >you have installed in slots, and any system features you have enabled > >(AHCI, on-board SCSI, etc.). > > Oh, sorry for the confusion - the installed OS is freebsd i386, BUT the underlying system is in fact a 64bit system. > It is the physical inaccessibility, and other logistical issues that prevent us from just upgrading the entire OS, etc. In this specific case it doesn't matter if the actual hardware supports 64-bit or not. The OS is i386, which means you're limited to 32-bit (4GB) addressing space. As such, devices which want memory-mapped regions are taking up some addressing space which could be used for physical RAM, but this would only work with a 64-bit OS. You know this already. My point is that the underlying hardware supporting 64-bit doesn't matter. Using i386 with PAE is your only other option, and I don't advise going that route. There are device drivers and other "catches" with PAE; amd64 is supported, while I'm not so sure PAE is. > So I will boot amd64 FreeBSD 8.x FIXIT, and will be running in a proper amd64 architecture, but the limitation is I > only have 4 GB physical ram. I understand. > So I am hoping that, all else being equal, if i386 can only see (from dmesg): > > real memory ?= 3489071104 (3327 MB) > avail memory = 3414245376 (3256 MB) > > then an amd64 system would see (roughly) 500 megabytes more memory than that, and the 3.072 GB process size I get on i386 > will then be (roughly) 3.572 on amd64. > > A reasonable guess ? I'll repeat myself: not exactly. This "500MByte" value you came up with is purely random. The amount that will become available will be decided based upon what sorts of devices are on your system, whether or not they actually need physical RAM as part of their addressing space (or if they just want addressing space for internal use), etc.. There are too many factors to consider. Read the following manual for a Supermicro X7SBA motherboard, specifically section 2-4 (see chart labelled "Possible System Memory Allocation & Availability"): http://www.supermicro.com/manuals/motherboard/3210/MNL-0976.pdf Some of what's shown there applies to i386 (32-bit) only, but ANY of those devices or features can require actual physical RAM (for example, a video card that has no on-board RAM and thus uses system RAM). AHCI, option ROMs, and other things all "take up" physical RAM regardless if they're on amd64 or i386. Understand? So, the "500MByte" value you're assuming is in no way accurate. The easiest way to find out what you'll get? Boot amd64! :-) For "tuning kern.maxdsiz", keep reading. > I hope I don't have to swap, but if I do, presumably this will only grow to 3.8 or 4.0 or 4.2 GB in process size, so that's just the last 1/8 of the job > that will be slow... Again, alternately, I would recommend you take the disk out and stick it in an amd64 system that has more memory (16-32GB would be good) and has a dedicated swap partition, and fsck from there. It's better to be safe than sorry. Here's a better way of looking at it: Do you really want to try and "wing it" with no swap, only to have the system panic, resulting in a potentially even more corrupt filesystem (fsck does make changes to the filesystem, so what happens if the kernel panics in the middle of that? :-), or would you rather fsck the filesystem on a system where you know you're not going to encounter memory exhaustion and risk making things worse? If you *really* must do this on the same system, then add a physical disk to the system which you can use temporarily for a swap partition, boot an 8.2-STABLE amd64 memstick / fixit environment, set up the disk for swap, then swapon it. After that, fsck the other disk. As I stated, I/O will be slow if swap is used, but at least you won't risk fsck segfaulting/dying off, or a kernel panic. Also, you might be wondering if kern.maxdsiz would include swap use or not; I imagine that it would. So in your case, you may need to set the loader variable to an extremely large value (many gigabytes) in the case that the fsck process ultimately requires lots of memory (physical + swap combined). There's no real way to calculate this value ahead of time, so I really can't tell you what to pick. The situation you are in isn't easily resolved. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri Dec 30 09:02:49 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80B28106564A for ; Fri, 30 Dec 2011 09:02:49 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id EFA728FC12 for ; Fri, 30 Dec 2011 09:02:48 +0000 (UTC) Received: by lahl5 with SMTP id l5so7489201lah.13 for ; Fri, 30 Dec 2011 01:02:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=zSWxQ3gcc7mPyYO+UWHkj4ma0SPwW/iHWMWyChRAOy4=; b=hdJsdJ9qzAPsJv9yANv0enW02006rDNC31LSDo47K7Tlqu+VqT1qDgqOIkXtXJi+Oc sB4L+C5FCdu8WljQLffbijMvSFdCyRCASlp0EKiAfepwzgW1Y0bSgCzJ+hPf6+9HUFBW FPgwR1fzIcijVY/jh27L4oY41hkENFelluzn0= Received: by 10.152.146.72 with SMTP id ta8mr19924097lab.51.1325235766052; Fri, 30 Dec 2011 01:02:46 -0800 (PST) Received: from localhost ([78.157.92.5]) by mx.google.com with ESMTPS id oy18sm6315819lab.3.2011.12.30.01.02.44 (version=SSLv3 cipher=OTHER); Fri, 30 Dec 2011 01:02:45 -0800 (PST) Date: Fri, 30 Dec 2011 11:02:56 +0200 From: Gleb Kurtsou To: David P Discher Message-ID: <20111230090256.GA6796@reks> References: <87E5C7D5-2D73-4A42-B871-8B27448B0DD9@bitgravity.com> <20111229215125.GA8539@reks> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: fusefs broken on 8-stable? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 09:02:49 -0000 On (29/12/2011 14:06), David P Discher wrote: > Is this a straight dump for p4 ? > > Earlier this week, I checked out //depot/projects/soc2011/ilya_fuse/... out of p4. But only used the kmod, not the libs. Unfortunately, it was 10x slower than the 0.3.9 version currently in the ports tree. I never got it to the same point ... after a few days, had only written about 6-10GB of data. It's the same as p4 (+ few minor tweaks). fuse-0.4 is more strict on resizing files which is very likely to be slow for encrypted file system (encfs). Try setting sysctl vfs.fuse.sync_resize=0. I don't know all the details but there was memory leak and arbitrary memory corruption in old fuse. It's believed to be fixed in fuse-0.4. My guess you are hitting the same bug. > My panic happens after about 500-600GB of data has been written via the fuse. > > However, my testing is easy (except for the waiting) ... I'll gladly give it a try. > > --- > David P. Discher > dpd@bitgravity.com * AIM: bgDavidDPD > BITGRAVITY * http://www.bitgravity.com > > On Dec 29, 2011, at 1:51 PM, Gleb Kurtsou wrote: > > > > > There is alternative fuse port at https://github.com/glk/fuse-freebsd-ports > > > > Could you try it and let us know if it works. > From owner-freebsd-fs@FreeBSD.ORG Sat Dec 31 07:18:04 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A41001065670 for ; Sat, 31 Dec 2011 07:18:04 +0000 (UTC) (envelope-from freebsd@deman.com) Received: from plato.corp.nas.com (plato.corp.nas.com [66.114.32.138]) by mx1.freebsd.org (Postfix) with ESMTP id 6879B8FC16 for ; Sat, 31 Dec 2011 07:18:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by plato.corp.nas.com (Postfix) with ESMTP id BE7651045D02C for ; Fri, 30 Dec 2011 22:58:11 -0800 (PST) X-Virus-Scanned: amavisd-new at corp.nas.com Received: from plato.corp.nas.com ([127.0.0.1]) by localhost (plato.corp.nas.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r5ielERA3ln0 for ; Fri, 30 Dec 2011 22:58:10 -0800 (PST) Received: from [192.168.2.247] (mono-sis1.s.bli.openaccess.org [66.114.32.149]) by plato.corp.nas.com (Postfix) with ESMTPSA id 042051045D01F for ; Fri, 30 Dec 2011 22:58:09 -0800 (PST) From: Michael DeMan Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 30 Dec 2011 22:58:08 -0800 Message-Id: <8EA721E0-977D-483C-AC06-1040B87E0AA7@deman.com> To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: zfs detach/replace X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 07:18:04 -0000 Hi All, The origination of the problem is entirely my fault on FreeBSD 8.1 = RELEASE #0. We had old notes that attempting a 'replace' (which is = appropriate for a mirror) leaves ZFS in a funky state on BSD. I = inadvertently did just that on a drive swap on a raidz2 pool. My old = notes show the only recovery that we knew of at the time was to rsync or = zfs-send the pool elsewhere, destroy the local and rebuild from scratch. Is there a better way to handle this nowadays? Thanks, - Mike DeMan # zpool status pool: zp1rz2 state: DEGRADED scrub: scrub in progress for 4h5m, 9.28% done, 39h55m to go config: NAME STATE READ WRITE CKSUM zp1rz2 DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 label/ada0LABEL ONLINE 0 0 0 label/ada1LABEL ONLINE 0 0 0 label/ada2LABEL ONLINE 0 0 0 label/ada3LABEL ONLINE 0 0 0 label/ada4LABEL ONLINE 0 0 0 replacing UNAVAIL 0 984K 0 = insufficient replicas label/ada5LABEL/old UNAVAIL 0 1.11M 0 cannot = open label/ada5LABEL UNAVAIL 0 1.11M 0 cannot = open label/ada6LABEL ONLINE 0 0 0 label/ada7LABEL ONLINE 0 0 0 errors: No known data errors From owner-freebsd-fs@FreeBSD.ORG Sat Dec 31 17:23:10 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 404CA106564A; Sat, 31 Dec 2011 17:23:10 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA078FC13; Sat, 31 Dec 2011 17:23:09 +0000 (UTC) Received: by iadj38 with SMTP id j38so34214050iad.13 for ; Sat, 31 Dec 2011 09:23:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=cBtIW3zWLn+J/CDCO5jqFH2FZLUNpNDuFBQH1tiZY68=; b=Vr23qaOdFvCKJiy00hEdTJK9GvO7E9kw5R5pB0uEe5C/2iASkArTdDP/XKM7bl0MHZ 50pO1rhSpBNAIULgq0jGq/so3G2omiCFL4zRqknsvlrWtdgjdA/U9NsIL2SvHfNn8Yo3 j6v9erjclEIBekZcFAskrPe08WpBc9StZUiwg= Received: by 10.50.45.195 with SMTP id p3mr50828397igm.2.1325352189231; Sat, 31 Dec 2011 09:23:09 -0800 (PST) Received: from DataIX.net (24-247-9-230.dhcp.aldl.mi.charter.com. [24.247.9.230]) by mx.google.com with ESMTPS id lu10sm71471926igc.0.2011.12.31.09.23.06 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 31 Dec 2011 09:23:07 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pBVHN48G006799 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 31 Dec 2011 12:23:04 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pBVHMuht006798; Sat, 31 Dec 2011 12:22:56 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 31 Dec 2011 12:22:56 -0500 From: Jason Hellenthal To: Artem Belevich Message-ID: <20111231172256.GA6434@DataIX.net> References: <20111225102719.GA44906@tolstoy.tols.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-fs@freebsd.org, Johannes Totz Subject: Re: Restoring received properties on a received filesystem. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 17:23:10 -0000 On Sun, Dec 25, 2011 at 04:10:43PM -0800, Artem Belevich wrote: > On Sun, Dec 25, 2011 at 2:44 PM, claudiu vasadi > wrote: > > Correct. This is to avoid, for instance, a zfs send/receive of a recursive > > (-r) snapshot of the ZFS boot pool. Since all properties (mountpoints too of > > course) are kept, upon reboot, the receive-ing side would mount the snapshot > > datasets over any existing (and same) datasets. > > > > Does it make sense ? > > I think I see why you may want it, but I don't think you'll get that > behavior from ZFS without hacking it. "zpool import -R root" is > probably your best bet. Alas altroot property is not persistent and > the pool would have to reimported on reboot. Another option may be to > backup/restore properties separately and ignore properties during > send/receive. > On the recieving server set 'canmount=noauto' on the whole pool. Use this pool for only recieving datasets from other machines and not snapshots of a whole pool. The canmount property should be inherited by any recieved dataset and thus thwart any ill filesystems from automagically being used on the server. You will most likely have to come up with a mechanism to allow users to mount these at theyre own will by using autofs or something similar like maybe a php web interface to the server that asks for a basic auth before it allows control of mounting. Personally I would think a remote reciever would have a central admin that would be contacted to instruct when backup archives are available but this is only one instance and is not ideal for everything. -- ;s =;