From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 00:19:34 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6DC8106564A; Sun, 2 Oct 2011 00:19:34 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 08C5D8FC0C; Sun, 2 Oct 2011 00:19:33 +0000 (UTC) Received: by wwe3 with SMTP id 3so3961374wwe.31 for ; Sat, 01 Oct 2011 17:19:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=LaO0Gm6CoMEAd8kcfQ6jZ0aIQGdWNnmKyb84kc55i30=; b=jbq/+Df79BP050WdYSF7YVg7Iy0wcR6ms//EpiQA6dbQUj+Y9pL20CYmU3MRnCkdj2 TEbALK0bUD1RrocGdlNa6voCf42mArdkxuMaPTAjzlGwQ6bc4iUjH+SGuUUN7SgKRj2E LlrCVJAalnhLBo5sseCZrKQnvq1abdZfp/6Bo= MIME-Version: 1.0 Received: by 10.216.229.86 with SMTP id g64mr1608669weq.42.1317514772717; Sat, 01 Oct 2011 17:19:32 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.216.182.3 with HTTP; Sat, 1 Oct 2011 17:19:32 -0700 (PDT) In-Reply-To: <201110012137.p91Lb6FI093841@chez.mckusick.com> References: <201110012137.p91Lb6FI093841@chez.mckusick.com> Date: Sun, 2 Oct 2011 02:19:32 +0200 X-Google-Sender-Auth: 0AZU2PJUCZ-d8xDlzYWASIq8nTQ Message-ID: From: Attilio Rao To: Kirk McKusick Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , Xin LI , freebsd-fs@freebsd.org Subject: Re: Need to force sync(2) before umounting UFS1 filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 00:19:34 -0000 2011/10/1 Kirk McKusick : >> Date: Sat, 1 Oct 2011 12:44:04 -0700 >> Subject: Re: Need to force sync(2) before umounting UFS1 filesystems? >> From: Garrett Cooper >> To: Attilio Rao >> Cc: Kostik Belousov , >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Kirk McKusick , freeb= sd-fs@freebsd.org, >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Xin LI >> >> Ok. Now that I know this is the direction you guys want to go, I'll >> start testing the change. >> Thanks! >> -Garrett > > Thanks for throwing some testing at this. Please test my latest > proposed change (included below so you do not have to dig through > earlier email) as I believe that it has the least likelyhood of > problems and is what I am currently proposing to put in. I'm sorry if it wasn't clear in kib/my latest message, but we don't need the coveredvnode unlocking logic because of the tegge's commit. I just think we should commit the change in policy Kirk initially submitted + a comment on top of vfs_busy() explaining why the deadlock with coveredvnode cannot happen. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 01:25:44 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27B22106564A; Sun, 2 Oct 2011 01:25:44 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id CA1B38FC1C; Sun, 2 Oct 2011 01:25:43 +0000 (UTC) Received: by ywp17 with SMTP id 17so3191566ywp.13 for ; Sat, 01 Oct 2011 18:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ls1W3t1RNoxQrJBaZl82CgLlwvROA1M2Njmucu98z24=; b=p3sJhC1q6elZZX9uEYi6Jn5fcUA5S4anxFcJcmak0CyTftBb7QYqe9KKPI8Y2WY93p QmnMALfx/k+6rsem1l1Hq7ng0tQsYhwzOKlSqH40A1E5LEXpiS1diaVd70T0VMegxKrh E76/09IkCyBZ9nwfcbKWTmz7PN6nMK5HE545w= MIME-Version: 1.0 Received: by 10.150.61.11 with SMTP id j11mr11770243yba.164.1317517359165; Sat, 01 Oct 2011 18:02:39 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Sat, 1 Oct 2011 18:02:39 -0700 (PDT) In-Reply-To: References: Date: Sat, 1 Oct 2011 18:02:39 -0700 Message-ID: From: Xin LI To: Chris Rees Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 01:25:44 -0000 Hi, On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: > I've also not heard of anyone using it with zfs successfully- it tends to > shrink rapidly. I'm quite surprised with this assertion. I use tmpfs on my own system and I never see such problem as long as one have sufficient swap space. Not to say there is no problem --there is no way to say "commit this amount of memory to ZFS" but really I have never hit this exact alleged problem... Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 01:59:54 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D3B5106564A; Sun, 2 Oct 2011 01:59:54 +0000 (UTC) (envelope-from Cy.Schubert@komquats.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id EDB208FC14; Sun, 2 Oct 2011 01:59:53 +0000 (UTC) Received: from lb7f8hsrpno-svcs.dcs.int.inet (HELO pd5ml3no-ssvc.prod.shaw.ca) ([10.0.144.222]) by pd6mo1no-svcs.prod.shaw.ca with ESMTP; 01 Oct 2011 19:44:51 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=K6fnQZae8TPX1i0cofjQtTsb/A4CHt4xfMPVU6P219U= c=1 sm=1 a=wsYwexm5Hl4A:10 a=QrugwKR0C_UA:10 a=wAGQQ9Az6v0A:10 a=BLceEmwcHowA:10 a=xA7i7079zcQA:10 a=kj9zAlcOel0A:10 a=2Er20JxOMs3KTlR2XTlUiQ==:17 a=6I5d2MoRAAAA:8 a=BWvPGDcYAAAA:8 a=h3bh4sfWgBWHQ8pq34wA:9 a=ReKk-9sT69euaDyOnQQA:7 a=CjuIK1q_8ugA:10 a=uhEn_2vkUPoA:10 a=V7tsTZBp22UA:10 a=SV7veod9ZcQA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Received: from unknown (HELO spqr.komquats.com) ([24.68.73.211]) by pd5ml3no-dmz.prod.shaw.ca with ESMTP; 01 Oct 2011 19:44:51 -0600 Received: from cwsys.cwsent.com (cwsys [10.1.1.1]) by spqr.komquats.com (Postfix) with ESMTP id 4BA9946B85; Sat, 1 Oct 2011 18:44:20 -0700 (PDT) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.14.5/8.14.5) with ESMTP id p921iJ6f055030; Sat, 1 Oct 2011 18:44:19 -0700 (PDT) (envelope-from Cy.Schubert@komquats.com) Message-Id: <201110020144.p921iJ6f055030@cwsys.cwsent.com> X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.komquats.com/ To: Chris Rees In-Reply-To: Message from Chris Rees of "Sat, 01 Oct 2011 16:48:41 BST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 01 Oct 2011 18:44:19 -0700 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Cy Schubert List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 01:59:54 -0000 In message , Chris Rees writes: > On 1 Oct 2011 16:41, "Benjamin Kaduk" wrote: > > > > On Sat, 1 Oct 2011, Robert Millan wrote: > > > >> Hi, > >> > >> Is TMPFS still considered highly experimental? I notice a warning > >> saying this was added in 2007: > >> > >> fs/tmpfs/tmpfs_vfsops.c: printf("WARNING: TMPFS is considered > >> to be a highly experimental " > >> > >> Since it's very old, I wonder if it still applies. After 4 years and > >> 54 commits, can someone tell if the maturity of this file system has > >> improved significantly? > > > > > > This thread: > > http://lists.freebsd.org/pipermail/freebsd-current/2011-June/025475.html > > has covered this topic somewhat. Peter Holm (pho) is known for running > pretty intensive filesystem (and other) stress tests, and did not come up > with a whole lot of crashes. > > Also, > http://www.freebsd.org/cgi/query-pr-summary.cgi?&sort=none&text=tmpfs > > is not too big, showing only a couple of new reports. > > Mayhaps it is not "highly" experimental, but probably still experimental, > at least. > > > > I've also not heard of anyone using it with zfs successfully- it tends to > shrink rapidly. I use it with ZFS however, having been an OSF/1 (and subsequently Tru64) admin in a previous life, I use it with a limit of 200 MB (10x the amount that OSF/1 did). I've used the same limits when I was a Solaris admin to limit the exposure to accidental DoS attacks. With the tmpfs limit in place I've never seen it shrink. I think tmpfs limits are always a good idea. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 02:02:35 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63DB61065677 for ; Sun, 2 Oct 2011 02:02:35 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta02.westchester.pa.mail.comcast.net [76.96.62.24]) by mx1.freebsd.org (Postfix) with ESMTP id D6D038FC19 for ; Sun, 2 Oct 2011 02:02:34 +0000 (UTC) Received: from omta15.westchester.pa.mail.comcast.net ([76.96.62.87]) by qmta01.westchester.pa.mail.comcast.net with comcast id fcPS1h0051swQuc51e2bm6; Sun, 02 Oct 2011 02:02:35 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta15.westchester.pa.mail.comcast.net with comcast id fe2Z1h00k1t3BNj3be2Z7f; Sun, 02 Oct 2011 02:02:34 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id ECC21102C19; Sat, 1 Oct 2011 19:02:31 -0700 (PDT) Date: Sat, 1 Oct 2011 19:02:31 -0700 From: Jeremy Chadwick To: Xin LI Message-ID: <20111002020231.GA70864@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 02:02:35 -0000 On Sat, Oct 01, 2011 at 06:02:39PM -0700, Xin LI wrote: > On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: > > I've also not heard of anyone using it with zfs successfully- it tends to > > shrink rapidly. > > I'm quite surprised with this assertion. I use tmpfs on my own system > and I never see such problem as long as one have sufficient swap > space. > > Not to say there is no problem --there is no way to say "commit this > amount of memory to ZFS" but really I have never hit this exact > alleged problem... Its been reported multiple times by multiple people, and there has been no official word on -fs, -stable, or zfs-devel that this specific problem has been fixed: * miyamoto moesasji - 2011/01/01 - 8.2-PRERELEASE (thus RELENG_8) - World/kernel build date unknown - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060850.html - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060852.html - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060860.html - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060861.html - Statement from Ivan Voras that it's a known problem: - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060867.html * Atilla Nagy - 2011/01/19 - 8.2-PRERELEASE (thus RELENG_8) - World/kernel build date of 2011/01/08 - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010496.html - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010497.html - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010498.html - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010499.html - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010501.html - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010569.html - Ivan Voras mentions a thread he started on -CURRENT circa 2010/11/21 about this problem: http://www.mail-archive.com/freebsd-current@freebsd.org/msg126491.html * Michael Loftis - 2009/12/08 - FreeBSD 8.0-RELEASE (RELENG_8_0) - World/kernel build date of 2009/11/21 - Might be a different problem altogether, but tmpfs is explicitly mentioned in the fix/commit text from avg@ - http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/141305 Plus Chris Rees' report. So that makes 3, possibly 4 if you consider the PR from Michael Loftis. Possibly it has something to do with the exact FreeBSD version they're using. Not just version (e.g. 8.2-RELEASE vs. 8.2-STABLE), but also kernel/world build date -- because as we've established, ZFS changes are happening all the time and there is very little transparency when it comes to communicating these changes to the community. Maybe the problem has been fixed in RELENG_8 sometime after 2011/01/08, but we do not know. ZFS committers would need to help. Else, folks should be reaching out to the first two people I mentioned above and asking if they can reproduce the problem on present-day RELENG_8. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 05:27:20 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1966106564A; Sun, 2 Oct 2011 05:27:20 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 384068FC0C; Sun, 2 Oct 2011 05:27:19 +0000 (UTC) Received: by gyf2 with SMTP id 2so3281698gyf.13 for ; Sat, 01 Oct 2011 22:27:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=xE6y2YbM49+7Sy0xwMNvR8qWYLtx6cgzFYUb+nWlODg=; b=llYTRsPzoyUfTOLtR1fI5svGZnerj0+a4xnpvZznKKAcmhZ8D2G8B+x2wHCzI7iPTn XlUy+5WcZwnh+It0yanHNREVb6mCaZBRJ+V00z7KsVS++FTidzIh1oFLBjGcwgW+EROU Cz7q0NYjKqK6Ifo120ou4niDck8A4q9S1M8Uc= MIME-Version: 1.0 Received: by 10.150.238.18 with SMTP id l18mr4463389ybh.335.1317533238350; Sat, 01 Oct 2011 22:27:18 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Sat, 1 Oct 2011 22:27:18 -0700 (PDT) In-Reply-To: <20111002020231.GA70864@icarus.home.lan> References: <20111002020231.GA70864@icarus.home.lan> Date: Sat, 1 Oct 2011 22:27:18 -0700 Message-ID: From: Xin LI To: Jeremy Chadwick Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 05:27:20 -0000 On Sat, Oct 1, 2011 at 7:02 PM, Jeremy Chadwick wrote: > On Sat, Oct 01, 2011 at 06:02:39PM -0700, Xin LI wrote: >> On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: >> > I've also not heard of anyone using it with zfs successfully- it tends= to >> > shrink rapidly. >> >> I'm quite surprised with this assertion. =C2=A0I use tmpfs on my own sys= tem >> and I never see such problem as long as one have sufficient swap >> space. >> >> Not to say there is no problem --there is no way to say "commit this >> amount of memory to ZFS" but really I have never hit this exact >> alleged problem... > > Its been reported multiple times by multiple people, and there has been > no official word on -fs, -stable, or zfs-devel that this specific > problem has been fixed: > > * miyamoto moesasji > =C2=A0- 2011/01/01 > =C2=A0- 8.2-PRERELEASE (thus RELENG_8) > =C2=A0- World/kernel build date unknown > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/06= 0850.html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/06= 0852.html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/06= 0860.html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/06= 0861.html > =C2=A0- Statement from Ivan Voras that it's a known problem: > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/06= 0867.html > > * Atilla Nagy > =C2=A0- 2011/01/19 > =C2=A0- 8.2-PRERELEASE (thus RELENG_8) > =C2=A0- World/kernel build date of 2011/01/08 > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010496= .html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010497= .html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010498= .html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010499= .html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010501= .html > =C2=A0- http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010569= .html > =C2=A0- Ivan Voras mentions a thread he started on -CURRENT circa 2010/11= /21 > =C2=A0 =C2=A0about this problem: > =C2=A0 =C2=A0http://www.mail-archive.com/freebsd-current@freebsd.org/msg1= 26491.html > > * Michael Loftis > =C2=A0- 2009/12/08 > =C2=A0- FreeBSD 8.0-RELEASE (RELENG_8_0) > =C2=A0- World/kernel build date of 2009/11/21 > =C2=A0- Might be a different problem altogether, but tmpfs is explicitly > =C2=A0 =C2=A0mentioned in the fix/commit text from avg@ > =C2=A0- http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/141305 > > Plus Chris Rees' report. =C2=A0So that makes 3, possibly 4 if you conside= r > the PR from Michael Loftis. > > Possibly it has something to do with the exact FreeBSD version they're > using. =C2=A0Not just version (e.g. 8.2-RELEASE vs. 8.2-STABLE), but also > kernel/world build date -- because as we've established, ZFS changes are > happening all the time and there is very little transparency when it > comes to communicating these changes to the community. > > Maybe the problem has been fixed in RELENG_8 sometime after 2011/01/08, > but we do not know. =C2=A0ZFS committers would need to help. =C2=A0Else, = folks > should be reaching out to the first two people I mentioned above and > asking if they can reproduce the problem on present-day RELENG_8. The problem Ivan have asserted was not confirmed by anyone who have swap configured properly. Gleb have pointed out that it might be related to a series of integer overflow by the way (he have also fixed a lot of tmpfs issues by the way). Cheers, --=20 Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 06:41:01 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DE01106564A; Sun, 2 Oct 2011 06:41:01 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id F0FEA8FC0C; Sun, 2 Oct 2011 06:41:00 +0000 (UTC) Received: by iadk27 with SMTP id k27so5409230iad.13 for ; Sat, 01 Oct 2011 23:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=rnnF3kACPL3mamDOLTbDZL3yGFm2iaeL6Hb51uCP91I=; b=RuOm18CoKxbyRZjwT2nuzdY/HQXkVjnEWWy/FkYEubgd91CPfpPEFWPbDjIZBWJje5 uXv/XQR1xr3Q2TyPyA8bLbZRSXI7ZoewN7aojnUwwGtKk3lazHNTif3SVvKf14gVgbPK MUMLf3FOgQMZEiMv/ppvmNYpzIN7QxYuWTez4= MIME-Version: 1.0 Received: by 10.231.65.73 with SMTP id h9mr8050327ibi.21.1317537660349; Sat, 01 Oct 2011 23:41:00 -0700 (PDT) Sender: utisoft@gmail.com Received: by 10.231.35.194 with HTTP; Sat, 1 Oct 2011 23:41:00 -0700 (PDT) Received: by 10.231.35.194 with HTTP; Sat, 1 Oct 2011 23:41:00 -0700 (PDT) In-Reply-To: <20111002020231.GA70864@icarus.home.lan> References: <20111002020231.GA70864@icarus.home.lan> Date: Sun, 2 Oct 2011 07:41:00 +0100 X-Google-Sender-Auth: LXFxCzb8B4vvzSgGEdqkf4PX_GE Message-ID: From: Chris Rees To: Jeremy Chadwick Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 06:41:01 -0000 On 2 Oct 2011 03:03, "Jeremy Chadwick" wrote: > > On Sat, Oct 01, 2011 at 06:02:39PM -0700, Xin LI wrote: > > On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: > > > I've also not heard of anyone using it with zfs successfully- it tends to > > > shrink rapidly. > > > > I'm quite surprised with this assertion. I use tmpfs on my own system > > and I never see such problem as long as one have sufficient swap > > space. > > > > Not to say there is no problem --there is no way to say "commit this > > amount of memory to ZFS" but really I have never hit this exact > > alleged problem... > > Its been reported multiple times by multiple people, and there has been > no official word on -fs, -stable, or zfs-devel that this specific > problem has been fixed: > > * miyamoto moesasji > - 2011/01/01 > - 8.2-PRERELEASE (thus RELENG_8) > - World/kernel build date unknown > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060850.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060852.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060860.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060861.html > - Statement from Ivan Voras that it's a known problem: > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060867.html > > * Atilla Nagy > - 2011/01/19 > - 8.2-PRERELEASE (thus RELENG_8) > - World/kernel build date of 2011/01/08 > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010496.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010497.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010498.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010499.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010501.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010569.html > - Ivan Voras mentions a thread he started on -CURRENT circa 2010/11/21 > about this problem: > http://www.mail-archive.com/freebsd-current@freebsd.org/msg126491.html > > * Michael Loftis > - 2009/12/08 > - FreeBSD 8.0-RELEASE (RELENG_8_0) > - World/kernel build date of 2009/11/21 > - Might be a different problem altogether, but tmpfs is explicitly > mentioned in the fix/commit text from avg@ > - http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/141305 > > Plus Chris Rees' report. So that makes 3, possibly 4 if you consider > the PR from Michael Loftis. > > Possibly it has something to do with the exact FreeBSD version they're > using. Not just version (e.g. 8.2-RELEASE vs. 8.2-STABLE), but also > kernel/world build date -- because as we've established, ZFS changes are > happening all the time and there is very little transparency when it > comes to communicating these changes to the community. > > Maybe the problem has been fixed in RELENG_8 sometime after 2011/01/08, > but we do not know. ZFS committers would need to help. Else, folks > should be reaching out to the first two people I mentioned above and > asking if they can reproduce the problem on present-day RELENG_8. > Thanks hugely for doing the linking and research I should have done-- I've not personally had these problems but I recalled the recent conversation, that's all. Chris From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 09:01:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49214106564A; Sun, 2 Oct 2011 09:01:09 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id BBE798FC17; Sun, 2 Oct 2011 09:01:08 +0000 (UTC) Received: by ywp17 with SMTP id 17so3301992ywp.13 for ; Sun, 02 Oct 2011 02:01:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=13BkUlcaIWr2II59cMksQ+yfRdzUYpU/XT2uWwlhj6Y=; b=UhXDYve88itn0GuHlVCV5DbV5IQbf+OBbUYIScGz/CzdWtL0QNnvy65vxMXd83F2Z+ Laz1GFGeuxZDhu17FkEulUmPPyacei9kjoY38m639RX/oMRCpv+DMR43RWmEQ44yJxhv 7iR88LtsmL5SHJa2okeQc9hl1M8Z4zhxyRBWE= MIME-Version: 1.0 Received: by 10.236.76.102 with SMTP id a66mr15890307yhe.25.1317546067753; Sun, 02 Oct 2011 02:01:07 -0700 (PDT) Received: by 10.236.105.166 with HTTP; Sun, 2 Oct 2011 02:01:07 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Sun, 2 Oct 2011 10:01:07 +0100 Message-ID: From: krad To: Chris Rees Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 09:01:09 -0000 On 2 October 2011 07:41, Chris Rees wrote: > On 2 Oct 2011 03:03, "Jeremy Chadwick" wrote: > > > > On Sat, Oct 01, 2011 at 06:02:39PM -0700, Xin LI wrote: > > > On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: > > > > I've also not heard of anyone using it with zfs successfully- it > tends > to > > > > shrink rapidly. > > > > > > I'm quite surprised with this assertion. I use tmpfs on my own system > > > and I never see such problem as long as one have sufficient swap > > > space. > > > > > > Not to say there is no problem --there is no way to say "commit this > > > amount of memory to ZFS" but really I have never hit this exact > > > alleged problem... > > > > Its been reported multiple times by multiple people, and there has been > > no official word on -fs, -stable, or zfs-devel that this specific > > problem has been fixed: > > > > * miyamoto moesasji > > - 2011/01/01 > > - 8.2-PRERELEASE (thus RELENG_8) > > - World/kernel build date unknown > > - > http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060850.html > > - > http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060852.html > > - > http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060860.html > > - > http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060861.html > > - Statement from Ivan Voras that it's a known problem: > > - > http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060867.html > > > > * Atilla Nagy > > - 2011/01/19 > > - 8.2-PRERELEASE (thus RELENG_8) > > - World/kernel build date of 2011/01/08 > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010496.html > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010497.html > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010498.html > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010499.html > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010501.html > > - > http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010569.html > > - Ivan Voras mentions a thread he started on -CURRENT circa 2010/11/21 > > about this problem: > > > http://www.mail-archive.com/freebsd-current@freebsd.org/msg126491.html > > > > * Michael Loftis > > - 2009/12/08 > > - FreeBSD 8.0-RELEASE (RELENG_8_0) > > - World/kernel build date of 2009/11/21 > > - Might be a different problem altogether, but tmpfs is explicitly > > mentioned in the fix/commit text from avg@ > > - http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/141305 > > > > Plus Chris Rees' report. So that makes 3, possibly 4 if you consider > > the PR from Michael Loftis. > > > > Possibly it has something to do with the exact FreeBSD version they're > > using. Not just version (e.g. 8.2-RELEASE vs. 8.2-STABLE), but also > > kernel/world build date -- because as we've established, ZFS changes are > > happening all the time and there is very little transparency when it > > comes to communicating these changes to the community. > > > > Maybe the problem has been fixed in RELENG_8 sometime after 2011/01/08, > > but we do not know. ZFS committers would need to help. Else, folks > > should be reaching out to the first two people I mentioned above and > > asking if they can reproduce the problem on present-day RELENG_8. > > > > Thanks hugely for doing the linking and research I should have done-- I've > not personally had these problems but I recalled the recent conversation, > that's all. > > Chris > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > It may seem a silly question, but I have been wondering about tmpfs and zfs, and whether there is any point to mixing the two? Surely if you have frequently accessed files under /tmp they are going to be in the arc or l2arc anyway so fairly speedy, or am I missing the point of tmpfs? From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 09:09:15 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00F26106566B; Sun, 2 Oct 2011 09:09:15 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7C2458FC12; Sun, 2 Oct 2011 09:09:14 +0000 (UTC) Received: by yxk36 with SMTP id 36so3410190yxk.13 for ; Sun, 02 Oct 2011 02:09:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MJ34FTER3Fa/FbtPnaNHQqcU7Jdo1ZYky6bgHJB9TMM=; b=rQbSIDlP6nQ5R5cpO4iuwcmh15afB1Sorw3kPUqiRuxPEGnCQ9vAFX+oUH8Zq2G04u RQO06TuNEMxrMDi97Lohg1++acziz/PqYfdI++moSoA3zUaGkkNiEp3Wc+nDPf4nZfSV 2bvuNuHiSlKvuG7YCHgaVdx+Gr1oorqtQLIdA= MIME-Version: 1.0 Received: by 10.150.238.18 with SMTP id l18mr4536903ybh.335.1317546552842; Sun, 02 Oct 2011 02:09:12 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Sun, 2 Oct 2011 02:09:12 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Sun, 2 Oct 2011 02:09:12 -0700 Message-ID: From: Xin LI To: krad Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 09:09:15 -0000 On Sun, Oct 2, 2011 at 2:01 AM, krad wrote: > It may seem a silly question, but I have been wondering about tmpfs and zfs, > and whether there is any point to mixing the two? > > Surely if you have frequently accessed files under /tmp they are going to be > in the arc or l2arc anyway so fairly speedy, or am I missing the point of > tmpfs? tmpfs is useful when persist storage is unnecessary -- when system goes off (or rebooted) the data are dropped. If you use a normal file system then persist semantics are obeyed say, if you do sync(), data is being flushed into data, etc. Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 10:24:00 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49828106566B; Sun, 2 Oct 2011 10:24:00 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id CC3A98FC08; Sun, 2 Oct 2011 10:23:59 +0000 (UTC) Received: by qyk10 with SMTP id 10so1530878qyk.13 for ; Sun, 02 Oct 2011 03:23:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.92.10 with SMTP id p10mr10117468qcm.183.1317551038221; Sun, 02 Oct 2011 03:23:58 -0700 (PDT) Received: by 10.229.223.196 with HTTP; Sun, 2 Oct 2011 03:23:58 -0700 (PDT) In-Reply-To: References: Date: Sun, 2 Oct 2011 12:23:58 +0200 Message-ID: From: Olivier Smedts To: Xin LI Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 10:24:00 -0000 2011/10/2 Xin LI : > Hi, > > On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: >> I've also not heard of anyone using it with zfs successfully- it tends t= o >> shrink rapidly. > > I'm quite surprised with this assertion. =A0I use tmpfs on my own system > and I never see such problem as long as one have sufficient swap > space. The problem here is "sufficient swap space". I've got 8G of RAM, and 2G of swap (just in case). When the ZFS ARC reaches 4G, there's no room for a single byte in tmpfs, even with 2G swap free and at least 2-3G RAM free. The swap size must be at least the RAM size if you plan on using ZFS and tmpfs. That's a problem for me because I'm short on disk space, and there's no point in having an enormous swap size (hey, minidumps !) when you already have lots of RAM, which is the case in most ZFS installs. Cheers > Not to say there is no problem --there is no way to say "commit this > amount of memory to ZFS" but really I have never hit this exact > alleged problem... > > Cheers, > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > --=20 Olivier Smedts=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0 _ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 ASCII ribbon campaign ( ) e-mail: olivier@gid0.org=A0 =A0 =A0 =A0 - against HTML email & vCards=A0 X www: http://www.gid0.org=A0 =A0 - against proprietary attachments / \ =A0 "Il y a seulement 10 sortes de gens dans le monde : =A0 ceux qui comprennent le binaire, =A0 et ceux qui ne le comprennent pas." From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 14:25:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA5E4106566B; Sun, 2 Oct 2011 14:25:09 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 763C68FC12; Sun, 2 Oct 2011 14:25:09 +0000 (UTC) Received: by iadk27 with SMTP id k27so5857498iad.13 for ; Sun, 02 Oct 2011 07:25:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=J7BoISsX00JHHuIgOOJ8xX2djGuzcNeflo+EjC1jJBE=; b=YYMZlMfWygwW2aw0oOoh3DlKFw8pCIABaUHAMMF/dAs0bte5m3C0iTvPv8OCpMsn38 NID+95IFwuWf7hHboRtjHbFC2cpBFjsnuSt8YNdEpomlmE5ctB00A7qIz53v7IbfS3UV o7jSyP9tONGTkZ34ku1QvElBUvr+KsilZOFSM= Received: by 10.231.69.80 with SMTP id y16mr21205435ibi.34.1317565509055; Sun, 02 Oct 2011 07:25:09 -0700 (PDT) MIME-Version: 1.0 Sender: utisoft@gmail.com Received: by 10.231.35.194 with HTTP; Sun, 2 Oct 2011 07:24:39 -0700 (PDT) In-Reply-To: References: From: Chris Rees Date: Sun, 2 Oct 2011 15:24:39 +0100 X-Google-Sender-Auth: n8LwwVYK4uhkxheEOdYxm24KsjM Message-ID: To: Olivier Smedts Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, delphij@freebsd.org, Adrian Chadd Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 14:25:09 -0000 On 2 October 2011 11:23, Olivier Smedts wrote: > 2011/10/2 Xin LI : >> Hi, >> >> On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: >>> I've also not heard of anyone using it with zfs successfully- it tends = to >>> shrink rapidly. >> >> I'm quite surprised with this assertion. =A0I use tmpfs on my own system >> and I never see such problem as long as one have sufficient swap >> space. > > The problem here is "sufficient swap space". > > I've got 8G of RAM, and 2G of swap (just in case). When the ZFS ARC > reaches 4G, there's no room for a single byte in tmpfs, even with 2G > swap free and at least 2-3G RAM free. The swap size must be at least > the RAM size if you plan on using ZFS and tmpfs. That's a problem for > me because I'm short on disk space, and there's no point in having an > enormous swap size (hey, minidumps !) when you already have lots of > RAM, which is the case in most ZFS installs. > It's common practice to set the swap size equal to or more than the physical RAM size, as well as assumed in many places-- how could you have a crash dump otherwise? Unfortunately you probably won't get a 'fix' to allow swap Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C745106564A; Sun, 2 Oct 2011 15:11:47 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 2BD518FC08; Sun, 2 Oct 2011 15:11:47 +0000 (UTC) Received: from delta.delphij.net (c-76-102-50-245.hsd1.ca.comcast.net [76.102.50.245]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id D377415690; Sun, 2 Oct 2011 08:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1317568307; bh=nbvKB1cH4w8CwhhWcC1oR6PC1bIV0/sk9AV5C8JMGdM=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=HxNC+HreYOYtfspMh6jNJ7kB2C1UbZmdWMJrTol2GNzsMRQHZ5qiaeq/aAY4MqltF 8Y8R+U30982FZEVtTY/olofWtu3SL327CdWoqvqy3xAknpWpltdhmS82HOGMrAvtI2 iScU8sGC0sRnUKcqyud/WDNUX3GXaXHF2pRtE8Jc= Message-ID: <4E887F31.8000407@delphij.net> Date: Sun, 02 Oct 2011 08:11:45 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Olivier Smedts References: In-Reply-To: OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 15:11:47 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/02/11 03:23, Olivier Smedts wrote: > 2011/10/2 Xin LI : >> Hi, >> >> On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees >> wrote: >>> I've also not heard of anyone using it with zfs successfully- >>> it tends to shrink rapidly. >> >> I'm quite surprised with this assertion. I use tmpfs on my own >> system and I never see such problem as long as one have >> sufficient swap space. > > The problem here is "sufficient swap space". > > I've got 8G of RAM, and 2G of swap (just in case). When the ZFS > ARC reaches 4G, there's no room for a single byte in tmpfs, even > with 2G swap free and at least 2-3G RAM free. The swap size must be > at least the RAM size if you plan on using ZFS and tmpfs. That's a > problem for me because I'm short on disk space, and there's no > point in having an enormous swap size (hey, minidumps !) when you > already have lots of RAM, which is the case in most ZFS installs. A possible workaround for this might be limiting ARC size? I can't think about any better idea at this point. My personal experience is always have at least RAM sized swap because the system may choose to write in-core data to swap which enables it to discard these memory without expensive swapout in emergency cases. Depending on workload, this could give better responsiveness and frankly I never consider having a few tens of GBs of swap as expensive since large disks are not expensive nowadays... Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOiH8xAAoJEATO+BI/yjfB/BsH/1IpXJioC+R5ZwUBE2mRDP/E rlwGjx138iHFzUjQn7Q8YYmmPhUKiad2fQOIwnErfMOjPUiEVP/CLBya6jQG8g6n wHco8ilIerMSM8i7R1iTbC9gJxte2orxTBrnLkTMr1AcW/MjxpSBL4Z2kzIrZdsx 8bEFCig0tKY5gluM8ZTBaMK+yVEFV+ff6PHiHu//XcnG03CFqiu9V4HMhvrXCTcB QBDl+/8+PdLPWEDHLzSW51te1YEoF/xDkDkHu7UtTX3egEkiDeZ2uCmazgWFu/Xw TPMRiXUfYklDpT+rMbRH44CN5VeEWhOyZRDyr/pGOzjylAe2ShFGNxpua+KpLJI= =0mDY -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 15:29:32 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8F111065673; Sun, 2 Oct 2011 15:29:32 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3F0218FC14; Sun, 2 Oct 2011 15:29:31 +0000 (UTC) Received: by wyj26 with SMTP id 26so3278879wyj.13 for ; Sun, 02 Oct 2011 08:29:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=cnEpH/KfOu9r43EJK4EInNq/l2WTb7Pc4d7V2ELon90=; b=ITsGmXVCH4z4mL31tpY2E+2+paBoFN9wsaDn+dJYNf1KgoZ0SzpabVX75LiNtnPXYY MPMs6U1tkD0YLfW7BkfMI6dMLuSsToXevIruZoZwDnyJX3LIr1TDd8z3AMinKhE8K+MW r9Z0iZI3GAMUj5vxmSC3PwO7cIU/rIIg7JwBo= Received: by 10.227.145.139 with SMTP id d11mr8735214wbv.60.1317567718142; Sun, 02 Oct 2011 08:01:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.227.208.16 with HTTP; Sun, 2 Oct 2011 08:01:27 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Sun, 2 Oct 2011 11:01:27 -0400 Message-ID: To: Chris Rees Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 15:29:32 -0000 > It's common practice to set the swap size equal to or more than the > physical RAM size, as well as assumed in many places-- how could you > have a crash dump otherwise? minidumps don't require swap => ram -- Eitan Adler From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 19:15:08 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2B88106566B; Sun, 2 Oct 2011 19:15:08 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6182D8FC13; Sun, 2 Oct 2011 19:15:08 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 159014AC1C; Sun, 2 Oct 2011 23:15:06 +0400 (MSD) Date: Sun, 2 Oct 2011 23:14:59 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <228926402.20111002231459@serebryakov.spb.ru> To: current@freebsd.org, fs@FreeBSD.org, geom@freebsd.org In-Reply-To: <1258376930.20111002193223@serebryakov.spb.ru> References: <1258376930.20111002193223@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Subject: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 19:15:09 -0000 Hello, Current. You wrote 2 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 19:32:23: It seems, that error only occurs when I call alq API from geom threads (from g_events, for example). Module, which is not GEOM class, could use this API without any problem! Is it normal, that GEOM could not use vnode API? Is it normal, that this leads to panic, and not some diagnostic messages, even with WITNESS and other diagnostic options turned on? I'm looping fs@ and geom@, as it seems to be related to them. > I'm trying to create logging queue with alq kernel API. I call > alq_open_flags() like this in my module: > error =3D alq_open_flags(&sc->sc_alq, sc->sc_vnode_name, > curthread->td_ucred, ALQ_DEFAULT_CMODE, > sizeof(struct g_log_entry), ALQ_ORDERED); > and my system (10-CURRENT) panics with this stack trace (top frames > are DDB-related, so I omit them): > #5 0xc06101c3 in kdb_trap (type=3D12, code=3D0, tf=3D0xc4e29990) > at /usr/home/lev/FreeBSD-head/sys/kern/subr_kdb.c:620 > #6 0xc08290af in trap_fatal (frame=3D0xc4e29990, eva=3D136) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:958 > #7 0xc08292c0 in trap_pfault (frame=3D0xc4e29990, usermode=3D0, eva=3D13= 6) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:880 > #8 0xc0829f46 in trap (frame=3D0xc4e29990) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:555 > #9 0xc0812e7c in calltrap () > at /usr/home/lev/FreeBSD-head/sys/i386/i386/exception.s:168 > #10 0xc05cafc0 in _mtx_lock_flags (m=3D0x78, opts=3D0,=20 > file=3D0xc088c140 "/usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c",= =20 > line=3D2169) at /usr/home/lev/FreeBSD-head/sys/kern/kern_mutex.c:194 > #11 0xc0672eb2 in vref (vp=3D0x0) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c:2169 > #12 0xc066932f in namei (ndp=3D0xc4e29b74) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_lookup.c:264 > #13 0xc0682900 in vn_open_cred (ndp=3D0xc4e29b74, flagp=3D0xc4e29be8, cmo= de=3D384, > vn_open_flags=3D0, cred=3D0xc50fee80, fp=3D0x0) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_vnops.c:137 > #14 0xc5c42609 in alq_open_flags (alqp=3D0xc550dc08,=20 > file=3D0xc5108d40 "/usr/ada4.log", cred=3D0xc50fee80, cmode=3D384, si= ze=3D28, > flags=3D16) > at > /usr/home/lev/FreeBSD-head/sys/modules/alq/../../kern/kern_alq.c:451 > It seems, that vref() get NULL instead of valid pointer to struct > vnode. But I have no idea -- why?! Yes, I have no such file created, > but man alq(9) says, that it will create file for me. And if I point > to existed file, it panic anyway. > What do I do wrong?! --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 22:33:32 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30DEA106564A for ; Sun, 2 Oct 2011 22:33:32 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id AFC9E8FC0C for ; Sun, 2 Oct 2011 22:33:31 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RAUbC-0004z2-HU for freebsd-fs@freebsd.org; Mon, 03 Oct 2011 00:33:30 +0200 Received: from 15-91.dsl.iskon.hr ([89.164.15.91]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Oct 2011 00:33:30 +0200 Received: from ivoras by 15-91.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Oct 2011 00:33:30 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Mon, 03 Oct 2011 00:33:15 +0200 Lines: 33 Message-ID: References: <20111002020231.GA70864@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 15-91.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 In-Reply-To: Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 22:33:32 -0000 On 02/10/2011 07:27, Xin LI wrote: > The problem Ivan have asserted was not confirmed by anyone who have > swap configured properly. Gleb have pointed out that it might be > related to a series of integer overflow by the way (he have also fixed > a lot of tmpfs issues by the way). Well, instead of guessing I can point you to the way I got the original situation - you said you have ZFS configured so it would be easy for you to check. You should to something like this: - configure your system to the best of your abilities (but post what you did different from the defaults) - install postgresql (8.4+, but I don't think the version is important), configure it so it gets half the system memory or 2/3 the system memory for its shared_buffers. - install pgbench - initialize pgbench so that the database definitely is larger than the entire memory you got (NOTE: THIS IS NOT A CONTRIVED TEST - lots of databases in practice are larger than the RAM in the server). - run pgbench & observe the results. This should create really bad contention problem for memory between postgresql and ZFS, which should manifest itself in tmpfs shrinking to 0 bytes free. If you don't get this problem then great, it might be solved! (for more info on pgbench, see this: http://www.westnet.com/~gsmith/content/postgresql/pgbench-scaling.htm). From owner-freebsd-fs@FreeBSD.ORG Sun Oct 2 22:40:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CFE7106566C for ; Sun, 2 Oct 2011 22:40:09 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 239258FC14 for ; Sun, 2 Oct 2011 22:40:08 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RAUhb-0006Zx-Va for freebsd-fs@freebsd.org; Mon, 03 Oct 2011 00:40:07 +0200 Received: from 15-91.dsl.iskon.hr ([89.164.15.91]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Oct 2011 00:40:07 +0200 Received: from ivoras by 15-91.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Oct 2011 00:40:07 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Mon, 03 Oct 2011 00:35:30 +0200 Lines: 11 Message-ID: References: <20111002020231.GA70864@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 15-91.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 In-Reply-To: Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 22:40:09 -0000 On 02/10/2011 11:01, krad wrote: > It may seem a silly question, but I have been wondering about tmpfs and zfs, > and whether there is any point to mixing the two? > > Surely if you have frequently accessed files under /tmp they are going to be > in the arc or l2arc anyway so fairly speedy, or am I missing the point of > tmpfs? You use it when you want to be more certain that your data will never touch a drive. It's not only for short lived files. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 07:34:25 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C92A2106566C; Mon, 3 Oct 2011 07:34:25 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 8C6468FC08; Mon, 3 Oct 2011 07:34:25 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 3452E4AC1C; Mon, 3 Oct 2011 11:34:24 +0400 (MSD) Date: Mon, 3 Oct 2011 11:34:17 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <349860851.20111003113417@serebryakov.spb.ru> To: current@freebsd.org, fs@FreeBSD.org, geom@freebsd.org In-Reply-To: <228926402.20111002231459@serebryakov.spb.ru> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 07:34:25 -0000 Hello, Current. You wrote 2 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 23:14:59: > It seems, that error only occurs when I call alq API from geom > threads (from g_events, for example). Module, which is not GEOM > class, could use this API without any problem! > Is it normal, that GEOM could not use vnode API? Is it normal, that > this leads to panic, and not some diagnostic messages, even with > WITNESS and other diagnostic options turned on? not holding (explicitly release before call) topology lock doesn't help. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 10:13:35 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 184091065673 for ; Mon, 3 Oct 2011 10:13:35 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id D220F8FC12 for ; Mon, 3 Oct 2011 10:13:34 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 2E1397E824; Mon, 3 Oct 2011 20:57:05 +1100 (EST) Message-ID: <4E8986F0.3050007@freebsd.org> Date: Mon, 03 Oct 2011 20:57:04 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20110914 Thunderbird/6.0.2 MIME-Version: 1.0 To: lev@freebsd.org References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> In-Reply-To: <349860851.20111003113417@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 10:13:35 -0000 [trimming current@ and geom@] On 10/03/11 18:34, Lev Serebryakov wrote: > Hello, Current. > You wrote 2 îêòÿáðÿ 2011 ã., 23:14:59: > >> It seems, that error only occurs when I call alq API from geom >> threads (from g_events, for example). Module, which is not GEOM >> class, could use this API without any problem! > >> Is it normal, that GEOM could not use vnode API? Is it normal, that >> this leads to panic, and not some diagnostic messages, even with >> WITNESS and other diagnostic options turned on? > > not holding (explicitly release before call) topology lock doesn't > help. I know nothing about VFS, but wonder if it's something to do with the credentials you pass in? Alternatively, the way we do the VFS related calls in alq_open_flags() could be at fault... does anyone see anything wrong with the following (from sys/kern/kern_alq.c): ########## int alq_open_flags(struct alq **alqp, const char *file, struct ucred *cred, int cmode, int size, int flags) { struct thread *td; struct nameidata nd; struct alq *alq; int oflags; int error; int vfslocked; KASSERT((size > 0), ("%s: size <= 0", __func__)); *alqp = NULL; td = curthread; NDINIT(&nd, LOOKUP, NOFOLLOW | MPSAFE, UIO_SYSSPACE, file, td); oflags = FWRITE | O_NOFOLLOW | O_CREAT; error = vn_open_cred(&nd, &oflags, cmode, 0, cred, NULL); if (error) return (error); vfslocked = NDHASGIANT(&nd); NDFREE(&nd, NDF_ONLY_PNBUF); /* We just unlock so we hold a reference */ VOP_UNLOCK(nd.ni_vp, 0); VFS_UNLOCK_GIANT(vfslocked); ########## Lev, are you able to share your code with us? Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 11:07:06 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D874106567C for ; Mon, 3 Oct 2011 11:07:06 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 438098FC18 for ; Mon, 3 Oct 2011 11:07:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p93B76cn033755 for ; Mon, 3 Oct 2011 11:07:06 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p93B75LJ033753 for freebsd-fs@FreeBSD.org; Mon, 3 Oct 2011 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Oct 2011 11:07:05 GMT Message-Id: <201110031107.p93B75LJ033753@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 11:07:06 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159971 fs [ffs] [panic] panic with soft updates journaling durin o kern/159930 fs [ufs] [panic] kernel core o kern/159418 fs [tmpfs] [panic] tmpfs kernel panic: recursing on non r o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159233 fs [ext2fs] [patch] fs/ext2fs: finish reallocblk implemen o kern/159232 fs [ext2fs] [patch] fs/ext2fs: merge ext2_readwrite into o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs [amd] amd(8) ICMP storm and unkillable process. o kern/158711 fs [ffs] [panic] panic in ffs_blkfree and ffs_valloc o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157722 fs [geli] unable to newfs a geli encrypted partition o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156168 fs [nfs] [panic] Kernel panic under concurrent access ove o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs o kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 o kern/154447 fs [zfs] [panic] Occasional panics - solaris assert somew p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153847 fs [nfs] [panic] Kernel panic from incorrect m_free in nf o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small p kern/152488 fs [tmpfs] [patch] mtime of file updated when only inode o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o kern/151845 fs [smbfs] [patch] smbfs should be upgraded to support Un o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/150207 fs zpool(1): zpool import -d /dev tries to open weird dev o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148204 fs [nfs] UDP NFS causes overload o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147790 fs [zfs] zfs set acl(mode|inherit) fails on existing zfs o kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133174 fs [msdosfs] [patch] msdosfs must support multibyte inter o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs f kern/130133 fs [panic] [zfs] 'kmem_map too small' caused by make clea o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs f kern/127375 fs [zfs] If vm.kmem_size_max>"1073741823" then write spee o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi f kern/126703 fs [panic] [zfs] _mtx_lock_sleep: recursed on non-recursi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files f sparc/123566 fs [zfs] zpool import issue: EOVERFLOW o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/120210 fs [zfs] [panic] reboot after panic: solaris assert: arc_ o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] [iconv] mount_msdosfs: msdosfs_iconv: Operat o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 249 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 11:48:16 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC660106566B; Mon, 3 Oct 2011 11:48:16 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id EE7E78FC08; Mon, 3 Oct 2011 11:48:15 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id DF6BEA8D868; Mon, 3 Oct 2011 13:29:19 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 18.4129] X-CRM114-CacheID: sfid-20111003_13291_8FB976E7 X-CRM114-Status: Good ( pR: 18.4129 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Oct 3 13:29:19 2011 X-DSPAM-Confidence: 0.9964 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4e899c8f617025446310241 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, FreeBSD, 0.00054, FreeBSD, 0.00054, >+On, 0.00099, wrote+>, 0.00179, bug, 0.00198, On+Sat, 0.00219, On+Sat, 0.00219, wrote+>>, 0.00267, >+>, 0.00288, >+>, 0.00288, org>+wrote, 0.00292, References*mail.gmail.com>, 0.00295, References*mail.gmail.com>, 0.00295, >>+>>, 0.00409, root, 0.00454, wrote, 0.00495, wrote, 0.00495, 2011+at, 0.00532, 2011+at, 0.00532, STABLE, 0.00556, STABLE, 0.00556, /tmp, 0.00612, /tmp, 0.00612, FreeBSD+8, 0.00612, Filesystem, 0.00679, X-Spambayes-Classification: ham; 0.00 Received: from japan.t-online.private (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 0BA3EA8D85B; Mon, 3 Oct 2011 13:29:19 +0200 (CEST) Message-ID: <4E899C8E.7040305@fsn.hu> Date: Mon, 03 Oct 2011 13:29:18 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Jeremy Chadwick References: <20111002020231.GA70864@icarus.home.lan> In-Reply-To: <20111002020231.GA70864@icarus.home.lan> X-Stationery: 0.7.5 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, delphij@freebsd.org, Adrian Chadd Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 11:48:16 -0000 Hi, On 10/02/11 04:02, Jeremy Chadwick wrote: > On Sat, Oct 01, 2011 at 06:02:39PM -0700, Xin LI wrote: >> On Sat, Oct 1, 2011 at 8:48 AM, Chris Rees wrote: >>> I've also not heard of anyone using it with zfs successfully- it tends to >>> shrink rapidly. >> I'm quite surprised with this assertion. I use tmpfs on my own system >> and I never see such problem as long as one have sufficient swap >> space. >> >> Not to say there is no problem --there is no way to say "commit this >> amount of memory to ZFS" but really I have never hit this exact >> alleged problem... > Its been reported multiple times by multiple people, and there has been > no official word on -fs, -stable, or zfs-devel that this specific > problem has been fixed: > > * miyamoto moesasji > - 2011/01/01 > - 8.2-PRERELEASE (thus RELENG_8) > - World/kernel build date unknown > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060850.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060852.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060860.html > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060861.html > - Statement from Ivan Voras that it's a known problem: > - http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/060867.html > > * Atilla Nagy > - 2011/01/19 > - 8.2-PRERELEASE (thus RELENG_8) > - World/kernel build date of 2011/01/08 > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010496.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010497.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010498.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010499.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010501.html > - http://lists.freebsd.org/pipermail/freebsd-fs/2011-January/010569.html > - Ivan Voras mentions a thread he started on -CURRENT circa 2010/11/21 > about this problem: > http://www.mail-archive.com/freebsd-current@freebsd.org/msg126491.html > For me, the bug is still here: $ uname -a FreeBSD b 8.2-STABLE FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 $ df -h /tmp Filesystem Size Used Avail Capacity Mounted on tmpfs 0B 0B 0B 100% /tmp I have no swap configured. The machine has 64 GB RAM. vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 12:19:09 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41D121065670; Mon, 3 Oct 2011 12:19:09 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 911DA8FC14; Mon, 3 Oct 2011 12:19:08 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 8C3F14AC1C; Mon, 3 Oct 2011 16:19:06 +0400 (MSD) Date: Mon, 3 Oct 2011 16:18:59 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1223820108.20111003161859@serebryakov.spb.ru> To: Lawrence Stewart In-Reply-To: <4E8986F0.3050007@freebsd.org> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------D513F1DB29DF10BB" Cc: fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 12:19:09 -0000 ------------D513F1DB29DF10BB Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Hello, Lawrence. You wrote 3 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 13:57:04: > I know nothing about VFS, but wonder if it's something to do with the > credentials you pass in? They looks Ok. Not NULL, at least :) I'm passing "curtrhead->td_ucred". > Lev, are you able to share your code with us? Yes, of course. Minimal code, which trigger this error, attached. Build, load module and PANIC! This is geom_zero module, which try to create ALQ with name "/var/log/zero.alq.log" on it load (not creation! So, you don't need even create such GEOM!). Please note, that "init" callback of GEOM class is called in g_event GEOM thread. Code is for 9.0/10.0 My code is rewritten with usage of working thread already, and it works without any panic. But worker thread looks like overkill for simple case "write 24 bytes record to ALQ for each BIO" to me. So I could prove, that this code PANIC because alq_open_flags() is called from one of three GEOM threads. --=20 // Black Lion AKA Lev Serebryakov ------------D513F1DB29DF10BB Content-Type: application/octet-stream; name="g_alq.c" Content-transfer-encoding: base64 Content-Disposition: attachment; filename="g_alq.c" LyotCiAqIENvcHlyaWdodCAoYykgMjAwNSBQYXdlbCBKYWt1YiBEYXdpZGVrIDxwamRARnJl ZUJTRC5vcmc+CiAqIEFsbCByaWdodHMgcmVzZXJ2ZWQuCiAqCiAqIFJlZGlzdHJpYnV0aW9u IGFuZCB1c2UgaW4gc291cmNlIGFuZCBiaW5hcnkgZm9ybXMsIHdpdGggb3Igd2l0aG91dAog KiBtb2RpZmljYXRpb24sIGFyZSBwZXJtaXR0ZWQgcHJvdmlkZWQgdGhhdCB0aGUgZm9sbG93 aW5nIGNvbmRpdGlvbnMKICogYXJlIG1ldDoKICogMS4gUmVkaXN0cmlidXRpb25zIG9mIHNv dXJjZSBjb2RlIG11c3QgcmV0YWluIHRoZSBhYm92ZSBjb3B5cmlnaHQKICogICAgbm90aWNl LCB0aGlzIGxpc3Qgb2YgY29uZGl0aW9ucyBhbmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVy LgogKiAyLiBSZWRpc3RyaWJ1dGlvbnMgaW4gYmluYXJ5IGZvcm0gbXVzdCByZXByb2R1Y2Ug dGhlIGFib3ZlIGNvcHlyaWdodAogKiAgICBub3RpY2UsIHRoaXMgbGlzdCBvZiBjb25kaXRp b25zIGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIgaW4gdGhlCiAqICAgIGRvY3VtZW50 YXRpb24gYW5kL29yIG90aGVyIG1hdGVyaWFscyBwcm92aWRlZCB3aXRoIHRoZSBkaXN0cmli dXRpb24uCiAqCiAqIFRISVMgU09GVFdBUkUgSVMgUFJPVklERUQgQlkgVEhFIEFVVEhPUlMg QU5EIENPTlRSSUJVVE9SUyBgYEFTIElTJycgQU5ECiAqIEFOWSBFWFBSRVNTIE9SIElNUExJ RUQgV0FSUkFOVElFUywgSU5DTFVESU5HLCBCVVQgTk9UIExJTUlURUQgVE8sIFRIRQogKiBJ TVBMSUVEIFdBUlJBTlRJRVMgT0YgTUVSQ0hBTlRBQklMSVRZIEFORCBGSVRORVNTIEZPUiBB IFBBUlRJQ1VMQVIgUFVSUE9TRQogKiBBUkUgRElTQ0xBSU1FRC4gIElOIE5PIEVWRU5UIFNI QUxMIFRIRSBBVVRIT1JTIE9SIENPTlRSSUJVVE9SUyBCRSBMSUFCTEUKICogRk9SIEFOWSBE SVJFQ1QsIElORElSRUNULCBJTkNJREVOVEFMLCBTUEVDSUFMLCBFWEVNUExBUlksIE9SIENP TlNFUVVFTlRJQUwKICogREFNQUdFUyAoSU5DTFVESU5HLCBCVVQgTk9UIExJTUlURUQgVE8s IFBST0NVUkVNRU5UIE9GIFNVQlNUSVRVVEUgR09PRFMKICogT1IgU0VSVklDRVM7IExPU1Mg T0YgVVNFLCBEQVRBLCBPUiBQUk9GSVRTOyBPUiBCVVNJTkVTUyBJTlRFUlJVUFRJT04pCiAq IEhPV0VWRVIgQ0FVU0VEIEFORCBPTiBBTlkgVEhFT1JZIE9GIExJQUJJTElUWSwgV0hFVEhF UiBJTiBDT05UUkFDVCwgU1RSSUNUCiAqIExJQUJJTElUWSwgT1IgVE9SVCAoSU5DTFVESU5H IE5FR0xJR0VOQ0UgT1IgT1RIRVJXSVNFKSBBUklTSU5HIElOIEFOWSBXQVkKICogT1VUIE9G IFRIRSBVU0UgT0YgVEhJUyBTT0ZUV0FSRSwgRVZFTiBJRiBBRFZJU0VEIE9GIFRIRSBQT1NT SUJJTElUWSBPRgogKiBTVUNIIERBTUFHRS4KICovCgojaW5jbHVkZSA8c3lzL2NkZWZzLmg+ Cl9fRkJTRElEKCIkRnJlZUJTRCQiKTsKCiNpbmNsdWRlIDxzeXMvcGFyYW0uaD4KI2luY2x1 ZGUgPHN5cy9iaW8uaD4KI2luY2x1ZGUgPHN5cy9rZXJuZWwuaD4KI2luY2x1ZGUgPHN5cy9s aW1pdHMuaD4KI2luY2x1ZGUgPHN5cy9tYWxsb2MuaD4KI2luY2x1ZGUgPHN5cy9xdWV1ZS5o PgojaW5jbHVkZSA8c3lzL3N5c2N0bC5oPgojaW5jbHVkZSA8c3lzL3N5c3RtLmg+CiNpbmNs dWRlIDxzeXMvYWxxLmg+CiNpbmNsdWRlIDxzeXMvcHJvYy5oPgoKI2luY2x1ZGUgPGdlb20v Z2VvbS5oPgoKCiNkZWZpbmUJR19aRVJPX0NMQVNTX05BTUUJIkFMUS1QQU5JQyIKClNZU0NU TF9ERUNMKF9rZXJuX2dlb20pOwpTWVNDVExfTk9ERShfa2Vybl9nZW9tLCBPSURfQVVUTywg emVybywgQ1RMRkxBR19SVywgMCwgIkdFT01fQUxRIHN0dWZmIik7CgpzdGF0aWMgdm9pZApn X2FscV9zdGFydChzdHJ1Y3QgYmlvICpicCkKewoJaW50IGVycm9yID0gRU5YSU87CgoJc3dp dGNoIChicC0+YmlvX2NtZCkgewoJY2FzZSBCSU9fUkVBRDoKCWNhc2UgQklPX0RFTEVURToK CWNhc2UgQklPX1dSSVRFOgoJCWJwLT5iaW9fY29tcGxldGVkID0gYnAtPmJpb19sZW5ndGg7 CgkJZXJyb3IgPSAwOwoJCWJyZWFrOwoJY2FzZSBCSU9fR0VUQVRUUjoKCWRlZmF1bHQ6CgkJ ZXJyb3IgPSBFT1BOT1RTVVBQOwoJCWJyZWFrOwoJfQoJZ19pb19kZWxpdmVyKGJwLCBlcnJv cik7Cn0KCnN0YXRpYyB2b2lkCmdfYWxxX2luaXQoc3RydWN0IGdfY2xhc3MgKm1wKQp7Cglz dHJ1Y3QgZ19nZW9tICpncDsKCXN0cnVjdCBnX3Byb3ZpZGVyICpwcDsKCXN0cnVjdCBhbHEg KmFscTsKCWludCBlcnJvcjsKCglnX3RvcG9sb2d5X2Fzc2VydCgpOwoJZ3AgPSBnX25ld19n ZW9tZihtcCwgImdhbHEiKTsKCWdwLT5zdGFydCA9IGdfYWxxX3N0YXJ0OwoJZ3AtPmFjY2Vz cyA9IGdfc3RkX2FjY2VzczsKCXBwID0gZ19uZXdfcHJvdmlkZXJmKGdwLCAiJXMiLCBncC0+ bmFtZSk7CglwcC0+bWVkaWFzaXplID0gMTE1MjkyMTUwNDYwNjg0Njk3NkxMVTsKCXBwLT5z ZWN0b3JzaXplID0gNTEyOwoJZ19lcnJvcl9wcm92aWRlcihwcCwgMCk7CgkKCS8qIFRyaWdn ZXIgcGFuaWMgKi8KCWVycm9yID0gYWxxX29wZW5fZmxhZ3MoJmFscSwgICIvdmFyL2xvZy96 ZXJvLmFscS5sb2ciLAoJICAgIGN1cnRocmVhZC0+dGRfdWNyZWQsIEFMUV9ERUZBVUxUX0NN T0RFLAoJICAgIDE2LCBBTFFfT1JERVJFRCk7CgkgICAgCglpZiAoIWVycm9yKQoJCWFscV9j bG9zZShhbHEpOwp9CgpzdGF0aWMgaW50CmdfYWxxX2Rlc3Ryb3lfZ2VvbShzdHJ1Y3QgZ2N0 bF9yZXEgKnJlcSBfX3VudXNlZCwgc3RydWN0IGdfY2xhc3MgKm1wIF9fdW51c2VkLAogICAg c3RydWN0IGdfZ2VvbSAqZ3ApCnsKCXN0cnVjdCBnX3Byb3ZpZGVyICpwcDsKCglnX3RvcG9s b2d5X2Fzc2VydCgpOwoJaWYgKGdwID09IE5VTEwpCgkJcmV0dXJuICgwKTsKCXBwID0gTElT VF9GSVJTVCgmZ3AtPnByb3ZpZGVyKTsKCWlmIChwcCA9PSBOVUxMKQoJCXJldHVybiAoMCk7 CglpZiAocHAtPmFjciA+IDAgfHwgcHAtPmFjdyA+IDAgfHwgcHAtPmFjZSA+IDApCgkJcmV0 dXJuIChFQlVTWSk7CglnX3dpdGhlcl9nZW9tKGdwLCBFTlhJTyk7CglyZXR1cm4gKDApOwp9 CgpzdGF0aWMgc3RydWN0IGdfY2xhc3MgZ19hbHFfY2xhc3MgPSB7CgkubmFtZSA9IEdfWkVS T19DTEFTU19OQU1FLAoJLnZlcnNpb24gPSBHX1ZFUlNJT04sCgkuaW5pdCA9IGdfYWxxX2lu aXQsCgkuZGVzdHJveV9nZW9tID0gZ19hbHFfZGVzdHJveV9nZW9tCn07CgpERUNMQVJFX0dF T01fQ0xBU1MoZ19hbHFfY2xhc3MsIGdfYWxxKTsKTU9EVUxFX0RFUEVORChnX2FscSwgYWxx LCAxLCAxLCAxKTsK ------------D513F1DB29DF10BB Content-Type: application/octet-stream; name=Makefile Content-transfer-encoding: base64 Content-Disposition: attachment; filename=Makefile Q0xBU1M9CWFscQoKS0VSTkJVSUxEUk9PVCE9CW1ha2UgLUMgL3Vzci9zcmMvc3lzIC1WIC5P QkpESVIKS0VSTk5BTUUhPQkJdW5hbWUgLWkKLmlmIGV4aXN0cygke0tFUk5CVUlMRFJPT1R9 LyR7S0VSTk5BTUV9L29wdF9nbG9iYWwuaCkgJiYgIWRlZmluZWQoS0VSTkJVSUxERElSKQpL RVJOQlVJTERESVI6PSR7S0VSTkJVSUxEUk9PVH0vJHtLRVJOTkFNRX0KLmVuZGlmCgpLTU9E RElSPz0JL2Jvb3QvbW9kdWxlcwoKS01PRD0JZ2VvbV8ke0NMQVNTfQpTUkNTPQlnXyR7Q0xB U1N9LmMKCi5pbmNsdWRlIDxic2Qua21vZC5taz4K ------------D513F1DB29DF10BB-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 14:58:06 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81D6E1065672 for ; Mon, 3 Oct 2011 14:58:06 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 15F3E8FC08 for ; Mon, 3 Oct 2011 14:58:05 +0000 (UTC) Received: by ywp17 with SMTP id 17so4426729ywp.13 for ; Mon, 03 Oct 2011 07:58:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=A8yW7Qu12/qi+p0wf+m1Bvv3yiVosdtw1zfVVWKinWI=; b=myIXKwO7N9jvSIBarWrZ0R4RUMnDy3DMcTatcWNAqy+/efbNE7dgEFfeC/v9LX5+Hx k2Vl7ut4JW4v8cLrbdnNHe41k+oU853Jemrp/+GOf9aTELFcA9JpfpdgApjAvJY3eM/Q SQtiZu87vWadL+kis1qj44ISiP8gG/lLOkcio= MIME-Version: 1.0 Received: by 10.236.184.198 with SMTP id s46mr300505yhm.23.1317653885260; Mon, 03 Oct 2011 07:58:05 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.102.147 with HTTP; Mon, 3 Oct 2011 07:58:04 -0700 (PDT) In-Reply-To: <4E899C8E.7040305@fsn.hu> References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> Date: Mon, 3 Oct 2011 07:58:04 -0700 X-Google-Sender-Auth: ps670i5IGhsLfiZv1udT2enTwv0 Message-ID: From: Artem Belevich To: Attila Nagy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 14:58:06 -0000 > For me, the bug is still here: > $ uname -a > FreeBSD b 8.2-STABLE FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 > =A0 root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT =A0amd64 > $ df -h /tmp > Filesystem =A0 =A0Size =A0 =A0Used =A0 Avail Capacity =A0Mounted on > tmpfs =A0 =A0 =A0 =A0 =A0 0B =A0 =A0 =A00B =A0 =A0 =A00B =A0 100% =A0 =A0= /tmp > > I have no swap configured. The machine has 64 GB RAM. > vm.kmem_size=3D60G; vfs.zfs.arc_max=3D55G; vfs.zfs.arc_min=3D20G I'm curious -- does your ARC size ever reaches configured limit of 55G? My hunch that it's probably hovering around some noticeably lower number. On my ZFS setups a lot of memory seems to be lost due to fragmentation. On a system with 24G of RAM and rac_max=3D16G, I typically see more than 20G of memory wired. With kmem_size=3D60G, ARC is likely to use up most of available kmem space and that's probably what affects tmpfs. Besides, with kmem_size that close to arc_max you may be risking meeting "kmem too small" panic, though, considering that your kmem_size is rather large chances of that are smaller than on a system with smaller amounts of memory and kmem_size. I'd start with doubling kmem_size and, possibly, reducing arc_max to the point where it stops putting pressure on tmpfs. --Artem From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 16:20:53 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C626B106567C for ; Mon, 3 Oct 2011 16:20:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 4C6AC8FC0C for ; Mon, 3 Oct 2011 16:20:52 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p93G1jtH067234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 3 Oct 2011 19:01:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p93G1jlN073675; Mon, 3 Oct 2011 19:01:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p93G1jFg073674; Mon, 3 Oct 2011 19:01:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 3 Oct 2011 19:01:45 +0300 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20111003160145.GP1511@deviant.kiev.zoral.com.ua> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mxmMT58/z/5uKrU0" Content-Disposition: inline In-Reply-To: <1223820108.20111003161859@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Lawrence Stewart , fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 16:20:53 -0000 --mxmMT58/z/5uKrU0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 03, 2011 at 04:18:59PM +0400, Lev Serebryakov wrote: > Hello, Lawrence. > You wrote 3 =CF=CB=D4=D1=C2=D2=D1 2011 =C7., 13:57:04: >=20 > > I know nothing about VFS, but wonder if it's something to do with the > > credentials you pass in? > They looks Ok. Not NULL, at least :) I'm passing > "curtrhead->td_ucred". >=20 > > Lev, are you able to share your code with us? > Yes, of course. Minimal code, which trigger this error, attached. > Build, load module and PANIC! >=20 > This is geom_zero module, which try to create ALQ with name > "/var/log/zero.alq.log" on it load (not creation! So, you don't need > even create such GEOM!). Please note, that "init" callback of GEOM > class is called in g_event GEOM thread. >=20 > Code is for 9.0/10.0 >=20 > My code is rewritten with usage of working thread already, and it > works without any panic. But worker thread looks like overkill for > simple case "write 24 bytes record to ALQ for each BIO" to me. >=20 > So I could prove, that this code PANIC because alq_open_flags() is > called from one of three GEOM threads. Look at the body of e.g. g_io_schedule_down(), the THREAD_NO_SLEEPING(); call right before the call to geom start method. Basically, you cannot use any kernel subsystem in the context of the up/down threads that could sleep. This includes VFS, VM and everything that is layered on top of it. The decision is deliberate. Up and down shall only schedule the processing, or perform the fast processing. BTW, it must be obvious that trying to perform any VFS operation from geom up or down causes deadlock. --mxmMT58/z/5uKrU0 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6J3GkACgkQC3+MBN1Mb4iblQCfWiHwQSBUgtuilPBx2j4bjEEU /4MAn16LlHk78mVV88fgIEaw+aX/pBYy =igD2 -----END PGP SIGNATURE----- --mxmMT58/z/5uKrU0-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 16:27:38 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B51811065673; Mon, 3 Oct 2011 16:27:38 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4E9848FC13; Mon, 3 Oct 2011 16:27:38 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 6F18D4AC1C; Mon, 3 Oct 2011 20:27:36 +0400 (MSD) Date: Mon, 3 Oct 2011 20:27:28 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1298204673.20111003202728@serebryakov.spb.ru> To: Kostik Belousov In-Reply-To: <20111003160145.GP1511@deviant.kiev.zoral.com.ua> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> <20111003160145.GP1511@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: Lawrence Stewart , fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 16:27:38 -0000 Hello, Kostik. You wrote 3 =CF=CB=D4=D1=C2=D2=D1 2011 =C7., 20:01:45: > Look at the body of e.g. g_io_schedule_down(), the > THREAD_NO_SLEEPING(); > call right before the call to geom start method. > Basically, you cannot use any kernel subsystem in the context of > the up/down threads that could sleep. This includes VFS, VM and > everything that is layered on top of it. VM too? Allocating of new element to put into working queue is prohibited? It is what (almost) every GEOM class does. Even geom_nop() allocate new struct bio from g_down thread. > The decision is deliberate. Up and down shall only schedule the processin= g, > or perform the fast processing. Schedule is impossible without memory allocations in most cases... > BTW, it must be obvious that trying to perform any VFS operation from geom > up or down causes deadlock. Could cause. alq uses its own thread for "real" VFS operations, and open() was called from g_event, not g_up/g_down... BTW, nothing stops me from calling msleep() in g_event thread (to wait when working thread stops on exit, for example). And, again, it is what many "base" geom classes uses, even if they use working thread concept. And panic due to NULL pointer derefernce is not best way to indicate, that logic is broken (but it is not broken in this case, as it is impossible to create consumer for occupied geom with file system) :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 16:31:06 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BDAF1065670; Mon, 3 Oct 2011 16:31:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6FCAF8FC1B; Mon, 3 Oct 2011 16:31:04 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p93GV1XU080742 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 3 Oct 2011 19:31:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p93GV0Qi073819; Mon, 3 Oct 2011 19:31:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p93GV06g073818; Mon, 3 Oct 2011 19:31:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 3 Oct 2011 19:31:00 +0300 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20111003163100.GT1511@deviant.kiev.zoral.com.ua> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> <20111003160145.GP1511@deviant.kiev.zoral.com.ua> <1298204673.20111003202728@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="n+Pw2xyN35UiJQWs" Content-Disposition: inline In-Reply-To: <1298204673.20111003202728@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Lawrence Stewart , fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 16:31:06 -0000 --n+Pw2xyN35UiJQWs Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 03, 2011 at 08:27:28PM +0400, Lev Serebryakov wrote: > Hello, Kostik. > You wrote 3 =CF=CB=D4=D1=C2=D2=D1 2011 =C7., 20:01:45: >=20 > > Look at the body of e.g. g_io_schedule_down(), the > > THREAD_NO_SLEEPING(); > > call right before the call to geom start method. >=20 > > Basically, you cannot use any kernel subsystem in the context of > > the up/down threads that could sleep. This includes VFS, VM and > > everything that is layered on top of it. > VM too? Allocating of new element to put into working queue is > prohibited? It is what (almost) every GEOM class does. > Even geom_nop() allocate new struct bio from g_down thread. Sleeping (and thus non-failing) allocation is prohibited. --n+Pw2xyN35UiJQWs Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6J40QACgkQC3+MBN1Mb4jbxACdGA87XkTHUnq29kstAeQqLaR8 V6sAoNMMc/eMzTJ+xOtAJboXOdt8qu7S =SnrY -----END PGP SIGNATURE----- --n+Pw2xyN35UiJQWs-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 17:14:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 791551065672; Mon, 3 Oct 2011 17:14:02 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5916F8FC08; Mon, 3 Oct 2011 17:14:02 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 756A415E79; Mon, 3 Oct 2011 10:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1317662042; bh=OEDOwC1I3JiPO0QnCuQ5OZyliUTtjxYnkMAmVyKMVEI=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=JKdgJeC+MO3Q9oyct/m821uDY9Nhta+AWRduq4YrZA2pK6iM69IemtdUqlBwsWVgN OiIq5e7A4+MviHkGKIK941mHsOQ8q9D+73Mn3Rmt5G8mR4hvYRMsaKIhKg1oL0Rri5 T/tCe2pAXVrxeT9jqu8FlIvoD+fyv7Z+I1tZwgLU= Message-ID: <4E89ED58.6020104@delphij.net> Date: Mon, 03 Oct 2011 10:14:00 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Attila Nagy References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> In-Reply-To: <4E899C8E.7040305@fsn.hu> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, delphij@freebsd.org, Adrian Chadd Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 17:14:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Attila, On 10/03/11 04:29, Attila Nagy wrote: > For me, the bug is still here: $ uname -a FreeBSD b 8.2-STABLE > FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 > root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 $ df > -h /tmp Filesystem Size Used Avail Capacity Mounted on > tmpfs 0B 0B 0B 100% /tmp > > I have no swap configured. The machine has 64 GB RAM. > vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G This sounds like a configuration issue. Running without swap is not recommended anyways. I'm setting up an environment to validate if Ivan's reported issue still exists. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOie1XAAoJEATO+BI/yjfB/bcIAKTWQtOu4HMduSZtV2n6knuI YbjuDQhLr6RcKzpHqXpOUJNfa6YX5c52R2hpD6GeLE/2+Rxsb5qyAU737GF9ZbPy 6V2u7rcQXuMdwUgLfjdqP8UXFWkDCVuBpWagc3Kyzs7+ebFAHEYZ7OTYXyDKQzsm 6anl6p0BXLYHEzTfKuwlogaTs+jWOLBgWk7GDqtt6szwztYz0zWfhpJZashBZDJP W8n928B1IPQYOcFIee9Es5l0DnzzcHkB3T92A2jCof94cq+QLkP0F3KW8AUx7X5D 6a28DaqV8brqthU5BkZ1NYQXcqhTmwirlXDc2+/bmagGgM194QY9B30ELBZnpe4= =dFop -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 18:34:00 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D86A106566C; Mon, 3 Oct 2011 18:34:00 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id A2BBD8FC23; Mon, 3 Oct 2011 18:33:59 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 4AED5A8E4A5; Mon, 3 Oct 2011 20:33:58 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 19.6271] X-CRM114-CacheID: sfid-20111003_20335_5B6892E0 X-CRM114-Status: Good ( pR: 19.6271 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Oct 3 20:33:58 2011 X-DSPAM-Confidence: 0.9971 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4e8a0016584931011434026 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, FreeBSD, 0.00054, FreeBSD, 0.00054, >+I, 0.00087, >+On, 0.00099, >+the, 0.00137, to+>, 0.00171, to+>, 0.00171, bug, 0.00198, of+>, 0.00236, cache, 0.00256, wrote+>>, 0.00267, >+and, 0.00279, )+>, 0.00279, References*mail.gmail.com>, 0.00295, References*mail.gmail.com>, 0.00295, I+>, 0.00341, >+that, 0.00361, >+of, 0.00361, >>+>>, 0.00409, In-Reply-To*mail.gmail.com>, 0.00454, root, 0.00454, wrote, 0.00495, STABLE, 0.00556, STABLE, 0.00556, /tmp, 0.00612, X-Spambayes-Classification: ham; 0.00 Received: from [192.168.3.2] (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 53A02A8E498; Mon, 3 Oct 2011 20:33:57 +0200 (CEST) Message-ID: <4E8A0013.4070008@fsn.hu> Date: Mon, 03 Oct 2011 20:33:55 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Artem Belevich References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 18:34:00 -0000 On 10/03/2011 04:58 PM, Artem Belevich wrote: >> For me, the bug is still here: >> $ uname -a >> FreeBSD b 8.2-STABLE FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 >> root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 >> $ df -h /tmp >> Filesystem Size Used Avail Capacity Mounted on >> tmpfs 0B 0B 0B 100% /tmp >> >> I have no swap configured. The machine has 64 GB RAM. >> vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G > I'm curious -- does your ARC size ever reaches configured limit of > 55G? My hunch that it's probably hovering around some noticeably lower > number. Yes, in some minutes. Current counters: kstat.zfs.misc.arcstats.c_min: 21474836480 kstat.zfs.misc.arcstats.c_max: 59055800320 kstat.zfs.misc.arcstats.size: 45691792856 > On my ZFS setups a lot of memory seems to be lost due to > fragmentation. On a system with 24G of RAM and rac_max=16G, I > typically see more than 20G of memory wired. > With kmem_size=60G, ARC is likely to use up most of available kmem > space and that's probably what affects tmpfs. Besides, with kmem_size > that close to arc_max you may be risking meeting "kmem too small" > panic, though, considering that your kmem_size is rather large chances > of that are smaller than on a system with smaller amounts of memory > and kmem_size. Sounds plausible. BTW, it may be possible that the ARC limits are not needed anymore, they are here from the times, where on a 64 GB machine ARC hovered around 2-5 GBs without setting these (arc_min was even higher then). BTW, the user space programs fit into around 1-2 GB RAM on this machine typically. Well, most of the time. :) > I'd start with doubling kmem_size and, possibly, reducing arc_max to > the point where it stops putting pressure on tmpfs. > I know there are several differences, but it would be very good to have similar behaviour with UFS. I guess it's quite evident that tmpfs can eat the file system cache, and I know it may be not so trivial to solve this with ZFS. :) Will try it, thanks. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 18:36:52 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3498106564A; Mon, 3 Oct 2011 18:36:52 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 333378FC0A; Mon, 3 Oct 2011 18:36:51 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id DBD91A8E5C7; Mon, 3 Oct 2011 20:36:50 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 14.4434] X-CRM114-CacheID: sfid-20111003_20365_36106619 X-CRM114-Status: Good ( pR: 14.4434 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Oct 3 20:36:50 2011 X-DSPAM-Confidence: 0.9960 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4e8a00c2599153750912134 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, FreeBSD, 0.00054, FreeBSD, 0.00054, >+On, 0.00099, >+Hi, 0.00171, wrote+>, 0.00179, bug, 0.00198, wrote+>>, 0.00267, >+>, 0.00288, >+>, 0.00288, References*mail.gmail.com>, 0.00295, References*mail.gmail.com>, 0.00295, not+>, 0.00383, >>+>>, 0.00409, root, 0.00454, wrote, 0.00495, wrote, 0.00495, STABLE, 0.00556, STABLE, 0.00556, /tmp, 0.00612, /tmp, 0.00612, FreeBSD+8, 0.00612, Filesystem, 0.00679, Capacity+Mounted, 0.00679, Capacity, 0.00679, Avail+Capacity, 0.00679, X-Spambayes-Classification: ham; 0.00 Received: from [192.168.3.2] (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 4FAFCA8E5AC; Mon, 3 Oct 2011 20:36:49 +0200 (CEST) Message-ID: <4E8A00BF.5000508@fsn.hu> Date: Mon, 03 Oct 2011 20:36:47 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: d@delphij.net References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> <4E89ED58.6020104@delphij.net> In-Reply-To: <4E89ED58.6020104@delphij.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , freebsd-fs@freebsd.org, delphij@freebsd.org, Xin LI Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 18:36:53 -0000 On 10/03/2011 07:14 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi, Attila, > > On 10/03/11 04:29, Attila Nagy wrote: >> For me, the bug is still here: $ uname -a FreeBSD b 8.2-STABLE >> FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 >> root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 $ df >> -h /tmp Filesystem Size Used Avail Capacity Mounted on >> tmpfs 0B 0B 0B 100% /tmp >> >> I have no swap configured. The machine has 64 GB RAM. >> vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G > This sounds like a configuration issue. Running without swap is not > recommended anyways. I guess it depends on the workload. In general, you are possibly right, but if you need predictable response times and you can tolerate dying processes, swapping may not be the right thing to do. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 18:37:19 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EA3D106566C for ; Mon, 3 Oct 2011 18:37:19 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id BD2EE8FC08 for ; Mon, 3 Oct 2011 18:37:18 +0000 (UTC) Received: by gyf2 with SMTP id 2so4719780gyf.13 for ; Mon, 03 Oct 2011 11:37:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=TkaN6FYArBBBOffnlJrhRvrOoW62uoiMzWNUkqCDqeo=; b=KObN0Y/etQUX1oJtOhmOGwJWeMRTaubEqmthEc6ItZemojBv7qz7Crf9f9FFNa++27 u0EoX/H2fSqwhtC9dHt9Vbq10JZQYEwpPKdrmx3hz0cUHUJIqWCMJ4ZjDaJgRoo04hAK 1La1scBaZzBbY9j1tYDiPgFqUq+TS4SqXmbPg= MIME-Version: 1.0 Received: by 10.150.91.2 with SMTP id o2mr465471ybb.107.1317667038190; Mon, 03 Oct 2011 11:37:18 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Mon, 3 Oct 2011 11:37:18 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Mon, 3 Oct 2011 11:37:18 -0700 Message-ID: From: Xin LI To: Ivan Voras Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 18:37:19 -0000 On Sun, Oct 2, 2011 at 3:33 PM, Ivan Voras wrote: > On 02/10/2011 07:27, Xin LI wrote: > >> The problem Ivan have asserted was not confirmed by anyone who have >> swap configured properly. =C2=A0Gleb have pointed out that it might be >> related to a series of integer overflow by the way (he have also fixed >> a lot of tmpfs issues by the way). > > Well, instead of guessing I can point you to the way I got the original > situation - you said you have ZFS configured so it would be easy for you > to check. > > You should to something like this: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0- configure your system to the best of your ab= ilities (but post what > you did different from the defaults) It's mostly "normal" configuration -- 6GB RAM, 14GB swap, ZFS as /usr; The following sysctl tweaks were done to make postgresql start (these are not scientific, just large enough to make it work): kern.ipc.shmmax=3D4294967296 kern.ipc.shmall=3D1048576 > =C2=A0 =C2=A0 =C2=A0 =C2=A0- install postgresql (8.4+, but I don't think = the version is > important), configure it so it gets half the system memory or 2/3 the > system memory for its shared_buffers. Configured to half of system memory (3072MB). # ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 65536 5432001 --rw------- pgsql pgsql pgsql pgsql 6 3295936512 2130 2130 10:57:12 11:09:17 10:57:12 Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME s 65536 5432001 --rw------- pgsql pgsql pgsql pgsql 17 11:09:25 10:57:12 s 65537 5432002 --rw------- pgsql pgsql pgsql pgsql 17 10:57:12 10:57:12 > =C2=A0 =C2=A0 =C2=A0 =C2=A0- install pgbench Done. > =C2=A0 =C2=A0 =C2=A0 =C2=A0- initialize pgbench so that the database defi= nitely is larger than the > entire memory you got (NOTE: THIS IS NOT A CONTRIVED TEST - lots of > databases in practice are larger than the RAM in the server). Initialized with pgbench -i -s 500 pgbench. > =C2=A0 =C2=A0 =C2=A0 =C2=A0- run pgbench & observe the results. pgbench -t 20000 -c 64 -S pgbench Can't seem to reproduce: # df -Httmpfs Filesystem Size Used Avail Capacity Mounted on tmpfs 10G 69k 10G 0% /tmp Any suggestions? > This should create really bad contention problem for memory between > postgresql and ZFS, which should manifest itself in tmpfs shrinking to 0 > bytes free. > > If you don't get this problem then great, it might be solved! > > > (for more info on pgbench, see this: > http://www.westnet.com/~gsmith/content/postgresql/pgbench-scaling.htm). > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > --=20 Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 18:43:19 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14C22106564A; Mon, 3 Oct 2011 18:43:19 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id E79668FC16; Mon, 3 Oct 2011 18:43:18 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 3E4CF152C7; Mon, 3 Oct 2011 11:43:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1317667398; bh=ZSulABkk4B9lxlrywwn4vQgHlaktCgtKYWYENrQnbfE=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=HbDmbiI4xSCCdDzj87d4ALGQeF/qi19newYz7/3hnUxyVv7PSpzju1ESudwlj/JEm dFxncWbir0c6mvxgSazE0pk3MMaEIPWaH8YkMOkwSt5BI7A0I9lqlS0VhNrFfaTGOu EwnlHkNyvBfyIHjktBx+BTQjv5ts7t/2YwZDPmRU= Message-ID: <4E8A0232.1020005@delphij.net> Date: Mon, 03 Oct 2011 11:42:58 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Attila Nagy References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> <4E89ED58.6020104@delphij.net> <4E8A00BF.5000508@fsn.hu> In-Reply-To: <4E8A00BF.5000508@fsn.hu> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , freebsd-fs@freebsd.org, delphij@freebsd.org, d@delphij.net Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 18:43:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/03/11 11:36, Attila Nagy wrote: > On 10/03/2011 07:14 PM, Xin LI wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >> >> Hi, Attila, >> >> On 10/03/11 04:29, Attila Nagy wrote: >>> For me, the bug is still here: $ uname -a FreeBSD b 8.2-STABLE >>> FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 >>> root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 $ >>> df -h /tmp Filesystem Size Used Avail Capacity Mounted >>> on tmpfs 0B 0B 0B 100% /tmp >>> >>> I have no swap configured. The machine has 64 GB RAM. >>> vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G >> This sounds like a configuration issue. Running without swap is >> not recommended anyways. > I guess it depends on the workload. In general, you are possibly > right, but if you need predictable response times and you can > tolerate dying processes, swapping may not be the right thing to > do. Well, in that case one will have to tolerate tmpfs have no page to allocate (note that it does need to reserve some pages for system use) at this point. Currently it's working like a credit line -- use it here and you can't use it elsewhere, e.g. if user process used it then tmpfs can't use it... I think the only solution we can have here is to teach tmpfs about "dedicated reserve for tmpfs" so it pre-allocate a few pages that can not be reused elsewhere? E.g. create a budget that make this part of memory "tmpfs fund" :) Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOigIyAAoJEATO+BI/yjfBVPwH/2iXNPkTmgzN0pxoB/Xhrw96 TYeJkDLevVuRTVi82VDuCqrUtJeMsf0rSLdNS201AZc6+7e/rPzBWVaKSKowH9BP vY67TnbSMgG6UEnVrVZ29JMy7RHoWDapqEd5+kYxVOwi+jX8tzh+HZsqgWUjOe+F mSvv0lcvEgGyeFkPD0HoJni73noXKgumd6Eben7hN+CAwYkOWlWLYKZiHTyFqlWK To41Bg9je3FUgL95RJpoMvvO4GVRWaU95UyarMuM1ulh7WKe8/JW8QEcDaVTtABJ QIlAROPmkeagqjkHWaprAz46sJhFczShSr6DX2unhsGGCPq83eUWkOip8GDdQw4= =l3Ji -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 18:50:23 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A70EB106566C; Mon, 3 Oct 2011 18:50:23 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3A8DC8FC08; Mon, 3 Oct 2011 18:50:22 +0000 (UTC) Received: by yxk36 with SMTP id 36so4806688yxk.13 for ; Mon, 03 Oct 2011 11:50:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=WK6EHV9rfPYThkhy2JZZhGH/TMrDe2Kv/YHgmZf2Ovg=; b=HRwAFJwEue+nn5UVt2XM+V5alRYp8x/VCJ3O6fK8SYfFzm/HmUn27HPvMpWqYc3hv+ JZJd8/xFKpjPMyPHFeOBgN0W3ZVVvZ8PEkRYMinHOL5ezATYBG1J8bv3lw2on3O3iYep gZF/rpWxmAkaPjwBWGjiwl9OOQIEuWxxcYX9Y= MIME-Version: 1.0 Received: by 10.236.185.37 with SMTP id t25mr1513595yhm.131.1317667822440; Mon, 03 Oct 2011 11:50:22 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Mon, 3 Oct 2011 11:50:22 -0700 (PDT) In-Reply-To: <4E8A0013.4070008@fsn.hu> References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> <4E8A0013.4070008@fsn.hu> Date: Mon, 3 Oct 2011 11:50:22 -0700 X-Google-Sender-Auth: HzRjBDkGPpOTJx_3decdyz5HLHU Message-ID: From: Artem Belevich To: Attila Nagy Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, Adrian Chadd , delphij@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 18:50:23 -0000 On Mon, Oct 3, 2011 at 11:33 AM, Attila Nagy wrote: > Sounds plausible. BTW, it may be possible that the ARC limits are not needed > anymore, they are here from the times, where on a 64 GB machine ARC hovered > around 2-5 GBs without setting these (arc_min was even higher then). You do need tuning. FreeBSD defaults are way too conservative for ZFS. While they may get you a working ZFS config, you will end up with ARC using only small fraction of your memory, which is probably not the best setup for a filesrever. > BTW, the user space programs fit into around 1-2 GB RAM on this machine > typically. Well, most of the time. :) Yup. 2GB for apps, 2-5GB for ZFS ARC and you end up with 50+GB of memory that's effectively wasted. The trick is in finding the balance where ZFS uses as much memory as you can give it without causing trouble for other subsystems. ARC back-pressure mechanism works somewhat better now than it used to, but it's still very far from perfect. Ideally it would be nice to integrate ARC with the FreeBSD's unified cache, but that's not a trivial task, as far as I can tell. --Artem >> >> I'd start with doubling kmem_size and, possibly, reducing arc_max to >> the point where it stops putting pressure on tmpfs. >> > I know there are several differences, but it would be very good to have > similar behaviour with UFS. I guess it's quite evident that tmpfs can eat > the file system cache, and I know it may be not so trivial to solve this > with ZFS. :) > > Will try it, thanks. > From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 19:10:47 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7F1D106564A; Mon, 3 Oct 2011 19:10:46 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 195958FC16; Mon, 3 Oct 2011 19:10:45 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 9E763A8E288; Mon, 3 Oct 2011 21:10:44 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 20.9654] X-CRM114-CacheID: sfid-20111003_21104_C19F9C6D X-CRM114-Status: Good ( pR: 20.9654 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Oct 3 21:10:44 2011 X-DSPAM-Confidence: 0.9969 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4e8a08b4761682717815091 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, FreeBSD, 0.00054, FreeBSD, 0.00054, >+I, 0.00087, >+On, 0.00099, to+>, 0.00171, wrote+>, 0.00179, bug, 0.00198, of+>, 0.00236, cache, 0.00256, cache, 0.00256, wrote+>>, 0.00267, it+>, 0.00279, )+>, 0.00279, >+>, 0.00288, >+>, 0.00288, References*mail.gmail.com>, 0.00295, References*mail.gmail.com>, 0.00295, root, 0.00454, wrote, 0.00495, wrote, 0.00495, it+does, 0.00510, STABLE, 0.00556, STABLE, 0.00556, /tmp, 0.00612, /tmp, 0.00612, X-Spambayes-Classification: ham; 0.00 Received: from [192.168.3.2] (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 011CAA8E26A; Mon, 3 Oct 2011 21:10:42 +0200 (CEST) Message-ID: <4E8A08B2.1010403@fsn.hu> Date: Mon, 03 Oct 2011 21:10:42 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: d@delphij.net References: <20111002020231.GA70864@icarus.home.lan> <4E899C8E.7040305@fsn.hu> <4E89ED58.6020104@delphij.net> <4E8A00BF.5000508@fsn.hu> <4E8A0232.1020005@delphij.net> In-Reply-To: <4E8A0232.1020005@delphij.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , freebsd-fs@freebsd.org, delphij@freebsd.org, Xin LI Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 19:10:47 -0000 On 10/03/2011 08:42 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 10/03/11 11:36, Attila Nagy wrote: >> On 10/03/2011 07:14 PM, Xin LI wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >>> >>> Hi, Attila, >>> >>> On 10/03/11 04:29, Attila Nagy wrote: >>>> For me, the bug is still here: $ uname -a FreeBSD b 8.2-STABLE >>>> FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST 2011 >>>> root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 $ >>>> df -h /tmp Filesystem Size Used Avail Capacity Mounted >>>> on tmpfs 0B 0B 0B 100% /tmp >>>> >>>> I have no swap configured. The machine has 64 GB RAM. >>>> vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G >>> This sounds like a configuration issue. Running without swap is >>> not recommended anyways. >> I guess it depends on the workload. In general, you are possibly >> right, but if you need predictable response times and you can >> tolerate dying processes, swapping may not be the right thing to >> do. > Well, in that case one will have to tolerate tmpfs have no page to > allocate (note that it does need to reserve some pages for system use) > at this point. Currently it's working like a credit line -- use it > here and you can't use it elsewhere, e.g. if user process used it then > tmpfs can't use it... Sure, but we are talking here about ARC, which is a cache. I guess tmpfs work quite OK with UFS, where it will shrink its cache space. BTW, I don't care if tmpfs shows 2 GB free space when there is (>)2 GB, but currently I have: Mem: 564M Active, 3091M Inact, 55G Wired, 129M Cache, 71M Buf, 2957M Free and: Filesystem Size Used Avail Capacity Mounted on tmpfs 0B 0B 0B 100% /tmp That's the first problem. (the second could be to shrink ARC from tmpfs-land if needed) > I think the only solution we can have here is to teach tmpfs about > "dedicated reserve for tmpfs" so it pre-allocate a few pages that can > not be reused elsewhere? E.g. create a budget that make this part of > memory "tmpfs fund" :) > > There is md for that. Sure, it's harder, and using UFS (or any other FS) on md seems to be a waste of resources. For me it would be quite enough if tmpfs wouldn't loose all its space when there is free memory. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 20:09:34 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F878106566C for ; Mon, 3 Oct 2011 20:09:34 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id E67E28FC16 for ; Mon, 3 Oct 2011 20:09:33 +0000 (UTC) Received: by wyj26 with SMTP id 26so4817293wyj.13 for ; Mon, 03 Oct 2011 13:09:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=v+wrECGMQ/wvtWPw0FH89c24T35s4PDe6zmGiv6QkKs=; b=c+rmDcG8RBdSg9UZx4tz45qtPjiKyDad0cw49e6ugV2K/JVabHl2nVtAWLl1spmXP0 ePAwsrd8V3/S0e2DluBGrH7aY6495017qiKs+ZW5sMzjFa7P9moDRmBAjlcdcJGBLYsQ e4IHAHrxxgpA4bXoRHl58oBnNETuldWOG4ZSQ= Received: by 10.227.6.199 with SMTP id a7mr351176wba.74.1317670679086; Mon, 03 Oct 2011 12:37:59 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.180.107.130 with HTTP; Mon, 3 Oct 2011 12:37:19 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> From: Ivan Voras Date: Mon, 3 Oct 2011 21:37:19 +0200 X-Google-Sender-Auth: fZpGeGIJiYUVw5uw8GGU3Jt6Dgg Message-ID: To: Xin LI Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 20:09:34 -0000 On 3 October 2011 20:37, Xin LI wrote: > It's mostly "normal" configuration -- 6GB RAM, 14GB swap, ZFS as /usr; > Initialized with pgbench -i -s 500 pgbench. Ok, this should be enough ~~ 7 GB database on a 6 GB machine. >> =C2=A0 =C2=A0 =C2=A0 =C2=A0- run pgbench & observe the results. > > pgbench -t 20000 -c 64 -S pgbench > > Can't seem to reproduce: > > # df -Httmpfs > Filesystem =C2=A0 =C2=A0Size =C2=A0 =C2=A0Used =C2=A0 Avail Capacity =C2= =A0Mounted on > tmpfs =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A010G =C2=A0 =C2=A0 69k =C2=A0 =C2= =A0 10G =C2=A0 =C2=A0 0% =C2=A0 =C2=A0/tmp > > Any suggestions? Can't think of anything to add :) I don't think increasing the database size would affect it much. Comparing what you did to my original report at http://www.mail-archive.com/freebsd-current@freebsd.org/msg126491.html, it seems like that bug is gone! But my machine was one of the bigger ones (for that time) - AFAIK 24 GB RAM and many cores (16?). Maybe there is some race condition which is exposed on large machines. >> This should create really bad contention problem for memory between >> postgresql and ZFS, which should manifest itself in tmpfs shrinking to 0 >> bytes free. >> >> If you don't get this problem then great, it might be solved! I don't have the equipment now to test it myself but I agree that based on what you posted it looks solved. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 20:38:17 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9F1F106566B; Mon, 3 Oct 2011 20:38:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 853808FC1F; Mon, 3 Oct 2011 20:38:16 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p93KcCkx010413 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 3 Oct 2011 23:38:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p93KcCFA075383; Mon, 3 Oct 2011 23:38:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p93KcBC8075382; Mon, 3 Oct 2011 23:38:11 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 3 Oct 2011 23:38:11 +0300 From: Kostik Belousov To: Attilio Rao Message-ID: <20111003203811.GA1511@deviant.kiev.zoral.com.ua> References: <201110012137.p91Lb6FI093841@chez.mckusick.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="R/ry0oax4LN2sDNq" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Kirk McKusick , Garrett Cooper , Xin LI , freebsd-fs@freebsd.org Subject: Re: Need to force sync(2) before umounting UFS1 filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 20:38:17 -0000 --R/ry0oax4LN2sDNq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 02, 2011 at 02:19:32AM +0200, Attilio Rao wrote: > I'm sorry if it wasn't clear in kib/my latest message, but we don't > need the coveredvnode unlocking logic because of the tegge's commit. >=20 > I just think we should commit the change in policy Kirk initially > submitted + a comment on top of vfs_busy() explaining why the deadlock > with coveredvnode cannot happen. Below is my take on the comment. commit 3981acdadcf4313dbdf813ec107f7bfbb4057d09 Author: Konstantin Belousov Date: Mon Oct 3 23:33:06 2011 +0300 Move parts of the commit log for r166167, where Tor explained the interaction between vnode locks and vfs_busy(), into comment. diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 7eb619a..3d7735d 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -348,6 +348,38 @@ SYSINIT(vfs, SI_SUB_VFS, SI_ORDER_FIRST, vntblinit, NU= LL); /* * Mark a mount point as busy. Used to synchronize access and to delay * unmounting. Eventually, mountlist_mtx is not released on failure. + * + * vfs_busy() is a custom lock, it can block the caller. + * vfs_busy() only sleeps if the unmount is active on the mount point. + * For a mountpoint mp, vfs_busy-enforced lock is before lock of any + * vnode belonging to mp. + * + * Lookup uses vfs_busy() to traverse mount points. + * root fs var fs + * / vnode lock A / vnode lock (/var) D + * /var vnode lock B /log vnode lock(/var/log) E + * vfs_busy lock C vfs_busy lock F + * + * Within each file system, the lock order is C->A->B and F->D->E. + * + * When traversing across mounts, the system follows that lock order: + * + * C->A->B + * | + * +->F->D->E + * + * The lookup() process for namei("/var") illustrates the process: + * VOP_LOOKUP() obtains B while A is held + * vfs_busy() obtains a shared lock on F while A and B are held + * vput() releases lock on B + * vput() releases lock on A + * VFS_ROOT() obtains lock on D while shared lock on F is held + * vfs_unbusy() releases shared lock on F + * vn_lock() obtains lock on deadfs vnode vp_crossmp instead of A. + * Attempt to lock A (instead of vp_crossmp) while D is held would + * violate the global order, causing deadlocks. + * + * dounmount() locks B while F is drained. */ int vfs_busy(struct mount *mp, int flags) --R/ry0oax4LN2sDNq Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6KHTMACgkQC3+MBN1Mb4gslQCcCAs5ZP1DSVPTGu0tJZW1TKD8 9UEAnAtN0EbYJBdxVqSSe/Aja41Kqu25 =l+uN -----END PGP SIGNATURE----- --R/ry0oax4LN2sDNq-- From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 21:36:50 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56126106564A for ; Mon, 3 Oct 2011 21:36:50 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id ED3FD8FC12 for ; Mon, 3 Oct 2011 21:36:49 +0000 (UTC) Received: by qyk4 with SMTP id 4so4519436qyk.13 for ; Mon, 03 Oct 2011 14:36:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.2.209 with SMTP id 17mr424024qck.29.1317677807853; Mon, 03 Oct 2011 14:36:47 -0700 (PDT) Received: by 10.229.224.82 with HTTP; Mon, 3 Oct 2011 14:36:47 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Mon, 3 Oct 2011 23:36:47 +0200 Message-ID: From: Olivier Smedts To: Xin LI Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-fs@freebsd.org" , Ivan Voras Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 21:36:50 -0000 Le lundi 3 octobre 2011, Xin LI a =E9crit : > On Sun, Oct 2, 2011 at 3:33 PM, Ivan Voras wrote: >> On 02/10/2011 07:27, Xin LI wrote: >> >>> The problem Ivan have asserted was not confirmed by anyone who have >>> swap configured properly. Gleb have pointed out that it might be >>> related to a series of integer overflow by the way (he have also fixed >>> a lot of tmpfs issues by the way). >> >> Well, instead of guessing I can point you to the way I got the original >> situation - you said you have ZFS configured so it would be easy for you >> to check. >> >> You should to something like this: >> >> - configure your system to the best of your abilities (but post what >> you did different from the defaults) > > It's mostly "normal" configuration -- 6GB RAM, 14GB swap, ZFS as /usr; > > The following sysctl tweaks were done to make postgresql start (these > are not scientific, just large enough to make it work): > > kern.ipc.shmmax=3D4294967296 > kern.ipc.shmall=3D1048576 > >> - install postgresql (8.4+, but I don't think the version is >> important), configure it so it gets half the system memory or 2/3 the >> system memory for its shared_buffers. > > Configured to half of system memory (3072MB). > > # ipcs -a > Message Queues: > T ID KEY MODE OWNER GROUP CREATOR > CGROUP CBYTES QNUM > QBYTES LSPID LRPID STIME RTIME CTIME > > Shared Memory: > T ID KEY MODE OWNER GROUP CREATOR > CGROUP NATTCH SEGSZ CPID LPID ATIME > DTIME CTIME > m 65536 5432001 --rw------- pgsql pgsql pgsql > pgsql 6 3295936512 2130 2130 10:57:12 > 11:09:17 10:57:12 > > Semaphores: > T ID KEY MODE OWNER GROUP CREATOR > CGROUP NSEMS OTIME CTIME > s 65536 5432001 --rw------- pgsql pgsql pgsql > pgsql 17 11:09:25 10:57:12 > s 65537 5432002 --rw------- pgsql pgsql pgsql > pgsql 17 10:57:12 10:57:12 > >> - install pgbench > > Done. > >> - initialize pgbench so that the database definitely is larger than the >> entire memory you got (NOTE: THIS IS NOT A CONTRIVED TEST - lots of >> databases in practice are larger than the RAM in the server). > > Initialized with pgbench -i -s 500 pgbench. > >> - run pgbench & observe the results. > > pgbench -t 20000 -c 64 -S pgbench > > Can't seem to reproduce: > > # df -Httmpfs > Filesystem Size Used Avail Capacity Mounted on > tmpfs 10G 69k 10G 0% /tmp > > Any suggestions? Try reducing the swap size to less than the RAM size. No "configuration issue", try with some RAM + swap that should fit all. And please excuse if we're not speaking of the same tmpfs caveat :-) > >> This should create really bad contention problem for memory between >> postgresql and ZFS, which should manifest itself in tmpfs shrinking to 0 >> bytes free. >> >> If you don't get this problem then great, it might be solved! >> >> >> (for more info on pgbench, see this: >> http://www.westnet.com/~gsmith/content/postgresql/pgbench-scaling.htm). >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > > > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > --=20 Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 22:13:43 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E22941065670; Mon, 3 Oct 2011 22:13:43 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 888828FC12; Mon, 3 Oct 2011 22:13:43 +0000 (UTC) Received: by ywp17 with SMTP id 17so4918491ywp.13 for ; Mon, 03 Oct 2011 15:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MYdLhI2QNR1DQPo6gPIJ1KQJAoeBA6mXE7OG0v3s4oE=; b=I7dArEJDX68ZNfA23LgIZl9/KOd1x3y/99ic8rUZA1fEQDZO7XN0ocrkcRHpVpYSoW LLkFhz8RcOBDngT8DIGHhbNTdGJUo+Zxa53xi+rieavX/QIVdTHneGXumr5fq+ETxgcs Ba0fHd79jQQbIa3i+nGlMbhfdjqGYV0rafEnU= MIME-Version: 1.0 Received: by 10.151.93.10 with SMTP id v10mr618131ybl.449.1317680022804; Mon, 03 Oct 2011 15:13:42 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Mon, 3 Oct 2011 15:13:42 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Mon, 3 Oct 2011 15:13:42 -0700 Message-ID: From: Xin LI To: Olivier Smedts Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-fs@freebsd.org" , Ivan Voras Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 22:13:44 -0000 On Mon, Oct 3, 2011 at 2:36 PM, Olivier Smedts wrote: [...] > Try reducing the swap size to less than the RAM size. No "configuration > issue", try with some RAM + swap that should fit all. But it's not ZFS+tmpfs specific, it can happen anywhere when memory and swap is not sufficient. Of course tmpfs and ZFS should play more well together but it's pretty much a "you get what you paid for" situation IMHO. One thing I can not yet reproduce, but sounds like a serious issue is that when system have sufficient RAM available, ZFS reports 0 in free space... If there is a test case for that then that's definitely something we need to solve sooner. Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Mon Oct 3 22:28:06 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58EA81065673 for ; Mon, 3 Oct 2011 22:28:06 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 11F718FC08 for ; Mon, 3 Oct 2011 22:28:05 +0000 (UTC) Received: by ggeq3 with SMTP id q3so654834gge.13 for ; Mon, 03 Oct 2011 15:28:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=gVkJJPPKgNougPa7S8dtpE7KqbO3/fNbSZ8+AWiNn5A=; b=TJinu3zY+yt8BodtfdksRaamv5sIg6wg2rUtcdiOulvQLBbL2tNxwc1eHsz5SJi3TB ZUjshInyWjqk9DtkA/xuhH0pj/Kizejr36m/EPnoou/+kyP8OguMlsai3HMcGt+sSKEM Qz000cB8oousc1zyImVW3kmTZSU45UOxd/ExM= Received: by 10.100.233.21 with SMTP id f21mr454471anh.44.1317680885082; Mon, 03 Oct 2011 15:28:05 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.100.43.9 with HTTP; Mon, 3 Oct 2011 15:27:25 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> From: Ivan Voras Date: Tue, 4 Oct 2011 00:27:25 +0200 X-Google-Sender-Auth: wLHWkyxHV_QSN-dLrRDr8Q1drAs Message-ID: To: Xin LI Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-fs@freebsd.org" Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 22:28:06 -0000 On 4 October 2011 00:13, Xin LI wrote: > On Mon, Oct 3, 2011 at 2:36 PM, Olivier Smedts wrote: > [...] >> Try reducing the swap size to less than the RAM size. No "configuration >> issue", try with some RAM + swap that should fit all. > > But it's not ZFS+tmpfs specific, it can happen anywhere when memory > and swap is not sufficient. =C2=A0Of course tmpfs and ZFS should play mor= e > well together but it's pretty much a "you get what you paid for" > situation IMHO. That would be ok... > One thing I can not yet reproduce, but sounds like a serious issue is > that when system have sufficient RAM available, ZFS reports 0 in free > space... =C2=A0If there is a test case for that then that's definitely > something we need to solve sooner. Yes, this is the point of my original bug report - I thought that this is what you were going to test. Note that on that machine I have both free memory and free swap and still 0 bytes free in tmpfs. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 00:22:17 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CC9C106566C; Tue, 4 Oct 2011 00:22:17 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id C2F558FC0C; Tue, 4 Oct 2011 00:22:16 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 162E27E896; Tue, 4 Oct 2011 11:22:15 +1100 (EST) Message-ID: <4E8A51B6.8090108@freebsd.org> Date: Tue, 04 Oct 2011 11:22:14 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20110914 Thunderbird/6.0.2 MIME-Version: 1.0 To: Kostik Belousov References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> <20111003160145.GP1511@deviant.kiev.zoral.com.ua> In-Reply-To: <20111003160145.GP1511@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 00:22:17 -0000 On 10/04/11 03:01, Kostik Belousov wrote: > On Mon, Oct 03, 2011 at 04:18:59PM +0400, Lev Serebryakov wrote: >> Hello, Lawrence. >> You wrote 3 ÏËÔÑÂÒÑ 2011 Ç., 13:57:04: >> >>> I know nothing about VFS, but wonder if it's something to do with the >>> credentials you pass in? >> They looks Ok. Not NULL, at least :) I'm passing >> "curtrhead->td_ucred". >> >>> Lev, are you able to share your code with us? >> Yes, of course. Minimal code, which trigger this error, attached. >> Build, load module and PANIC! >> >> This is geom_zero module, which try to create ALQ with name >> "/var/log/zero.alq.log" on it load (not creation! So, you don't need >> even create such GEOM!). Please note, that "init" callback of GEOM >> class is called in g_event GEOM thread. >> >> Code is for 9.0/10.0 >> >> My code is rewritten with usage of working thread already, and it >> works without any panic. But worker thread looks like overkill for >> simple case "write 24 bytes record to ALQ for each BIO" to me. >> >> So I could prove, that this code PANIC because alq_open_flags() is >> called from one of three GEOM threads. > Look at the body of e.g. g_io_schedule_down(), the > THREAD_NO_SLEEPING(); > call right before the call to geom start method. Thanks Kostik, that's the context I was missing. > Basically, you cannot use any kernel subsystem in the context of > the up/down threads that could sleep. This includes VFS, VM and > everything that is layered on top of it. > > The decision is deliberate. Up and down shall only schedule the processing, > or perform the fast processing. > > BTW, it must be obvious that trying to perform any VFS operation from geom > up or down causes deadlock. So Lev, I believe what you can do is alq_open_flags() in a separate context (module init routine or a sysctl handler are good places) and then you should be able to use it in the sleep sensitive context, as ALQ is designed to work in such places if you pass the ALQ_NOWAIT flag to the other API calls. If you want an example, have a look at sys/netinet/siftr.c. The packet fast path is non-sleepable, so I create the ALQ when the enable sysctl is called (siftr_sysctl_enabled_handler -> siftr_manage_ops -> alq_open), although I defer the pkt processing work to a separate thread which does the alq_writes. You should be able to do the writes in the geom fast path without a separate thread if you prefer. Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 04:13:05 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 780A4106566B; Tue, 4 Oct 2011 04:13:05 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward7.mail.yandex.net (forward7.mail.yandex.net [IPv6:2a02:6b8:0:202::2]) by mx1.freebsd.org (Postfix) with ESMTP id DB59B8FC0A; Tue, 4 Oct 2011 04:13:04 +0000 (UTC) Received: from smtp6.mail.yandex.net (smtp6.mail.yandex.net [77.88.61.56]) by forward7.mail.yandex.net (Yandex) with ESMTP id 374421C1AF0; Tue, 4 Oct 2011 08:13:03 +0400 (MSD) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1317701583; bh=+QN6SQ/+67e06IUYzBntVV2taM+ijNRai8V2WAcRce0=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=AfS0w4LJzgL7X+QMPZLmkFYyrzYEsXhWni2KNP35cqzJ4F/vXCvY1awyUAMThTjSW LS0LtfYU1+SlfxDVOi3NEELQA6W8TY9JN5pgEdV9z3MRqjNJcm24H32YNQHaAC265f oNvB0f2MVSTLHouRU5B1mn+l6S0Zud64/jhqJkrc= Received: from smtp6.mail.yandex.net (localhost [127.0.0.1]) by smtp6.mail.yandex.net (Yandex) with ESMTP id 0D3D5164033A; Tue, 4 Oct 2011 08:13:03 +0400 (MSD) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1317701583; bh=+QN6SQ/+67e06IUYzBntVV2taM+ijNRai8V2WAcRce0=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=AfS0w4LJzgL7X+QMPZLmkFYyrzYEsXhWni2KNP35cqzJ4F/vXCvY1awyUAMThTjSW LS0LtfYU1+SlfxDVOi3NEELQA6W8TY9JN5pgEdV9z3MRqjNJcm24H32YNQHaAC265f oNvB0f2MVSTLHouRU5B1mn+l6S0Zud64/jhqJkrc= Received: from mail.kirov.so-ups.ru (mail.kirov.so-ups.ru [178.74.170.1]) by smtp6.mail.yandex.net (nwsmtp/Yandex) with ESMTP id D2u0o1O3-D2uCGiV9; Tue, 4 Oct 2011 08:13:02 +0400 X-Yandex-Spam: 1 Message-ID: <4E8A87CD.2080001@yandex.ru> Date: Tue, 04 Oct 2011 08:13:01 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: lev@FreeBSD.org References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> In-Reply-To: <1223820108.20111003161859@serebryakov.spb.ru> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: Lawrence Stewart , fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 04:13:05 -0000 On 03.10.2011 16:18, Lev Serebryakov wrote: > This is geom_zero module, which try to create ALQ with name > "/var/log/zero.alq.log" on it load (not creation! So, you don't need > even create such GEOM!). Please note, that "init" callback of GEOM > class is called in g_event GEOM thread. Hi, Lev Did you try just release the topology lock before alq_open and acquire it back before exit from _init method? -- WBR, Andrey V. Elsukov From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 04:22:01 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADF261065672 for ; Tue, 4 Oct 2011 04:22:01 +0000 (UTC) (envelope-from ari@ish.com.au) Received: from fish.ish.com.au (eth5921.nsw.adsl.internode.on.net [59.167.240.32]) by mx1.freebsd.org (Postfix) with ESMTP id 6FE8B8FC13 for ; Tue, 4 Oct 2011 04:22:01 +0000 (UTC) Received: from ip-136.ish.com.au ([203.29.62.136]:62217) by fish.ish.com.au with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76) (envelope-from ) id 1RAwL5-0000tW-12; Tue, 04 Oct 2011 15:10:44 +1100 X-CTCH-RefID: str=0001.0A150203.4E8A8744.000A:SCFSTAT15613948, ss=1, re=-4.000, fgs=0 Message-ID: <4E8A8740.100@ish.com.au> Date: Tue, 04 Oct 2011 15:10:40 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0) Gecko/20110916 Thunderbird/7.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 04:22:01 -0000 Having just suffered from a FreeBSD 8.2 machine hit with the "kmem_map too small" error [1], what is the current recommendation for a machine running RELEASE-8.2? A. vm.kmem_size_scale=1 (this is the new default in 8-STABLE but 8.2-RELEASE had it set to 3) B. vm.kmem_size_scale=0.66 (this is aligned with Pawel's recommendation of setting vm.kmem_size to 150% of your RAM. [2] C. Leaving vm.kmem_size_scale alone and manually setting vm.kmem_size directly as Pawel is doing Thanks Ari Maniatis [1] http://freebsd.1045724.n5.nabble.com/kmem-map-too-small-with-ZFS-and-8-2-RELEASE-td4029979.html [2] http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 04:56:11 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CD0A106566B for ; Tue, 4 Oct 2011 04:56:11 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id B03CD8FC08 for ; Tue, 4 Oct 2011 04:56:09 +0000 (UTC) Received: from omta20.westchester.pa.mail.comcast.net ([76.96.62.71]) by qmta10.westchester.pa.mail.comcast.net with comcast id gUsU1h0021YDfWL5AUwANZ; Tue, 04 Oct 2011 04:56:10 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta20.westchester.pa.mail.comcast.net with comcast id gUw81h00t1t3BNj3gUw9qh; Tue, 04 Oct 2011 04:56:10 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 3874B102C1C; Mon, 3 Oct 2011 21:56:07 -0700 (PDT) Date: Mon, 3 Oct 2011 21:56:07 -0700 From: Jeremy Chadwick To: Aristedes Maniatis Message-ID: <20111004045607.GA8611@icarus.home.lan> References: <4E8A8740.100@ish.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E8A8740.100@ish.com.au> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 04:56:11 -0000 On Tue, Oct 04, 2011 at 03:10:40PM +1100, Aristedes Maniatis wrote: > Having just suffered from a FreeBSD 8.2 machine hit with the "kmem_map too small" error [1], what is the current recommendation for a machine running RELEASE-8.2? > > A. vm.kmem_size_scale=1 (this is the new default in 8-STABLE but 8.2-RELEASE had it set to 3) > B. vm.kmem_size_scale=0.66 (this is aligned with Pawel's recommendation of setting vm.kmem_size to 150% of your RAM. [2] > C. Leaving vm.kmem_size_scale alone and manually setting vm.kmem_size directly as Pawel is doing > > [1] http://freebsd.1045724.n5.nabble.com/kmem-map-too-small-with-ZFS-and-8-2-RELEASE-td4029979.html > [2] http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration Even if you were to mess about with vm.kmem_size_scale (which IMO should be set to 1 -- it defaults to that in 8.2-STABLE as well as other upcoming releases), you will still experience problems with kmem map exhaustion **even if** you limit the ARC. There are fixes in 8.2-STABLE that address this problem (there were multiple fixes put in place). So if you're already experiencing kmem map exhaustion, I strongly recommend you upgrade to 8.2-STABLE and then adjust one single tunable: vfs.zfs.arc_max You didn't disclose how much RAM your machine has, nor what other daemons are running on it, so it's very hard to give you an estimate. I tend to tell people to set vfs.zfs.arc_max to about 60% of their memory. E.g. if the machine has 8GB RAM, set vfs.zfs.arc_max="5120M". We use this on all of our servers with 8GB, including ones that run mysqld with some tunings. Example machine in question: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 60030 mysql 16 76 0 785M 294M sigwai 0 15:38 0.00% [mysqld] And relevant bits from /boot/loader.conf: kern.maxdsiz="2560M" kern.dfldsiz="2560M" kern.maxssiz="256M" vfs.zfs.arc_max="5120M" You need to keep in mind that ZFS can still use more than what you limit the ARC to. The mailing list has explanations regarding this; I believe it has to do with fragmentation or something like that. So don't go thinking you're being smart by setting it to something like "7500M"; there is a very good chance you will still experience the same problem. So my advice is to "start small" and work your way up after multiple weeks (not days!) of utilising the filesystems on ZFS, to really stress the ARC. Also be aware there may still be problems with applications that use sendfile(2) -- known programs which use this are ftpd (not adjustable), Apache (you can disable it), nginx (I think you can disable it), and Samba (you can disable it). There are some sendfile(2) fixes were which committed, but I'm not 100% sure if all situations have been accounted for. P.S. -- I am assuming you're using amd64. If you're using i386 this whole situation becomes much, much more complex. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 08:03:40 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61459106566C for ; Tue, 4 Oct 2011 08:03:40 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id EF9768FC12 for ; Tue, 4 Oct 2011 08:03:39 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 1F7794AC1C; Tue, 4 Oct 2011 12:03:38 +0400 (MSD) Date: Tue, 4 Oct 2011 12:03:30 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <81957544.20111004120330@serebryakov.spb.ru> To: "Andrey V. Elsukov" In-Reply-To: <4E8A87CD.2080001@yandex.ru> References: <1258376930.20111002193223@serebryakov.spb.ru> <228926402.20111002231459@serebryakov.spb.ru> <349860851.20111003113417@serebryakov.spb.ru> <4E8986F0.3050007@freebsd.org> <1223820108.20111003161859@serebryakov.spb.ru> <4E8A87CD.2080001@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: fs@freebsd.org Subject: Re: code in GEOM thread could not use vnode API (Was: alq_open_flags() panics in _mtx_lock_flags()) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 08:03:40 -0000 Hello, Andrey. You wrote 4 =CF=CB=D4=D1=C2=D2=D1 2011 =C7., 8:13:01: >> This is geom_zero module, which try to create ALQ with name >> "/var/log/zero.alq.log" on it load (not creation! So, you don't need >> even create such GEOM!). Please note, that "init" callback of GEOM >> class is called in g_event GEOM thread. > Did you try just release the topology lock before alq_open and acquire it= back > before exit from _init method? Yep. Doesn't help. Same panic in same place. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 09:13:12 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FC281065675 for ; Tue, 4 Oct 2011 09:13:12 +0000 (UTC) (envelope-from ari@ish.com.au) Received: from fish.ish.com.au (eth5921.nsw.adsl.internode.on.net [59.167.240.32]) by mx1.freebsd.org (Postfix) with ESMTP id B50728FC08 for ; Tue, 4 Oct 2011 09:13:11 +0000 (UTC) Received: from ip-136.ish.com.au ([203.29.62.136]:56515) by fish.ish.com.au with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76) (envelope-from ) id 1RB13i-0006Wz-0O; Tue, 04 Oct 2011 20:13:06 +1100 X-CTCH-RefID: str=0001.0A150202.4E8ACE22.0103:SCFSTAT15613948, ss=1, re=-4.000, fgs=0 Message-ID: <4E8ACE1E.4060608@ish.com.au> Date: Tue, 04 Oct 2011 20:13:02 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0) Gecko/20110916 Thunderbird/7.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4E8A8740.100@ish.com.au> In-Reply-To: <4E8A8740.100@ish.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 09:13:12 -0000 > Jeremy Chadwick wrote: > There are fixes in 8.2-STABLE that address this problem (there were multiple fixes put in place). So if you're already experiencing kmem map exhaustion, I strongly recommend you upgrade to 8.2-STABLE and then adjust one single tunable: > > vfs.zfs.arc_max I'm using freebsd-update on all our servers (there are about a dozen with this type of setup) so moving to -STABLE isn't simple for us. It looks like we are stuck on 8.2-RELEASE plus security patches for now. With no 8.3 on the near horizon, we need to patch things up for the short term. Also, it looks like the 8.2 errata page has nothing about the issues which have been fixed. [1] That would have made it more useful. :-) Is there a summary of the "multiple fixes"? > You didn't disclose how much RAM your machine has, nor what other daemons are running on it, so it's very hard to give you an estimate. Sorry, yes. This particular machine has 24Gb. Some others as little as 8Gb. All are 64bit of course. The workload of the machines are different (mysql, tomcat, httpd, etc) and I understand that will affect tuning. But I'm less interested in tuning here and more interested in making sure the server doesn't crash. > I tend to tell people to set vfs.zfs.arc_max to about 60% of their memory. E.g. if the machine has 8GB RAM, set vfs.zfs.arc_max="5120M". That is different to the official wiki [2] which suggests that 8.2 is completely self-tuning and has a reasonable default value. On the 24Gb machine I'm seeing arc_max as 92% of physical memory and vm.kmem_size as 97% of physical memory. From everything I've read, that sounds reasonable since arc should automatically decrease as other applications use up memory, so ZFS should not be a reason for the machine swapping RAM to disk. hw.physmem: 25744949248 vm.kmem_size: 24956616704 (97%) hw.realmem: 26843545600 (104% !) vfs.zfs.arc_max: 23882874880 (92%) vm.kmem_size_scale: 1 Curious that realmem is exactly 1Gb larger than the actual memory in the machine. And that size_scale = 1 means 97% rather than 100%, but I guess that is some allowance for other kernel buffers. > We use this on all of our servers with 8GB, including ones that run mysqld with some tunings. Example machine in question: > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 60030 mysql 16 76 0 785M 294M sigwai 0 15:38 0.00% [mysqld] > > And relevant bits from /boot/loader.conf: > > kern.maxdsiz="2560M" kern.dfldsiz="2560M" kern.maxssiz="256M" vfs.zfs.arc_max="5120M" > > You need to keep in mind that ZFS can still use more than what you limit the ARC to. The mailing list has explanations regarding this; I believe it has to do with fragmentation or something like that. So don't go thinking you're being smart by setting it to something like "7500M"; there is a very good chance you will still experience the same problem. So my advice is to "start small" and work your way up after multiple weeks (not days!) of utilising the filesystems on ZFS, to really stress the ARC. This system recently has a kernel lockup after several months without having its settings changed. So some particular workload caused the problem. Since it happened at 1am, I expect that backup scripts touching lots of files were the factor that caused the issue to appear. > Also be aware there may still be problems with applications that use sendfile(2) -- known programs which use this are ftpd (not adjustable), Apache (you can disable it), nginx (I think you can disable it), and Samba (you can disable it). There are some sendfile(2) fixes were which committed, but I'm not 100% sure if all situations have been accounted for. My understanding is that the sendfile issue is a performance one only, not stability. Is that correct? But back to the original question. Pawel recommends in his 1 year old blog entry that kmem should be 150% of actual RAM (I don't really understand why, but he is the expert). Andriy committed scale=1 earlier this year which is more like 97% of actual RAM. Which is correct? I understand how ARC works, but I don't understand why kmem is tunable in ordinary operation or why one value should be preferred. Thanks Ari [1] http://www.freebsd.org/releases/8.2R/errata.html [2] http://wiki.freebsd.org/ZFSTuningGuide -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 13:24:22 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05923106566C for ; Tue, 4 Oct 2011 13:24:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3F44A8FC0A for ; Tue, 4 Oct 2011 13:24:20 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA18701; Tue, 04 Oct 2011 16:24:11 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E8B08FA.1020900@FreeBSD.org> Date: Tue, 04 Oct 2011 16:24:10 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0) Gecko/20110928 Thunderbird/7.0 MIME-Version: 1.0 To: Aristedes Maniatis References: <4E8A8740.100@ish.com.au> In-Reply-To: <4E8A8740.100@ish.com.au> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 13:24:22 -0000 on 04/10/2011 07:10 Aristedes Maniatis said the following: > B. vm.kmem_size_scale=0.66 This parameter is integer. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 14:09:48 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9997C106564A for ; Tue, 4 Oct 2011 14:09:48 +0000 (UTC) (envelope-from maurizio.vairani@cloverinformatica.it) Received: from smtplq03.aruba.it (smtplq-out18.aruba.it [62.149.158.38]) by mx1.freebsd.org (Postfix) with SMTP id E4AF68FC08 for ; Tue, 4 Oct 2011 14:09:47 +0000 (UTC) Received: (qmail 29675 invoked by uid 89); 4 Oct 2011 13:43:06 -0000 Received: from unknown (HELO smtp6.aruba.it) (62.149.158.226) by smtplq03.aruba.it with SMTP; 4 Oct 2011 13:43:06 -0000 Received: (qmail 11745 invoked by uid 89); 4 Oct 2011 13:43:06 -0000 Received: from unknown (HELO clover.dyndns.biz) (info@cloverinformatica.it@151.55.87.177) by smtp6.ad.aruba.it with SMTP; 4 Oct 2011 13:43:06 -0000 Received: from [192.168.0.185] ([192.168.0.185]) by clover.dyndns.biz ; Tue, 4 Oct 2011 15:43:06 +0200 Message-ID: <4E8B0D6A.7090009@cloverinformatica.it> Date: Tue, 04 Oct 2011 15:43:06 +0200 From: Maurizio Vairani User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20111003120027.E4FF51065746@hub.freebsd.org> In-Reply-To: <20111003120027.E4FF51065746@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: smtp6.ad.aruba.it 1.6.2 0/1000/N X-Spam-Rating: smtplq03.aruba.it 1.6.2 0/1000/N Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 14:09:48 -0000 On 03/10/2011 14.00, Attila Nagy wrote: > For me, the bug is still here: > $ uname -a > FreeBSD b 8.2-STABLE FreeBSD 8.2-STABLE #5: Wed Sep 14 15:01:25 CEST > 2011 root@buildervm:/data/usr/obj/data/usr/src/sys/BOOTCLNT amd64 > $ df -h /tmp > Filesystem Size Used Avail Capacity Mounted on > tmpfs 0B 0B 0B 100% /tmp > > I have no swap configured. The machine has 64 GB RAM. > vm.kmem_size=60G; vfs.zfs.arc_max=55G; vfs.zfs.arc_min=20G I had the same problem with 8G of swap and 16G of RAM. After adding 20G of swap space, TMPFS works, at least from yesterday. [ssh@clover-nas ~]$ swapinfo -h Device 1K-blocks Used Avail Capacity /dev/zvol/pool500gb/swap 20971520 7.0M 20G 0% /dev/gpt/SWAP-BIS 4194304 4.7M 4.0G 0% /dev/gpt/SWAP 4194304 4.7M 4.0G 0% Total 29360128 16M 28G 0% [ssh@clover-nas ~]$ Why TMPFS reports only 15G of free memory when there are more than 28G free (RAM + swap) ? [ssh@clover-nas ~]$ df -h /tmp Filesystem Size Used Avail Capacity Mounted on tmpfs 15G 12K 15G 0% /tmp [ssh@clover-nas ~]$ top -b last pid: 10895; load averages: 0.27, 0.26, 0.22 up 26+02:28:18 15:03:37 112 processes: 1 running, 110 sleeping, 1 zombie Mem: 19M Active, 262M Inact, 15G Wired, 66M Buf, 679M Free Swap: 28G Total, 16M Used, 28G Free Maurizio P.S. FreeBSD 8.2-RELEASE-p3. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 14:19:05 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6836106564A; Tue, 4 Oct 2011 14:19:05 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 55F6A8FC0C; Tue, 4 Oct 2011 14:19:04 +0000 (UTC) Received: by qadz30 with SMTP id z30so469734qad.13 for ; Tue, 04 Oct 2011 07:19:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.12.139 with SMTP id x11mr1035418qcx.105.1317737944384; Tue, 04 Oct 2011 07:19:04 -0700 (PDT) Received: by 10.229.224.82 with HTTP; Tue, 4 Oct 2011 07:19:04 -0700 (PDT) In-Reply-To: References: <20111002020231.GA70864@icarus.home.lan> Date: Tue, 4 Oct 2011 16:19:04 +0200 Message-ID: From: Olivier Smedts To: Xin LI Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-fs@freebsd.org" , Ivan Voras Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 14:19:05 -0000 2011/10/4 Xin LI : > On Mon, Oct 3, 2011 at 2:36 PM, Olivier Smedts wrote: > [...] >> Try reducing the swap size to less than the RAM size. No "configuration >> issue", try with some RAM + swap that should fit all. > > But it's not ZFS+tmpfs specific, it can happen anywhere when memory > and swap is not sufficient. =A0Of course tmpfs and ZFS should play more > well together but it's pretty much a "you get what you paid for" > situation IMHO. But there's a problem in this configuration and when memory and swap are sufficient (swap free, some RAM free). tmpfs size drops to 0 because it does not seem to calculate the free size correctly. If there is sufficient RAM (some RAM free), sufficient swap (all free) but less swap than RAM, and wired memory usage is high (ZFS ARC), tmpfs size drops to 0. > One thing I can not yet reproduce, but sounds like a serious issue is > that when system have sufficient RAM available, ZFS reports 0 in free > space... =A0If there is a test case for that then that's definitely > something we need to solve sooner. > > Cheers, > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > --=20 Olivier Smedts=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0 _ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 ASCII ribbon campaign ( ) e-mail: olivier@gid0.org=A0 =A0 =A0 =A0 - against HTML email & vCards=A0 X www: http://www.gid0.org=A0 =A0 - against proprietary attachments / \ =A0 "Il y a seulement 10 sortes de gens dans le monde : =A0 ceux qui comprennent le binaire, =A0 et ceux qui ne le comprennent pas." From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 22:08:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB9801065887 for ; Tue, 4 Oct 2011 22:08:09 +0000 (UTC) (envelope-from syshackmin@gmail.com) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id 00C398FC25 for ; Tue, 4 Oct 2011 22:08:08 +0000 (UTC) Received: by wwn22 with SMTP id 22so5542076wwn.1 for ; Tue, 04 Oct 2011 15:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=sLPpPfC3iXNzwHLOyF44amqEFJWlWg4TTKLc5Gq2sOw=; b=PX9nDOTMRrQSNqQqTyCOiMoamJtBPI89ctGiDDvUt/DsrU/H2/R3nxbApCarHtsxQl geiRHG6y0waRcgHmCjkomRjFU6L2JGkLCzW+BywRRonWAzJZDjG+6t886EKNrV8k4U5+ M+OECa3/2NgGY/QfpY62dz0mvr3OpfQbK2foU= MIME-Version: 1.0 Received: by 10.216.137.223 with SMTP id y73mr2096660wei.6.1317764736224; Tue, 04 Oct 2011 14:45:36 -0700 (PDT) Received: by 10.216.53.21 with HTTP; Tue, 4 Oct 2011 14:45:36 -0700 (PDT) In-Reply-To: References: Date: Tue, 4 Oct 2011 17:45:36 -0400 Message-ID: From: Dave Cundiff To: questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ZFS Write Lockup X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 22:08:09 -0000 Hi, Decided to cross post this over here as well since it seems like it could be something actually wrong and not me being an idiot. Feel free to let me know if I'm an idiot. :) gstat is showing almost no IO hitting da1 which is my zpool(21 disk raid50 3x7). Yet the zvols are all backed up. 0 272 7 71 11.6 264 4439 0.2 4.8| da1 0 4 2 26 6.7 2 23 557.0 126.7| zvol/san/a2s= 61 0 14 0 0 0.0 14 86 91.5 132.5| zvol/san/a2s= 66 1 14 0 1 13.8 14 154 100.6 140.2| zvol/san/sol= semi1 1 19 1 5 0.1 18 156 76.6 139.8| zvol/san/sol= man1 1 6 1 26 8.4 5 112 275.3 140.9| zvol/san/a2s= 62 1 16 1 5 9.1 16 317 88.1 139.7| zvol/san/sol= man2 1 29 1 2 6.6 29 214 48.8 139.8| zvol/san/sol= semi2 1 7 1 2 8.5 6 50 232.5 140.4| zvol/san/sol= man4 I've tweaked only a few settings from default. [root@san2 ~]# cat /boot/loader.conf console=3D"comconsole,vidconsole" comconsole_speed=3D"115200" vm.kmem_size=3D"30G" vfs.zfs.arc_max=3D"22G" kern.hz=3D100 loader_logo=3D"beastie" [root@san2 ~]# cat /etc/sysctl.conf net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.sendspace=3D65536 net.inet.tcp.recvspace=3D131072 [root@san2 ~]# zpool status pool: san state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM san ONLINE 0 0 0 da1 ONLINE 0 0 0 logs mirror ONLINE 0 0 0 ad6s1b ONLINE 0 0 0 ad14s1b ONLINE 0 0 0 cache ad6s1d ONLINE 0 0 0 ad14s1d ONLINE 0 0 0 errors: No known data errors All my volumes are the same. I don't manually adjust any properties. [root@san2 ~]# zfs get all san NAME PROPERTY VALUE SOURCE san type filesystem - san creation Tue Feb 8 9:58 2011 - san used 9.10T - san available 3.33T - san referenced 221M - san compressratio 1.00x - san mounted yes - san quota none default san reservation none default san recordsize 128K default san mountpoint /san default san sharenfs off default san checksum off local san compression off default san atime on default san devices on default san exec on default san setuid on default san readonly off default san jailed off default san snapdir hidden default san aclmode groupmask default san aclinherit restricted default san canmount on default san shareiscsi off default san xattr off temporary san copies 1 default san version 4 - san utf8only off - san normalization none - san casesensitivity sensitive - san vscan off default san nbmand off default san sharesmb off default san refquota none default san refreservation none default san primarycache all default san secondarycache all default san usedbysnapshots 0 - san usedbydataset 221M - san usedbychildren 9.10T - san usedbyrefreservation 0 - [root@san2 ~]# zfs get all san/a2s66 NAME PROPERTY VALUE SOURCE san/a2s66 type volume - san/a2s66 creation Wed Sep 21 16:25 2011 - san/a2s66 used 770G - san/a2s66 available 3.33T - san/a2s66 referenced 753G - san/a2s66 compressratio 1.00x - san/a2s66 reservation none default san/a2s66 volsize 750G - san/a2s66 volblocksize 4K - san/a2s66 checksum off inherited from san san/a2s66 compression off default san/a2s66 readonly off default san/a2s66 shareiscsi off default san/a2s66 copies 1 default san/a2s66 refreservation none default san/a2s66 primarycache all default san/a2s66 secondarycache all default san/a2s66 usedbysnapshots 17.3G - san/a2s66 usedbydataset 753G - san/a2s66 usedbychildren 0 - san/a2s66 usedbyrefreservation 0 - last pid: 60292; load averages: 0.96, 0.67, 0.80 up 9+17:31:59 17:41:52 63 processes: 2 running, 61 sleeping CPU: 1.3% user, 0.0% nice, 46.4% system, 1.1% interrupt, 51.2% idle Mem: 37M Active, 32M Inact, 22G Wired, 15M Cache, 1940M Buf, 1075M Free Swap: 28G Total, 13M Used, 28G Free [root@san2 ~]# sysctl -a | grep zfs vfs.zfs.l2c_only_size: 81245392896 vfs.zfs.mfu_ghost_data_lsize: 51142656 vfs.zfs.mfu_ghost_metadata_lsize: 10687021568 vfs.zfs.mfu_ghost_size: 10738164224 vfs.zfs.mfu_data_lsize: 757547008 vfs.zfs.mfu_metadata_lsize: 954693120 vfs.zfs.mfu_size: 2612401664 vfs.zfs.mru_ghost_data_lsize: 1983434752 vfs.zfs.mru_ghost_metadata_lsize: 3657913344 vfs.zfs.mru_ghost_size: 5641348096 vfs.zfs.mru_data_lsize: 9817952768 vfs.zfs.mru_metadata_lsize: 395397632 vfs.zfs.mru_size: 10833757184 vfs.zfs.anon_data_lsize: 0 vfs.zfs.anon_metadata_lsize: 0 vfs.zfs.anon_size: 34037760 vfs.zfs.l2arc_norw: 1 vfs.zfs.l2arc_feed_again: 1 vfs.zfs.l2arc_noprefetch: 0 vfs.zfs.l2arc_feed_min_ms: 200 vfs.zfs.l2arc_feed_secs: 1 vfs.zfs.l2arc_headroom: 2 vfs.zfs.l2arc_write_boost: 8388608 vfs.zfs.l2arc_write_max: 8388608 vfs.zfs.arc_meta_limit: 5905580032 vfs.zfs.arc_meta_used: 5906093432 vfs.zfs.mdcomp_disable: 0 vfs.zfs.arc_min: 2952790016 vfs.zfs.arc_max: 23622320128 vfs.zfs.zfetch.array_rd_sz: 1048576 vfs.zfs.zfetch.block_cap: 256 vfs.zfs.zfetch.min_sec_reap: 2 vfs.zfs.zfetch.max_streams: 8 vfs.zfs.prefetch_disable: 1 vfs.zfs.check_hostid: 1 vfs.zfs.recover: 0 vfs.zfs.txg.write_limit_override: 0 vfs.zfs.txg.synctime: 5 vfs.zfs.txg.timeout: 30 vfs.zfs.scrub_limit: 10 vfs.zfs.vdev.cache.bshift: 16 vfs.zfs.vdev.cache.size: 10485760 vfs.zfs.vdev.cache.max: 16384 vfs.zfs.vdev.aggregation_limit: 131072 vfs.zfs.vdev.ramp_rate: 2 vfs.zfs.vdev.time_shift: 6 vfs.zfs.vdev.min_pending: 4 vfs.zfs.vdev.max_pending: 10 vfs.zfs.cache_flush_disable: 0 vfs.zfs.zil_disable: 0 vfs.zfs.zio.use_uma: 0 vfs.zfs.version.zpl: 4 vfs.zfs.version.spa: 15 vfs.zfs.version.dmu_backup_stream: 1 vfs.zfs.version.dmu_backup_header: 2 vfs.zfs.version.acl: 1 vfs.zfs.debug: 0 vfs.zfs.super_owner: 0 kstat.zfs.misc.zfetchstats.hits: 1622932834 kstat.zfs.misc.zfetchstats.misses: 300700562 kstat.zfs.misc.zfetchstats.colinear_hits: 144156 kstat.zfs.misc.zfetchstats.colinear_misses: 300556406 kstat.zfs.misc.zfetchstats.stride_hits: 1138458507 kstat.zfs.misc.zfetchstats.stride_misses: 386271 kstat.zfs.misc.zfetchstats.reclaim_successes: 5313527 kstat.zfs.misc.zfetchstats.reclaim_failures: 295242879 kstat.zfs.misc.zfetchstats.streams_resets: 141691 kstat.zfs.misc.zfetchstats.streams_noresets: 484474231 kstat.zfs.misc.zfetchstats.bogus_streams: 0 kstat.zfs.misc.arcstats.hits: 2877951340 kstat.zfs.misc.arcstats.misses: 677132553 kstat.zfs.misc.arcstats.demand_data_hits: 1090801028 kstat.zfs.misc.arcstats.demand_data_misses: 142078773 kstat.zfs.misc.arcstats.demand_metadata_hits: 760631826 kstat.zfs.misc.arcstats.demand_metadata_misses: 15429069 kstat.zfs.misc.arcstats.prefetch_data_hits: 77566631 kstat.zfs.misc.arcstats.prefetch_data_misses: 412415335 kstat.zfs.misc.arcstats.prefetch_metadata_hits: 948951855 kstat.zfs.misc.arcstats.prefetch_metadata_misses: 107209376 kstat.zfs.misc.arcstats.mru_hits: 762645834 kstat.zfs.misc.arcstats.mru_ghost_hits: 25063620 kstat.zfs.misc.arcstats.mfu_hits: 1792233827 kstat.zfs.misc.arcstats.mfu_ghost_hits: 107949685 kstat.zfs.misc.arcstats.allocated: 1220368604 kstat.zfs.misc.arcstats.deleted: 881708637 kstat.zfs.misc.arcstats.stolen: 324197286 kstat.zfs.misc.arcstats.recycle_miss: 393866103 kstat.zfs.misc.arcstats.mutex_miss: 47835019 kstat.zfs.misc.arcstats.evict_skip: 16800403516 kstat.zfs.misc.arcstats.evict_l2_cached: 3404346428416 kstat.zfs.misc.arcstats.evict_l2_eligible: 906780261888 kstat.zfs.misc.arcstats.evict_l2_ineligible: 1712274098176 kstat.zfs.misc.arcstats.hash_elements: 12932367 kstat.zfs.misc.arcstats.hash_elements_max: 26675689 kstat.zfs.misc.arcstats.hash_collisions: 1195027725 kstat.zfs.misc.arcstats.hash_chains: 524288 kstat.zfs.misc.arcstats.hash_chain_max: 114 kstat.zfs.misc.arcstats.p: 13900189206 kstat.zfs.misc.arcstats.c: 16514222070 kstat.zfs.misc.arcstats.c_min: 2952790016 kstat.zfs.misc.arcstats.c_max: 23622320128 kstat.zfs.misc.arcstats.size: 16514197960 kstat.zfs.misc.arcstats.hdr_size: 698646840 kstat.zfs.misc.arcstats.data_size: 13480204800 kstat.zfs.misc.arcstats.other_size: 222586112 kstat.zfs.misc.arcstats.l2_hits: 236859220 kstat.zfs.misc.arcstats.l2_misses: 440273314 kstat.zfs.misc.arcstats.l2_feeds: 998879 kstat.zfs.misc.arcstats.l2_rw_clash: 41492 kstat.zfs.misc.arcstats.l2_read_bytes: 1523423294976 kstat.zfs.misc.arcstats.l2_write_bytes: 2108729975808 kstat.zfs.misc.arcstats.l2_writes_sent: 908755 kstat.zfs.misc.arcstats.l2_writes_done: 908755 kstat.zfs.misc.arcstats.l2_writes_error: 0 kstat.zfs.misc.arcstats.l2_writes_hdr_miss: 125029 kstat.zfs.misc.arcstats.l2_evict_lock_retry: 78155 kstat.zfs.misc.arcstats.l2_evict_reading: 52 kstat.zfs.misc.arcstats.l2_free_on_write: 735076 kstat.zfs.misc.arcstats.l2_abort_lowmem: 2368 kstat.zfs.misc.arcstats.l2_cksum_bad: 9 kstat.zfs.misc.arcstats.l2_io_error: 0 kstat.zfs.misc.arcstats.l2_size: 88680833024 kstat.zfs.misc.arcstats.l2_hdr_size: 2275280224 kstat.zfs.misc.arcstats.memory_throttle_count: 0 kstat.zfs.misc.arcstats.l2_write_trylock_fail: 160181805 kstat.zfs.misc.arcstats.l2_write_passed_headroom: 48073379 kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 kstat.zfs.misc.arcstats.l2_write_in_l2: 101326826532 kstat.zfs.misc.arcstats.l2_write_io_in_progress: 3016312 kstat.zfs.misc.arcstats.l2_write_not_cacheable: 16631379447 kstat.zfs.misc.arcstats.l2_write_full: 158541 kstat.zfs.misc.arcstats.l2_write_buffer_iter: 998879 kstat.zfs.misc.arcstats.l2_write_pios: 908755 kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 881025143301120 kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 62580701 kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 2040782 kstat.zfs.misc.vdev_cache_stats.delegations: 2167916 kstat.zfs.misc.vdev_cache_stats.hits: 2801310 kstat.zfs.misc.vdev_cache_stats.misses: 5448597 On Tue, Oct 4, 2011 at 2:43 AM, Dave Cundiff wrote: > Hi, > > I'm running 8.2-RELEASE and running into an IO lockup on ZFS that is > happening pretty regularly. The system is stock except for the > following set in loader.conf > > vm.kmem_size=3D"30G" > vfs.zfs.arc_max=3D"22G" > kern.hz=3D100 > > I know the kmem settings aren't SUPPOSED to be necessary now, buy my > ZFS boxes were crashing until I added them. The machine has 24 gigs of > RAM. The kern.hz=3D100 was to stretch out the l2arc bug that pops up at > 28days with it set to 1000. > > [root@san2 ~]# zpool status > =A0pool: san > =A0state: ONLINE > =A0scrub: none requested > config: > > =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0 STATE =A0 =A0 READ WRITE CKSUM > =A0 =A0 =A0 =A0san =A0 =A0 =A0 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 > =A0 =A0 =A0 =A0 =A0da1 =A0 =A0 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 > =A0 =A0 =A0 =A0logs > =A0 =A0 =A0 =A0 =A0mirror =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 > =A0 =A0 =A0 =A0 =A0 =A0ad6s1b =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 > =A0 =A0 =A0 =A0 =A0 =A0ad14s1b =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 > =A0 =A0 =A0 =A0cache > =A0 =A0 =A0 =A0 =A0ad6s1d =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 > =A0 =A0 =A0 =A0 =A0ad14s1d =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 > > errors: No known data errors > > > Here's a zpool iostat from a machine in trouble. > > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A07.92K > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0447 =A0 =A0 =A00 = =A05.77M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0309 =A0 =A0 =A00 = =A02.83M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 62 =A0 =A0 =A00 =A02.22M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A02 =A0 =A0 =A00= =A023.5K > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0254 =A0 =A0 =A00 = =A06.62M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0249 =A0 =A0 =A00 = =A03.16M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 34 =A0 =A0 =A00 =A0 491K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A06 =A0 =A0 =A00= =A062.7K > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 85 =A0 =A0 =A00 = =A06.59M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0452 =A0 =A0 =A00 = =A04.88M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0109 =A0 =A0 =A00 =A03.12M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A07.84K > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0434 =A0 =A0 =A00 = =A06.41M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0 =A00 =A0 =A0 =A00= =A0 =A0 =A00 > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 =A00 =A0 =A0304 =A0 =A0 =A00 = =A02.90M > san =A0 =A0 =A0 =A0 9.08T =A03.55T =A0 =A0 37 =A0 =A0 =A00 =A0 628K =A0 = =A0 =A00 > > Its supposed to look like > > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0162 =A0 =A0167 =A03.75M =A06.09= M > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 =A05 =A0 =A0 =A00 =A047.4K =A0= =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 19 =A0 =A0 =A00 =A0 213K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0120 =A0 =A0 =A00 =A03.26M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 92 =A0 =A0 =A00 =A0 741K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0114 =A0 =A0 =A00 =A02.86M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 72 =A0 =A0 =A00 =A0 579K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 14 =A0 =A0 =A00 =A0 118K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 24 =A0 =A0 =A00 =A0 213K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 25 =A0 =A0 =A00 =A0 324K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 =A08 =A0 =A0 =A00 =A0 126K =A0= =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 28 =A0 =A0 =A00 =A0 505K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 15 =A0 =A0 =A00 =A0 126K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 11 =A0 =A0 =A00 =A0 158K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 19 =A0 =A0 =A00 =A0 356K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0198 =A0 =A0 =A00 =A03.55M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 21 =A0 =A0 =A00 =A0 173K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 18 =A0 =A0 =A00 =A0 150K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 23 =A0 =A0 =A00 =A0 260K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 =A09 =A0 =A0 =A00 =A078.3K =A0= =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 21 =A0 =A0 =A00 =A0 173K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 =A02 =A04.59K =A016.8K =A0 142= M > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 12 =A0 =A0 =A00 =A0 103K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 26 =A0 =A0454 =A0 312K =A04.35= M > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0111 =A0 =A0 =A00 =A03.34M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 28 =A0 =A0 =A00 =A0 870K =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 75 =A0 =A0 =A00 =A03.88M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 43 =A0 =A0 =A00 =A01.22M =A0 = =A0 =A00 > san =A0 =A0 =A0 =A0 9.07T =A03.56T =A0 =A0 26 =A0 =A0 =A00 =A0 270K =A0 = =A0 =A00 > > I don't know what triggers the problem but I know how to fix it. If I > perform a couple snapshot deletes the IO will come back in line every > single time. Fortunately I have LOTS of snapshots to delete. > > [root@san2 ~]# zfs list -r -t snapshot | wc -l > =A0 =A05236 > [root@san2 ~]# zfs list -r -t volume | wc -l > =A0 =A0 =A017 > > Being fairly new to FreeBSD and ZFS I'm pretty clueless on where to > begin tracking this down. I've been staring at gstat trying to see if > a zvol is getting a big burst of writes that may be flooding the drive > controller but I haven't caught anything yet. top -S -H shows > zio_write_issue threads consuming massive amounts of CPU during the > lockup. Normally they sit around 5-10%. =A0Any suggestions on where I > could start to track this down would be greatly appreciated. > --=20 Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com From owner-freebsd-fs@FreeBSD.ORG Tue Oct 4 23:30:19 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85534106566C for ; Tue, 4 Oct 2011 23:30:19 +0000 (UTC) (envelope-from ari@ish.com.au) Received: from fish.ish.com.au (eth5921.nsw.adsl.internode.on.net [59.167.240.32]) by mx1.freebsd.org (Postfix) with ESMTP id 433D48FC12 for ; Tue, 4 Oct 2011 23:30:18 +0000 (UTC) Received: from ip-136.ish.com.au ([203.29.62.136]:53515 helo=Aris-MacBook-Pro.local) by fish.ish.com.au with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76) (envelope-from ) id 1RBERC-0003cf-0x; Wed, 05 Oct 2011 10:30:14 +1100 X-CTCH-RefID: str=0001.0A150204.4E8B9706.00DD:SCFSTAT15613948, ss=1, re=-4.000, fgs=0 Message-ID: <4E8B9703.4070303@ish.com.au> Date: Wed, 05 Oct 2011 10:30:11 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0) Gecko/20110916 Thunderbird/7.0 MIME-Version: 1.0 To: Andriy Gapon References: <4E8A8740.100@ish.com.au> <4E8B08FA.1020900@FreeBSD.org> In-Reply-To: <4E8B08FA.1020900@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 23:30:19 -0000 On 5/10/11 12:24 AM, Andriy Gapon wrote: > on 04/10/2011 07:10 Aristedes Maniatis said the following: >> B. vm.kmem_size_scale=0.66 > > This parameter is integer. Ah, of course. Would it then be the recommendation that kmem be set manually to 150% of physical memory, or should it normally be left to its default in a machine with a medium amount of RAM (24Gb) and heavy usage of ZFS? I've googled but I just can't understand what the purpose would be of setting kmem to greater than physical memory. But if it helps... Ari -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 00:31:28 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C06691065670 for ; Wed, 5 Oct 2011 00:31:28 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 817FA8FC14 for ; Wed, 5 Oct 2011 00:31:28 +0000 (UTC) Received: by gyf2 with SMTP id 2so1345171gyf.13 for ; Tue, 04 Oct 2011 17:31:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=WJycy8tSknFC+w6CRcFqiVtmesWlRGzkpToLezgcLMM=; b=agoIMlk3Ia9vZpB0X8U0GaiH2yED/kxHgMl4yC8pCb+n71MYqEmFI/naudWvK09hTs HVCQpD2Wdv33J2D0lY/Jb7U459mJA0iPjtS37wIfQe2bjz2JleJnI6eIReds0rFD+sZF zj+AXLQDnGe05U2hOFgaj2uQeZ7tMWinI7W0s= MIME-Version: 1.0 Received: by 10.236.185.37 with SMTP id t25mr10289303yhm.131.1317774687658; Tue, 04 Oct 2011 17:31:27 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Tue, 4 Oct 2011 17:31:27 -0700 (PDT) In-Reply-To: <4E8ACE1E.4060608@ish.com.au> References: <4E8A8740.100@ish.com.au> <4E8ACE1E.4060608@ish.com.au> Date: Wed, 5 Oct 2011 00:31:27 +0000 X-Google-Sender-Auth: dPLa79rLG3E1jPjFIaNDr75XivI Message-ID: From: Artem Belevich To: Aristedes Maniatis Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 00:31:28 -0000 On Tue, Oct 4, 2011 at 2:13 AM, Aristedes Maniatis wrote: > But back to the original question. Pawel recommends in his 1 year old blog > entry that kmem should be 150% of actual RAM (I don't really understand why, > but he is the expert). Andriy committed scale=1 earlier this year which is > more like 97% of actual RAM. Which is correct? > > I understand how ARC works, but I don't understand why kmem is tunable in > ordinary operation or why one value should be preferred. Think of packing randomly sized round pebbles (randomly sized ARC data chunks in this case) into square boxes (power-of-2 allocator bins or perhaps multiples of 4K pages for larger allocations). It's obvious that you will not be able to reach 100% utilization. kmem_size provides address space for the allocator which will then map physical memory there. Hence if you want to use certain amount of physical memory in kernel, you should make sure that kmem_map is large enough to accommodate it including the overhead. --Artem From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 01:01:41 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A7751065672 for ; Wed, 5 Oct 2011 01:01:41 +0000 (UTC) (envelope-from ari@ish.com.au) Received: from fish.ish.com.au (eth5921.nsw.adsl.internode.on.net [59.167.240.32]) by mx1.freebsd.org (Postfix) with ESMTP id 349B78FC12 for ; Wed, 5 Oct 2011 01:01:40 +0000 (UTC) Received: from ip-136.ish.com.au ([203.29.62.136]:57753 helo=Aris-MacBook-Pro.local) by fish.ish.com.au with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76) (envelope-from ) id 1RBFra-0001m7-1B; Wed, 05 Oct 2011 12:01:34 +1100 X-CTCH-RefID: str=0001.0A150204.4E8BAC6E.0093:SCFSTAT15613948, ss=1, re=-4.000, fgs=0 Message-ID: <4E8BAC6A.3060301@ish.com.au> Date: Wed, 05 Oct 2011 12:01:30 +1100 From: Aristedes Maniatis User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0) Gecko/20110916 Thunderbird/7.0 MIME-Version: 1.0 To: Artem Belevich References: <4E8A8740.100@ish.com.au> <4E8ACE1E.4060608@ish.com.au> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: vm.kmem_size_scale recommendation for ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 01:01:41 -0000 On 5/10/11 11:31 AM, Artem Belevich wrote: > On Tue, Oct 4, 2011 at 2:13 AM, Aristedes Maniatis wrote: >> But back to the original question. Pawel recommends in his 1 year old blog >> entry that kmem should be 150% of actual RAM (I don't really understand why, >> but he is the expert). Andriy committed scale=1 earlier this year which is >> more like 97% of actual RAM. Which is correct? >> >> I understand how ARC works, but I don't understand why kmem is tunable in >> ordinary operation or why one value should be preferred. > > Think of packing randomly sized round pebbles (randomly sized ARC data > chunks in this case) into square boxes (power-of-2 allocator bins or > perhaps multiples of 4K pages for larger allocations). It's obvious > that you will not be able to reach 100% utilization. kmem_size > provides address space for the allocator which will then map physical > memory there. Hence if you want to use certain amount of physical > memory in kernel, you should make sure that kmem_map is large enough > to accommodate it including the overhead. So then I assume because this kernel memory is wired to physical RAM, assigning kmem to a large number will never cause ARC to end up in swap? Is the only downside to a larger value of kmem (say double actual RAM) that there are some allocation map entries which take up space and subtract from the actual space available? Are they significant in size? If not, then setting kmem fairly high would avoid any problems of letting ARC bump into the kmem value. Thanks for this information. This is really helpful. I don't have rights to add it to the FreeBSD wiki, but I'll try and post a blog entry on my own site once I understand it all. Ari -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 05:06:58 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90D7A106564A; Wed, 5 Oct 2011 05:06:58 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 68B388FC14; Wed, 5 Oct 2011 05:06:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p9556w15090298; Wed, 5 Oct 2011 05:06:58 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p9556wWU090294; Wed, 5 Oct 2011 05:06:58 GMT (envelope-from linimon) Date: Wed, 5 Oct 2011 05:06:58 GMT Message-Id: <201110050506.p9556wWU090294@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/160893: [zfs] [panic] 9.0-BETA2 kernel panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 05:06:58 -0000 Old Synopsis: [panic] 9.0-BETA2 kernel panic New Synopsis: [zfs] [panic] 9.0-BETA2 kernel panic Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Wed Oct 5 05:06:43 UTC 2011 Responsible-Changed-Why: reassign. http://www.freebsd.org/cgi/query-pr.cgi?pr=160893 From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 09:55:27 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22789106566C; Wed, 5 Oct 2011 09:55:27 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 702848FC1A; Wed, 5 Oct 2011 09:55:26 +0000 (UTC) Received: by eyg7 with SMTP id 7so1945590eyg.13 for ; Wed, 05 Oct 2011 02:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=9NDCaG4dktuQcGJB/LyMjdF+QQqO+uVK/+zsJABJ6Wg=; b=FaWYA915jhgvIrN477G7zTkbWdYnzZTeppo2BDV2L7WvxnJh1lBBifGdnF+DOavnQk 1dgRkHs//I0itQARoMu5zeYXZmVbdnh6rDrlwOt6gMpjd5688q+QxPfwBMDpL4YTfXZ4 X+fd9Ia6pdi5wasrTrtYXJMKV5e3kYizaZu4M= Received: by 10.213.8.144 with SMTP id h16mr1791673ebh.57.1317806883685; Wed, 05 Oct 2011 02:28:03 -0700 (PDT) Received: from localhost (150-109-35-84.ipact.nl. [84.35.109.150]) by mx.google.com with ESMTPS id y1sm34826813eea.1.2011.10.05.02.28.01 (version=SSLv3 cipher=OTHER); Wed, 05 Oct 2011 02:28:02 -0700 (PDT) Date: Wed, 5 Oct 2011 12:26:04 +0300 From: Gleb Kurtsou To: Olivier Smedts Message-ID: <20111005092603.GA1874@tops> References: <20111002020231.GA70864@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" , Ivan Voras Subject: Re: is TMPFS still highly experimental? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 09:55:27 -0000 On (04/10/2011 16:19), Olivier Smedts wrote: > 2011/10/4 Xin LI : > > On Mon, Oct 3, 2011 at 2:36 PM, Olivier Smedts wrote: > > [...] > >> Try reducing the swap size to less than the RAM size. No "configuration > >> issue", try with some RAM + swap that should fit all. > > > > But it's not ZFS+tmpfs specific, it can happen anywhere when memory > > and swap is not sufficient.  Of course tmpfs and ZFS should play more > > well together but it's pretty much a "you get what you paid for" > > situation IMHO. > > But there's a problem in this configuration and when memory and swap > are sufficient (swap free, some RAM free). tmpfs size drops to 0 > because it does not seem to calculate the free size correctly. If > there is sufficient RAM (some RAM free), sufficient swap (all free) > but less swap than RAM, and wired memory usage is high (ZFS ARC), > tmpfs size drops to 0. Free RAM is a bit tricky with virtual memory and overcommit support all over the place. There are at least 3 memory hungry subsystems: buffer cache, ZFS ARC, tmpfs. For the first two there is defined maximum size and they can be shrunk in low memory situations. Tmpfs grows as much as it can trying to calculate "free" memory available. Another difference is that tmpfs can't be shrunk in low memory situation. I proposed a patch changing tmpfs memory allocation: - Define maximum file system size (RAM/2 by default) - Don't try to check if free memory available, check free swap instead and allocate more aggressively, i.e. allocate until swap or file system limit is reached. Patch: http://marc.info/?l=freebsd-fs&m=129747367322954&w=2 https://github.com/glk/freebsd-head/tree/tmpfs Thanks, Gleb. > > One thing I can not yet reproduce, but sounds like a serious issue is > > that when system have sufficient RAM available, ZFS reports 0 in free > > space...  If there is a test case for that then that's definitely > > something we need to solve sooner. > > > > Cheers, > > -- > > Xin LI https://www.delphij.net/ > > FreeBSD - The Power to Serve! Live free or die > > > > -- > Olivier Smedts                                                 _ >                                         ASCII ribbon campaign ( ) > e-mail: olivier@gid0.org        - against HTML email & vCards  X > www: http://www.gid0.org    - against proprietary attachments / \ > >   "Il y a seulement 10 sortes de gens dans le monde : >   ceux qui comprennent le binaire, >   et ceux qui ne le comprennent pas." > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 10:23:18 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCCD5106566B; Wed, 5 Oct 2011 10:23:18 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [70.36.157.235]) by mx1.freebsd.org (Postfix) with ESMTP id ADE6B8FC16; Wed, 5 Oct 2011 10:23:18 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id p95ANJr5005769; Wed, 5 Oct 2011 03:23:19 -0700 (PDT) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201110051023.p95ANJr5005769@chez.mckusick.com> To: Kostik Belousov In-reply-to: <20111003203811.GA1511@deviant.kiev.zoral.com.ua> Date: Wed, 05 Oct 2011 03:23:19 -0700 From: Kirk McKusick X-Spam-Status: No, score=0.0 required=5.0 tests=MISSING_MID, UNPARSEABLE_RELAY autolearn=failed version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on chez.mckusick.com Cc: Attilio Rao , Garrett Cooper , Xin LI , freebsd-fs@freebsd.org Subject: Re: Need to force sync(2) before umounting UFS1 filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 10:23:19 -0000 > Date: Mon, 3 Oct 2011 23:38:11 +0300 > From: Kostik Belousov > To: Attilio Rao > Cc: Kirk McKusick , > Garrett Cooper , > freebsd-fs@freebsd.org, Xin LI > Subject: Re: Need to force sync(2) before umounting UFS1 filesystems? > > On Sun, Oct 02, 2011 at 02:19:32AM +0200, Attilio Rao wrote: > > I'm sorry if it wasn't clear in kib/my latest message, but we don't > > need the coveredvnode unlocking logic because of the tegge's commit. > > > > I just think we should commit the change in policy Kirk initially > > submitted + a comment on top of vfs_busy() explaining why the deadlock > > with coveredvnode cannot happen. > > Below is my take on the comment. Sorry for going silent. I have just finished my travel to the EuroBSD conference which is no small journey from Berkeley California :-) I believe that your comment below nicely summarizes Tor's commit log entry. It also convinces me that my original change is indeed correct. Looked at another way, if my original fix was not correct then the forced unmount case would also have been incorrect. So, I am glad that we did this exercise. Please go ahead and commit the comment (and MFC it to at least 9 if not 8 and 7 to which I believe it also applies). I am awaiting further testing of my change from Garrett Cooper before proceeding with checking it in to head. Kirk McKusick =-=-= commit 3981acdadcf4313dbdf813ec107f7bfbb4057d09 Author: Konstantin Belousov Date: Mon Oct 3 23:33:06 2011 +0300 Move parts of the commit log for r166167, where Tor explained the interaction between vnode locks and vfs_busy(), into comment. diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 7eb619a..3d7735d 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -348,6 +348,38 @@ SYSINIT(vfs, SI_SUB_VFS, SI_ORDER_FIRST, vntblinit, NU= LL); /* * Mark a mount point as busy. Used to synchronize access and to delay * unmounting. Eventually, mountlist_mtx is not released on failure. + * + * vfs_busy() is a custom lock, it can block the caller. + * vfs_busy() only sleeps if the unmount is active on the mount point. + * For a mountpoint mp, vfs_busy-enforced lock is before lock of any + * vnode belonging to mp. + * + * Lookup uses vfs_busy() to traverse mount points. + * root fs var fs + * / vnode lock A / vnode lock (/var) D + * /var vnode lock B /log vnode lock(/var/log) E + * vfs_busy lock C vfs_busy lock F + * + * Within each file system, the lock order is C->A->B and F->D->E. + * + * When traversing across mounts, the system follows that lock order: + * + * C->A->B + * | + * +->F->D->E + * + * The lookup() process for namei("/var") illustrates the process: + * VOP_LOOKUP() obtains B while A is held + * vfs_busy() obtains a shared lock on F while A and B are held + * vput() releases lock on B + * vput() releases lock on A + * VFS_ROOT() obtains lock on D while shared lock on F is held + * vfs_unbusy() releases shared lock on F + * vn_lock() obtains lock on deadfs vnode vp_crossmp instead of A. + * Attempt to lock A (instead of vp_crossmp) while D is held would + * violate the global order, causing deadlocks. + * + * dounmount() locks B while F is drained. */ int vfs_busy(struct mount *mp, int flags) From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 10:50:11 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3ECD0106564A for ; Wed, 5 Oct 2011 10:50:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 153258FC14 for ; Wed, 5 Oct 2011 10:50:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p95AoAPB077649 for ; Wed, 5 Oct 2011 10:50:10 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p95AoAHu077647; Wed, 5 Oct 2011 10:50:10 GMT (envelope-from gnats) Date: Wed, 5 Oct 2011 10:50:10 GMT Message-Id: <201110051050.p95AoAHu077647@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Edho Arief Cc: Subject: Re: kern/160591: [zfs] Fail to boot on zfs root with degraded raidz2 [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Edho Arief List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 10:50:11 -0000 The following reply was made to PR kern/160591; it has been noted by GNATS. From: Edho Arief To: bug-followup@freebsd.org, ighor@ighor.ru Cc: Subject: Re: kern/160591: [zfs] Fail to boot on zfs root with degraded raidz2 [regression] Date: Wed, 5 Oct 2011 17:10:25 +0700 Also fail on 9.0-BETA3, including when using bootloader set (pmbr, gptzfsboot, and zfsloader) from 9.0-BETA3 on 8.2-RELEASE. I had to use 9.0-BETA3 set since my root pool is 4K but it doesn't boot anymore when one of the disk failed (at least after waiting the spinning cursor for one hour and still spinning). From owner-freebsd-fs@FreeBSD.ORG Wed Oct 5 22:37:19 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E9B7106564A; Wed, 5 Oct 2011 22:37:19 +0000 (UTC) (envelope-from syshackmin@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B41FC8FC1A; Wed, 5 Oct 2011 22:37:18 +0000 (UTC) Received: by wyj26 with SMTP id 26so3003258wyj.13 for ; Wed, 05 Oct 2011 15:37:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=s0DbMPy1RBojdkNBPHxPxzrv5lKM7//GFFazhCMQo24=; b=L0AzshxtbCF5hMtOr3Sy76HYNY/MFeZYx4ivjhMJpLrBuVwRr8HtcrkqrymFTYEAEJ imvu4kihlGijebmXmQSOugn8XAifYQf/OjohnYOp3jFwr9aykIYWn9BYQu8dC4uUihgE hfgT/4H/8bvAIpGyS0FQS22mSvwuMieYOyYsw= MIME-Version: 1.0 Received: by 10.216.137.223 with SMTP id y73mr38421wei.6.1317854237513; Wed, 05 Oct 2011 15:37:17 -0700 (PDT) Received: by 10.216.53.21 with HTTP; Wed, 5 Oct 2011 15:37:17 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Oct 2011 18:37:17 -0400 Message-ID: From: Dave Cundiff To: Daniel Staal Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, questions@freebsd.org Subject: Re: ZFS Write Lockup X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 22:37:19 -0000 On Wed, Oct 5, 2011 at 5:52 PM, Daniel Staal wrote: > --As of October 4, 2011 2:43:45 AM -0400, Dave Cundiff is alleged to have > said: > >> I don't know what triggers the problem but I know how to fix it. If I >> perform a couple snapshot deletes the IO will come back in line every >> single time. Fortunately I have LOTS of snapshots to delete. >> >> [root@san2 ~]# zfs list -r -t snapshot | wc -l >> =A0 =A05236 >> [root@san2 ~]# zfs list -r -t volume | wc -l >> =A0 =A0 =A017 > > --As for the rest, it is mine. > > I have no good advice, but I have a thought. =A0;) > > The thought is: Why so many snapshots? =A0And: How many other people have= that > many snapshots? =A0I know that ZFS is supposed to be able to handle huge > numbers of snapshots (far more than a few thousand, from my understanding= ), > but if it hasn't been used much in that config, there may be bugs lurking= . > > You might try weeding through and figuring out if you can drop a good amo= unt > of those snapshots. =A0Also, try the filesystems list. =A0They may have b= etter > thoughts. > > Daniel T. Staal > Its for a backup service I've been working on. It takes a snapshot hourly of all 17 zvols. I was planning on keeping them for a month. I had the same thought about the snapshots and deleted them all yesterday. It appears there is some issue with keeping that many. I removed them all and the zvols are now functioning correctly. Its strange that the large number didn't cause incremental slowdown. While the snapshots were still there the IO was normal when it wasn't acting up. Just it would have spurts of almost total lockup until I performed a snapshot removal operation or 2. Thanks, --=20 Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 02:07:18 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E19FA106564A for ; Thu, 6 Oct 2011 02:07:18 +0000 (UTC) (envelope-from DStaal@usa.net) Received: from mail.magehandbook.com (173-8-4-45-WashingtonDC.hfc.comcastbusiness.net [173.8.4.45]) by mx1.freebsd.org (Postfix) with ESMTP id B55228FC08 for ; Thu, 6 Oct 2011 02:07:18 +0000 (UTC) Received: from [192.168.1.50] (Mac-Pro.magehandbook.com [192.168.1.50]) by mail.magehandbook.com (Postfix) with ESMTP id 39295835; Wed, 5 Oct 2011 21:51:42 -0400 (EDT) Date: Wed, 05 Oct 2011 21:51:42 -0400 From: Daniel Staal To: Dave Cundiff Message-ID: <15370081D51E465D77FAAA41@mac-pro.magehandbook.com> In-Reply-To: References: X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: freebsd-fs@freebsd.org, questions@freebsd.org Subject: Re: ZFS Write Lockup X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: FreeBSD Questions List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 02:07:19 -0000 --As of October 5, 2011 6:37:17 PM -0400, Dave Cundiff is alleged to have said: > Its for a backup service I've been working on. It takes a snapshot > hourly of all 17 zvols. I was planning on keeping them for a month. > > I had the same thought about the snapshots and deleted them all > yesterday. It appears there is some issue with keeping that many. I > removed them all and the zvols are now functioning correctly. Its > strange that the large number didn't cause incremental slowdown. While > the snapshots were still there the IO was normal when it wasn't acting > up. Just it would have spurts of almost total lockup until I performed > a snapshot removal operation or 2. --As for the rest, it is mine. Look at sysutils/zfs-periodic. It does that type of backup, although it doesn't keep quite as many. (Several hourly, daily, weekly, and monthly. Configurable. I have 38 filesystems, and 780 snapshots. (Only one volume, though.)) And that definitely sounds like a bug. Gradual slowdowns would be expected if you were just reaching performance limits, but sudden stops sound like there's a condition that doesn't work someplace, or some loop the system can get into. Daniel T. Staal --------------------------------------------------------------- This email copyright the author. Unless otherwise noted, you are expressly allowed to retransmit, quote, or otherwise use the contents for non-commercial purposes. This copyright will expire 5 years after the author's death, or in 30 years, whichever is longer, unless such a period is in excess of local copyright law. --------------------------------------------------------------- From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 03:45:05 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAF98106566B for ; Thu, 6 Oct 2011 03:45:05 +0000 (UTC) (envelope-from freebsd@penx.com) Received: from Elmer.dco.penx.com (elmer.dco.penx.com [174.46.214.165]) by mx1.freebsd.org (Postfix) with ESMTP id 932BF8FC0C for ; Thu, 6 Oct 2011 03:45:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by Elmer.dco.penx.com (8.14.5/8.14.4) with ESMTP id p962bKku045328 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 5 Oct 2011 20:37:22 -0600 (MDT) (envelope-from freebsd@penx.com) Date: Wed, 5 Oct 2011 20:37:20 -0600 (MDT) From: Dennis Glatting X-X-Sender: dennisg@Elmer.dco.penx.com To: freebsd-fs@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: Odd incident with ZFS and an Aerca 1880i X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 03:45:05 -0000 I have a Aerca 1880i plugged into a Gigabyte GA-X58A-UD7 running RELENG_8 with a 10 2TB RAIDz pool with a OCZ Revo L2Arc and a SSD ZIL, among other volumes. The last scrub indicated there was an error with the pool and today I finally got around to cleaning things up. During the scrub I initiated today, a running "zpool status" displayed a message that said I should clear the pool, which I did, while the scrub was running, which probably wasn't such a good idea. The result was my disks through the Aerca disappeared. On a power switch reboot the Aerca no longer displayed its boot-time messages and it appears the Aerca's firmware is kaput. Specifically, during the boot sequence the following: "arcmsr0: timed out waiting for firmware ready" "arcmsr0: wait 'get adapter firmware miscellaneous data' timeout" From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 04:05:12 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0780106566C for ; Thu, 6 Oct 2011 04:05:12 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id A83D48FC14 for ; Thu, 6 Oct 2011 04:05:12 +0000 (UTC) Received: from omta15.emeryville.ca.mail.comcast.net ([76.96.30.71]) by qmta09.emeryville.ca.mail.comcast.net with comcast id hC6W1h0081Y3wxoA9G55vp; Thu, 06 Oct 2011 04:05:05 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta15.emeryville.ca.mail.comcast.net with comcast id hG8l1h00C1t3BNj8bG8lao; Thu, 06 Oct 2011 04:08:45 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id A3730102C1C; Wed, 5 Oct 2011 21:05:11 -0700 (PDT) Date: Wed, 5 Oct 2011 21:05:11 -0700 From: Jeremy Chadwick To: Dennis Glatting Message-ID: <20111006040511.GA54920@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: Odd incident with ZFS and an Aerca 1880i X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 04:05:12 -0000 On Wed, Oct 05, 2011 at 08:37:20PM -0600, Dennis Glatting wrote: > I have a Aerca 1880i plugged into a Gigabyte GA-X58A-UD7 running > RELENG_8 with a 10 2TB RAIDz pool with a OCZ Revo L2Arc and a SSD > ZIL, among other volumes. The last scrub indicated there was an > error with the pool and today I finally got around to cleaning > things up. > > During the scrub I initiated today, a running "zpool status" > displayed a message that said I should clear the pool, which I did, > while the scrub was running, which probably wasn't such a good idea. > The result was my disks through the Aerca disappeared. On a power > switch reboot the Aerca no longer displayed its boot-time messages > and it appears the Aerca's firmware is kaput. > > Specifically, during the boot sequence the following: > > "arcmsr0: timed out waiting for firmware ready" > "arcmsr0: wait 'get adapter firmware miscellaneous data' timeout" 1) The controller brand/vendor is Areca, not Aerca. (This isn't just a typo, you said it 3 times in your mail (including Subject). :-) ) 2) The arcmsr(4) man page does not mention the 1880i controller being supported. Readers should note this controller is ***super*** new. It's a SATA600/SAS controller with PCIe 2.0: http://www.areca.com.tw/products/1880.htm My point:I am not surprised this is not working in RELENG_8. This controller, again, is **very** new on the market. 3) Areca provides FreeBSD support natively. You should absolutely Email them and bring the problem up with them via a support ticket. They have a good track record when it comes to support for FreeBSD in general. You are absolutely going to need to mail them for such a new controller. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 04:29:45 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F287E106566B for ; Thu, 6 Oct 2011 04:29:45 +0000 (UTC) (envelope-from freebsd@penx.com) Received: from Elmer.dco.penx.com (elmer.dco.penx.com [174.46.214.165]) by mx1.freebsd.org (Postfix) with ESMTP id ADF1B8FC08 for ; Thu, 6 Oct 2011 04:29:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by Elmer.dco.penx.com (8.14.5/8.14.4) with ESMTP id p964TgEJ056418 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 5 Oct 2011 22:29:45 -0600 (MDT) (envelope-from freebsd@penx.com) Date: Wed, 5 Oct 2011 22:29:42 -0600 (MDT) From: Dennis Glatting X-X-Sender: dennisg@Elmer.dco.penx.com To: Jeremy Chadwick In-Reply-To: <20111006040511.GA54920@icarus.home.lan> Message-ID: References: <20111006040511.GA54920@icarus.home.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Odd incident with ZFS and an Aerca 1880i X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 04:29:46 -0000 On Wed, 5 Oct 2011, Jeremy Chadwick wrote: > On Wed, Oct 05, 2011 at 08:37:20PM -0600, Dennis Glatting wrote: >> I have a Aerca 1880i plugged into a Gigabyte GA-X58A-UD7 running >> RELENG_8 with a 10 2TB RAIDz pool with a OCZ Revo L2Arc and a SSD >> ZIL, among other volumes. The last scrub indicated there was an >> error with the pool and today I finally got around to cleaning >> things up. >> >> During the scrub I initiated today, a running "zpool status" >> displayed a message that said I should clear the pool, which I did, >> while the scrub was running, which probably wasn't such a good idea. >> The result was my disks through the Aerca disappeared. On a power >> switch reboot the Aerca no longer displayed its boot-time messages >> and it appears the Aerca's firmware is kaput. >> >> Specifically, during the boot sequence the following: >> >> "arcmsr0: timed out waiting for firmware ready" >> "arcmsr0: wait 'get adapter firmware miscellaneous data' timeout" > > 1) The controller brand/vendor is Areca, not Aerca. (This isn't just a > typo, you said it 3 times in your mail (including Subject). :-) ) > > 2) The arcmsr(4) man page does not mention the 1880i controller being > supported. Readers should note this controller is ***super*** new. > It's a SATA600/SAS controller with PCIe 2.0: > http://www.areca.com.tw/products/1880.htm > > My point:I am not surprised this is not working in RELENG_8. This > controller, again, is **very** new on the market. > I have two of these cards across two systems, both RELENG_8 x86_64 (24GB of RAM in one, 12GB in the other). They have worked mostly without trouble since April (the first card) and July (the second). The only trouble was an ASUS RAMPAGE III EXTREME motherboard where the two cards in the same motherboard caused interrupt storms. When I replaced the ASUS with a Gigabyte GA-X58A-UD7 the storm problem went away but I later moved the second card to a different machine. The second card is in a Gigabyte GA-X58A-UD9, BTW. > 3) Areca provides FreeBSD support natively. You should absolutely Email > them and bring the problem up with them via a support ticket. They have > a good track record when it comes to support for FreeBSD in general. You > are absolutely going to need to mail them for such a new controller. > Already have. I was surprised and confused that somehow I blew the firmware. I figured the fbsd fs wizards would be interested, if not amused. From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 08:29:17 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C35DA1065673 for ; Thu, 6 Oct 2011 08:29:17 +0000 (UTC) (envelope-from srg.gavrilov@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 93C568FC1A for ; Thu, 6 Oct 2011 08:29:17 +0000 (UTC) Received: by iage36 with SMTP id e36so189402iag.13 for ; Thu, 06 Oct 2011 01:29:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=xsTKivzjDZ46uMMYxpIGi6/nKpEJBRPP16FBvzWKMPI=; b=KekBrMDUTxyrKHo+rQjq6C/EoMu30mb4MTm7mCKBORZLzzhWwEKx4ps8F/mK1nOAho 0gwKBcI1yLzcWXGsUy63FSLgIY31d+QSjrxX//h595t2oYwduK4gYYFmSnxZ9yMucWig ETXJc9cIbX2XolkXSjwxcHeoWIa/BOERVxo/A= MIME-Version: 1.0 Received: by 10.231.41.69 with SMTP id n5mr733543ibe.92.1317888270978; Thu, 06 Oct 2011 01:04:30 -0700 (PDT) Received: by 10.231.166.12 with HTTP; Thu, 6 Oct 2011 01:04:30 -0700 (PDT) Date: Thu, 6 Oct 2011 12:04:30 +0400 Message-ID: From: Sergey Gavrilov To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 08:29:17 -0000 Hello, all. After "chown -R" hanging and reboot I cannot mount one of the datasets, while other pools work well. CTRL-T shows state [tx->tx_sync_done_cv)] Hardware: 3ware 9690SA-8I, 8 SEAGATE ST32000445SS in raidz2, 48 Gb RAM, 2 E5520 CPU Software: FreeBSD 8.2-STABLE #3: Wed Oct 5 18:04:50 MSD 2011 /usr/obj/usr/src/sys/GENERIC amd64, ZFS V28 compression=on I can clone and mount any snapshot of the dataset, but not itself. No errors in zpool status or system messages. Scrub has finished clear. Let me know if I could provide any additional information to solve the issue. -- Best regards, Sergey Gavrilov From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 09:25:29 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CB76106564A; Thu, 6 Oct 2011 09:25:29 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (tignes.restart.be [IPv6:2001:41d0:2:56bf:0:1::]) by mx1.freebsd.org (Postfix) with ESMTP id 8EE518FC14; Thu, 6 Oct 2011 09:25:28 +0000 (UTC) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:2:56bf:1:ffff::]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTPS id 3BDAB1385D; Thu, 6 Oct 2011 11:25:27 +0200 (CEST) Received: from morzine.restart.bel (morzine.restart.be [IPv6:2001:41d0:2:56bf:1:2::]) (authenticated bits=0) by restart.be (8.14.5/8.14.5) with ESMTP id p969PQEn096993; Thu, 6 Oct 2011 11:25:26 +0200 (CEST) (envelope-from hlh@restart.be) X-DKIM: Sendmail DKIM Filter v2.8.3 restart.be p969PQEn096993 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1317893126; bh=l/pa9SY+Tn2WSuA/zpoazUztrzGy0bHoZF44LUF89/g=; h=Message-ID:Date:From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=DSfpE5EnidyvfaYy6eI82SrLUIF2kS3X+bvn1nEqcpQeZzVB8gP0ysd5yZ3jKqvkr Uvm2UsCWS/UdPV3iktdog== X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 restart.be p969PQEn096993 DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to: subject:content-type:content-transfer-encoding; b=Lur+NJgv0r2cX+5sudgC5/S6r1A71JaffZn7U7AqMsc9K0i0Ze1ccKDWwgr7xokRC n0Bn+eKRT8GoIBcBq0DRA== Message-ID: <4E8D7406.4090302@restart.be> Date: Thu, 06 Oct 2011 11:25:26 +0200 From: Henri Hennebert Organization: RestartSoft User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:6.0.2) Gecko/20110911 Thunderbird/6.0.2 MIME-Version: 1.0 To: current@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 09:25:29 -0000 Hello all, I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: ZFS: i/o error - all block copies unavailable can't open '/boot/menu.rc': no such file or directory. I pxe boot mfsbsd 8.2-RELEASE + zfs v28 then: mkdir /rpool zpool import -R /rpool rpool mount -t zfs rpool/boot /mnt mv /mnt/boot/menu.rc /mnt/boot/Menu.rc cp /mnt/boot/Menu.rc /mnt/boot/menu.rc umount /mnt shutdown -r now and the next boot run smoothly... PS. I have to boot from pxe because after the i/o error, the kernel was booted but encounter a double fault. I try 5 times with the same result. I think that on my configuration only the kernel without ACPI encouter this trap. Henri From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 10:45:04 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB0F21065672 for ; Thu, 6 Oct 2011 10:45:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C77C08FC15 for ; Thu, 6 Oct 2011 10:45:03 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA00490; Thu, 06 Oct 2011 13:44:51 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E8D86A2.1040508@FreeBSD.org> Date: Thu, 06 Oct 2011 13:44:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: Henri Hennebert References: <4E8D7406.4090302@restart.be> In-Reply-To: <4E8D7406.4090302@restart.be> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 10:45:04 -0000 on 06/10/2011 12:25 Henri Hennebert said the following: > Hello all, > > I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: > > ZFS: i/o error - all block copies unavailable > can't open '/boot/menu.rc': no such file or directory. > > I pxe boot mfsbsd 8.2-RELEASE + zfs v28 > > then: > > mkdir /rpool > zpool import -R /rpool rpool > mount -t zfs rpool/boot /mnt > mv /mnt/boot/menu.rc /mnt/boot/Menu.rc > cp /mnt/boot/Menu.rc /mnt/boot/menu.rc > umount /mnt > > shutdown -r now > > and the next boot run smoothly... Does your root fs have compression enabled? > PS. > I have to boot from pxe because after the i/o error, the kernel was booted but > encounter a double fault. I try 5 times with the same result. > > I think that on my configuration only the kernel without ACPI encouter this trap. I think that you should try to gather and report more information about the panic (separately). -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 12:30:18 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BF2B1065672; Thu, 6 Oct 2011 12:30:18 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (tignes.restart.be [IPv6:2001:41d0:2:56bf:0:1::]) by mx1.freebsd.org (Postfix) with ESMTP id BB6D98FC15; Thu, 6 Oct 2011 12:30:17 +0000 (UTC) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:2:56bf:1:ffff::]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTPS id 0F019135E9; Thu, 6 Oct 2011 14:30:17 +0200 (CEST) Received: from morzine.restart.bel (morzine.restart.be [IPv6:2001:41d0:2:56bf:1:2::]) (authenticated bits=0) by restart.be (8.14.5/8.14.5) with ESMTP id p96CUFmN001256; Thu, 6 Oct 2011 14:30:15 +0200 (CEST) (envelope-from hlh@restart.be) X-DKIM: Sendmail DKIM Filter v2.8.3 restart.be p96CUFmN001256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1317904215; bh=WtaaYbvTgrRZeHQCjJhzPDoxyVnylZ6/7taNPTmXJSU=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=UAZOJkh5Yj/GhOi+TUGnYptckAgSM2iU8ZtLHDWcxS0OOV5wdS6/E8VEnhXjqOLmb 2NmyVkSthcexIW8MGSMaQ== X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 restart.be p96CUFmN001256 DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to:cc: subject:references:in-reply-to:content-type:content-transfer-encoding; b=tITDobZ4PYm4pXSAQKgOBJBM7DLGC5mXbiAyFVb6VCjurQwo670ZNLH3sM43YLZ1H pEaTQCFQf7n1KXpgxgBHQ== Message-ID: <4E8D9F57.70506@restart.be> Date: Thu, 06 Oct 2011 14:30:15 +0200 From: Henri Hennebert Organization: RestartSoft User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:6.0.2) Gecko/20110911 Thunderbird/6.0.2 MIME-Version: 1.0 To: Andriy Gapon References: <4E8D7406.4090302@restart.be> <4E8D86A2.1040508@FreeBSD.org> In-Reply-To: <4E8D86A2.1040508@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 12:30:18 -0000 On 10/06/2011 12:44, Andriy Gapon wrote: > on 06/10/2011 12:25 Henri Hennebert said the following: >> Hello all, >> >> I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: >> >> ZFS: i/o error - all block copies unavailable >> can't open '/boot/menu.rc': no such file or directory. >> >> I pxe boot mfsbsd 8.2-RELEASE + zfs v28 >> >> then: >> >> mkdir /rpool >> zpool import -R /rpool rpool >> mount -t zfs rpool/boot /mnt >> mv /mnt/boot/menu.rc /mnt/boot/Menu.rc >> cp /mnt/boot/Menu.rc /mnt/boot/menu.rc >> umount /mnt >> >> shutdown -r now >> >> and the next boot run smoothly... > > Does your root fs have compression enabled? The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAME PROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 > >> PS. >> I have to boot from pxe because after the i/o error, the kernel was booted but >> encounter a double fault. I try 5 times with the same result. >> >> I think that on my configuration only the kernel without ACPI encouter this trap. > > I think that you should try to gather and report more information about the panic > (separately). I'll try this saterday From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 13:36:42 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13CA1106564A; Thu, 6 Oct 2011 13:36:42 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 344868FC13; Thu, 6 Oct 2011 13:36:40 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA03892; Thu, 06 Oct 2011 16:36:37 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E8DAEE5.4020004@FreeBSD.org> Date: Thu, 06 Oct 2011 16:36:37 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: Henri Hennebert References: <4E8D7406.4090302@restart.be> <4E8D86A2.1040508@FreeBSD.org> <4E8D9F57.70506@restart.be> In-Reply-To: <4E8D9F57.70506@restart.be> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 13:36:42 -0000 on 06/10/2011 15:30 Henri Hennebert said the following: > The pool is a mirror: > > [root@morzine ~]# zpool status rpool > pool: rpool > state: ONLINE > scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 > config: > > NAME STATE READ WRITE CKSUM > rpool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 > gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 > > errors: No known data errors > > and rpool/root is not compressed: > > [root@morzine ~]# zfs get compression rpool/root > NAME PROPERTY VALUE SOURCE > rpool/root compression off inherited from rpool > > pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 14:00:07 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3EB81065674; Thu, 6 Oct 2011 14:00:07 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (tignes.restart.be [IPv6:2001:41d0:2:56bf:0:1::]) by mx1.freebsd.org (Postfix) with ESMTP id 32D7E8FC1D; Thu, 6 Oct 2011 14:00:07 +0000 (UTC) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:2:56bf:1:ffff::]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTPS id 8081213BC8; Thu, 6 Oct 2011 16:00:06 +0200 (CEST) Received: from morzine.restart.bel (morzine.restart.be [IPv6:2001:41d0:2:56bf:1:2::]) (authenticated bits=0) by restart.be (8.14.5/8.14.5) with ESMTP id p96E04b2003373; Thu, 6 Oct 2011 16:00:05 +0200 (CEST) (envelope-from hlh@restart.be) X-DKIM: Sendmail DKIM Filter v2.8.3 restart.be p96E04b2003373 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1317909605; bh=jEHSuhXQK2CbsUuV6ksfFRaUx9ZVtUylB9Hta/mAKkE=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=XRvW3DRiKwBbVyZgm7F7PsjajlwlXieEXXOv+eKhZTb0fBMvlkmGblctIzMRprBBt btUd8aak0fdxidGAUhJWA== X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 restart.be p96E04b2003373 DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to:cc: subject:references:in-reply-to:content-type:content-transfer-encoding; b=ja+/4H3RXDXIF3SqdXj+WxPGl1dFcgJ0v0uRk4DTvKRTaeDMJl0f0xMl08GnU5GaK VsRfwSZj7Zj1R8BMS/WCg== Message-ID: <4E8DB464.80202@restart.be> Date: Thu, 06 Oct 2011 16:00:04 +0200 From: Henri Hennebert Organization: RestartSoft User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:6.0.2) Gecko/20110911 Thunderbird/6.0.2 MIME-Version: 1.0 To: Andriy Gapon References: <4E8D7406.4090302@restart.be> <4E8D86A2.1040508@FreeBSD.org> <4E8D9F57.70506@restart.be> <4E8DAEE5.4020004@FreeBSD.org> In-Reply-To: <4E8DAEE5.4020004@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 14:00:08 -0000 On 10/06/2011 15:36, Andriy Gapon wrote: > on 06/10/2011 15:30 Henri Hennebert said the following: >> The pool is a mirror: >> >> [root@morzine ~]# zpool status rpool >> pool: rpool >> state: ONLINE >> scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 >> config: >> >> NAME STATE READ WRITE CKSUM >> rpool ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >> gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >> >> errors: No known data errors >> >> and rpool/root is not compressed: >> >> [root@morzine ~]# zfs get compression rpool/root >> NAME PROPERTY VALUE SOURCE >> rpool/root compression off inherited from rpool >> >> pool is v28 and filesystems are v5 > > No particular recipes for this environment, just a general suggestion. > If you run into a situation like this again, please try to use > tools/tools/zfsboottest to diagnose where exactly an error originates. > I try [ please note _M_enu.rc ]: [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest /boot/Menu.rc /dev/da0p2 /dev/da1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE \ Menu.rc \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ \ \ Load required Forth modules include /boot/version.4th include /boot/brand.4th include /boot/menu.4th include /boot/menu-commands.4th include /boot/shortcuts.4th \ Screen prep clear \ clear the screen (see `screen.4th') print_version \ print version string (bottom-right; see `version.4th') draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') draw-brand \ draw the FreeBSD title (top-left; see `brand.4th') menu-init \ initialize the menu area (see `menu.4th') \ Initialize main menu constructs (see `menu.4th') \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) set menu_timeout_command="boot" \ Display the main menu (see `menu.4th') menu-display [root@morzine ~] The line `ZFS: SPA version 28' come from my local patch: Index: sys/boot/zfs/zfsimpl.c =================================================================== --- sys/boot/zfs/zfsimpl.c (revision 225759) +++ sys/boot/zfs/zfsimpl.c (working copy) @@ -63,6 +63,8 @@ STAILQ_INIT(&zfs_vdevs); STAILQ_INIT(&zfs_pools); + printf("ZFS: SPA version %u\n", (unsigned) SPA_VERSION); + zfs_temp_buf = malloc(TEMP_SIZE); zfs_temp_end = zfs_temp_buf + TEMP_SIZE; zfs_temp_ptr = zfs_temp_buf; Is it what you sugest ? From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 14:20:08 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1AB2106564A; Thu, 6 Oct 2011 14:20:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E93738FC19; Thu, 6 Oct 2011 14:20:07 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA04529; Thu, 06 Oct 2011 17:20:05 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E8DB915.6000502@FreeBSD.org> Date: Thu, 06 Oct 2011 17:20:05 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: Henri Hennebert References: <4E8D7406.4090302@restart.be> <4E8D86A2.1040508@FreeBSD.org> <4E8D9F57.70506@restart.be> <4E8DAEE5.4020004@FreeBSD.org> <4E8DB464.80202@restart.be> In-Reply-To: <4E8DB464.80202@restart.be> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 14:20:09 -0000 on 06/10/2011 17:00 Henri Hennebert said the following: > On 10/06/2011 15:36, Andriy Gapon wrote: >> on 06/10/2011 15:30 Henri Hennebert said the following: >>> The pool is a mirror: >>> >>> [root@morzine ~]# zpool status rpool >>> pool: rpool >>> state: ONLINE >>> scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> rpool ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >>> gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> >>> and rpool/root is not compressed: >>> >>> [root@morzine ~]# zfs get compression rpool/root >>> NAME PROPERTY VALUE SOURCE >>> rpool/root compression off inherited from rpool >>> >>> pool is v28 and filesystems are v5 >> >> No particular recipes for this environment, just a general suggestion. >> If you run into a situation like this again, please try to use >> tools/tools/zfsboottest to diagnose where exactly an error originates. >> > I try [ please note _M_enu.rc ]: > > [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest > /boot/Menu.rc /dev/da0p2 /dev/da1p2 > ZFS: SPA version 28 > pool: rpool > config: > > NAME STATE > rpool ONLINE > mirror ONLINE > gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE > gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE > \ Menu.rc > \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ > \ > \ Load required Forth modules > include /boot/version.4th > include /boot/brand.4th > include /boot/menu.4th > include /boot/menu-commands.4th > include /boot/shortcuts.4th > > \ Screen prep > clear \ clear the screen (see `screen.4th') > print_version \ print version string (bottom-right; see `version.4th') > draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') > draw-brand \ draw the FreeBSD title (top-left; see `brand.4th') > menu-init \ initialize the menu area (see `menu.4th') > > \ Initialize main menu constructs (see `menu.4th') > \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) > > set menu_timeout_command="boot" > > \ Display the main menu (see `menu.4th') > menu-display > [root@morzine ~] > > The line `ZFS: SPA version 28' > > come from my local patch: > > Index: sys/boot/zfs/zfsimpl.c > =================================================================== > --- sys/boot/zfs/zfsimpl.c (revision 225759) > +++ sys/boot/zfs/zfsimpl.c (working copy) > @@ -63,6 +63,8 @@ > STAILQ_INIT(&zfs_vdevs); > STAILQ_INIT(&zfs_pools); > > + printf("ZFS: SPA version %u\n", (unsigned) SPA_VERSION); > + > zfs_temp_buf = malloc(TEMP_SIZE); > zfs_temp_end = zfs_temp_buf + TEMP_SIZE; > zfs_temp_ptr = zfs_temp_buf; > > > Is it what you sugest ? Yes. And this report indicates that the boot code (built from your source tree) should be able to read that file. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 18:15:38 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F5D51065672 for ; Thu, 6 Oct 2011 18:15:38 +0000 (UTC) (envelope-from tomi.pievilainen@iki.fi) Received: from mail.kapsi.fi (mx1.kapsi.fi [IPv6:2001:1bc8:1004::1:25]) by mx1.freebsd.org (Postfix) with ESMTP id E3F1F8FC13 for ; Thu, 6 Oct 2011 18:15:37 +0000 (UTC) Received: from hoasnet-fe2edd00-136.dhcp.inet.fi ([80.221.46.136] helo=shirio.kyla.fi) by mail.kapsi.fi with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RBsTn-0002Fo-Ko for freebsd-fs@freebsd.org; Thu, 06 Oct 2011 21:15:35 +0300 Received: by shirio.kyla.fi (Postfix, from userid 1000) id 43922456EF; Thu, 6 Oct 2011 21:15:35 +0300 (EEST) Date: Thu, 6 Oct 2011 21:15:35 +0300 From: Tomi =?iso-8859-1?Q?Pievil=E4inen?= To: freebsd-fs@freebsd.org Message-ID: <20111006181535.GE9549@shirio.kyla.fi> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GLp9dJVi+aaipsRk" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: 80.221.46.136 X-SA-Exim-Mail-From: tomi.pievilainen@iki.fi X-SA-Exim-Scanned: No (on mail.kapsi.fi); SAEximRunCond expanded to false Subject: System freezes when trying to enable ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 18:15:38 -0000 --GLp9dJVi+aaipsRk Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I recently had a system disk break and had to do a reinstall. The machine also had a five disk raidz pool for files, that worked fine after the reinstall to a new disk and doing an zfs import. However after booting the machine again, doing a zfs list or zpool list "hangs" the machine. A message about prefetch problems, filesystem version 4 and pool version 15 is printed to the first console, then I can change between consoles and write to the first console, but cannot log in via ssh or other consoles (trying to write the username outputs nothing in console, and ssh never asks for password). The machine does answer to ping however, so it's not really completely out of the network either. I haven't figured out any other way to unfreeze the machine except hard reboot, after which the problem can be reproduced by doing zfs/zpool list. Before that the machine works fine. Any idea what's going on? --=20 Tomi Pievil=E4inen, +358 400 487 504 A: Because it disrupts the natural way of thinking. Q: Why is top posting frowned upon? --GLp9dJVi+aaipsRk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk6N8EcACgkQeCRKnjAFERcjJQCfdOtCOwc3PiKcLd0rFxEnkWWA YT0Ani4yAMEzjeDs0f2JFpqrsR7ukx35 =RU19 -----END PGP SIGNATURE----- --GLp9dJVi+aaipsRk-- From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 18:17:50 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BE0F1065672 for ; Thu, 6 Oct 2011 18:17:50 +0000 (UTC) (envelope-from tomi.pievilainen@iki.fi) Received: from mail.kapsi.fi (mx1.kapsi.fi [IPv6:2001:1bc8:1004::1:25]) by mx1.freebsd.org (Postfix) with ESMTP id C9EAD8FC15 for ; Thu, 6 Oct 2011 18:17:49 +0000 (UTC) Received: from hoasnet-fe2edd00-136.dhcp.inet.fi ([80.221.46.136] helo=shirio.kyla.fi) by mail.kapsi.fi with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RBsVx-0002QH-2J for freebsd-fs@freebsd.org; Thu, 06 Oct 2011 21:17:49 +0300 Received: by shirio.kyla.fi (Postfix, from userid 1000) id B3FED456EF; Thu, 6 Oct 2011 21:17:48 +0300 (EEST) Date: Thu, 6 Oct 2011 21:17:48 +0300 From: Tomi =?iso-8859-1?Q?Pievil=E4inen?= To: freebsd-fs@freebsd.org Message-ID: <20111006181748.GF9549@shirio.kyla.fi> References: <20111006181535.GE9549@shirio.kyla.fi> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GUPx2O/K0ibUojHx" Content-Disposition: inline In-Reply-To: <20111006181535.GE9549@shirio.kyla.fi> User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: 80.221.46.136 X-SA-Exim-Mail-From: tomi.pievilainen@iki.fi X-SA-Exim-Scanned: No (on mail.kapsi.fi); SAEximRunCond expanded to false Subject: Re: System freezes when trying to enable ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 18:17:50 -0000 --GUPx2O/K0ibUojHx Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 06, 2011 at 09:15:35PM +0300, Tomi Pievil=E4inen wrote: > I recently had a system disk break and had to do a reinstall. The > machine also had a five disk raidz pool for files, that worked fine > after the reinstall to a new disk and doing an zfs import. However > after booting the machine again, doing a zfs list or zpool list > "hangs" the machine. >=20 > A message about prefetch problems, filesystem version 4 and pool > version 15 is printed to the first console, then I can change between > consoles and write to the first console, but cannot log in via ssh or > other consoles (trying to write the username outputs nothing in > console, and ssh never asks for password). The machine does answer to > ping however, so it's not really completely out of the network either. >=20 > I haven't figured out any other way to unfreeze the machine except > hard reboot, after which the problem can be reproduced by doing > zfs/zpool list. Before that the machine works fine. Any idea what's > going on? Oh, and if it matters, the disks in the raidz pool give errors like "secondary GPT table is corrupt or invalid" and "secondary GPT header is not in the last LBA" when booting. Not sure if these happened already before I did the reinstall, since I hadn't had any problems and hadn't checked the dmesg then. --=20 Tomi Pievil=E4inen, +358 400 487 504 A: Because it disrupts the natural way of thinking. Q: Why is top posting frowned upon? --GUPx2O/K0ibUojHx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk6N8MwACgkQeCRKnjAFERfUHACePX7DL3sgADPQAQIMh1EXyAh8 AB4AoIgOY8mAYlqTDEVPzkORVJK0RWXE =NAkV -----END PGP SIGNATURE----- --GUPx2O/K0ibUojHx-- From owner-freebsd-fs@FreeBSD.ORG Thu Oct 6 23:44:39 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25D13106564A for ; Thu, 6 Oct 2011 23:44:39 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id DB7378FC12 for ; Thu, 6 Oct 2011 23:44:38 +0000 (UTC) Received: by ggeq3 with SMTP id q3so2846945gge.13 for ; Thu, 06 Oct 2011 16:44:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=25/XSikQC9wwK3O4EX4jndVOzeCz6t3gWCmfRXrb4yw=; b=JKRCO23cpyLuQukwmMql0l/s017tVZ+2f+LAS4mU0eHWLX1PMgyB73yIyNBdycW93W lpTneGz0DY1WDAcGTQzR556nFfXe2WPUkHuksgdnXUBJgUa+EsiqESN9tiLl71hl/v8O nxFJDadumVsZ+Lx3e7uf3hbTaogPdBN6lVm7A= MIME-Version: 1.0 Received: by 10.151.93.10 with SMTP id v10mr1063050ybl.449.1317944678176; Thu, 06 Oct 2011 16:44:38 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Thu, 6 Oct 2011 16:44:38 -0700 (PDT) In-Reply-To: References: Date: Thu, 6 Oct 2011 16:44:38 -0700 Message-ID: From: Xin LI To: Sergey Gavrilov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 23:44:39 -0000 On Thu, Oct 6, 2011 at 1:04 AM, Sergey Gavrilov wr= ote: > Hello, all. > > After "chown -R" hanging and reboot I cannot mount one of the > datasets, while other pools work well. > CTRL-T shows state [tx->tx_sync_done_cv)] > > Hardware: 3ware 9690SA-8I, 8 SEAGATE ST32000445SS in raidz2, 48 Gb > RAM, 2 E5520 CPU > Software: FreeBSD 8.2-STABLE #3: Wed Oct =C2=A05 18:04:50 MSD 2011 > /usr/obj/usr/src/sys/GENERIC amd64, ZFS V28 compression=3Don > > I can clone and mount any snapshot of the dataset, but not itself. > No errors in zpool status or system messages. Scrub has finished clear. > > Let me know if I could provide any additional information to solve the is= sue. What exactly does 'zpool status' say (your pool configuration IS needed)? Cheers, --=20 Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Fri Oct 7 07:37:43 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87634106564A for ; Fri, 7 Oct 2011 07:37:43 +0000 (UTC) (envelope-from srg.gavrilov@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 534888FC1F for ; Fri, 7 Oct 2011 07:37:43 +0000 (UTC) Received: by iage36 with SMTP id e36so1819411iag.13 for ; Fri, 07 Oct 2011 00:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=WkR7fQsJwRyCUdYtdQsVc4PIrHJ0fN05sVOrDBxgFdA=; b=LDOfIwzoXOxOyS+pjxO0AC7fQv2KirTDjcDDwZwgg24GLxe/yETYBu8v7zDTHqyTVk YUQCNJlPKuBehxdoG6lkJ3oFJ9agNFpKggYQ5UhoxKwz6JNWriRrvIrGzUTk1ltI7qAL F99NZC5olKoR7ekNj7xK0Z6KB1mlKBaMTr6lA= MIME-Version: 1.0 Received: by 10.231.41.69 with SMTP id n5mr2717738ibe.92.1317973061411; Fri, 07 Oct 2011 00:37:41 -0700 (PDT) Received: by 10.231.166.12 with HTTP; Fri, 7 Oct 2011 00:37:41 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 11:37:41 +0400 Message-ID: From: Sergey Gavrilov To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 07:37:43 -0000 pool: pool2 state: ONLINE scan: scrub repaired 0 in 7h29m with 0 errors on Thu Oct 6 02:15:50 2011 config: NAME STATE READ WRITE CKSUM pool2 ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 errors: No known data errors 2011/10/7 Xin LI > On Thu, Oct 6, 2011 at 1:04 AM, Sergey Gavrilov > wrote: > > Hello, all. > > > > After "chown -R" hanging and reboot I cannot mount one of the > > datasets, while other pools work well. > > CTRL-T shows state [tx->tx_sync_done_cv)] > > > > Hardware: 3ware 9690SA-8I, 8 SEAGATE ST32000445SS in raidz2, 48 Gb > > RAM, 2 E5520 CPU > > Software: FreeBSD 8.2-STABLE #3: Wed Oct 5 18:04:50 MSD 2011 > > /usr/obj/usr/src/sys/GENERIC amd64, ZFS V28 compression=on > > > > I can clone and mount any snapshot of the dataset, but not itself. > > No errors in zpool status or system messages. Scrub has finished clear. > > > > Let me know if I could provide any additional information to solve the > issue. > > What exactly does 'zpool status' say (your pool configuration IS needed)? > > Cheers, > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -- Best regards, Sergey Gavrilov From owner-freebsd-fs@FreeBSD.ORG Fri Oct 7 07:41:13 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 201FA106566B for ; Fri, 7 Oct 2011 07:41:13 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id D37A88FC12 for ; Fri, 7 Oct 2011 07:41:12 +0000 (UTC) Received: by ywp17 with SMTP id 17so4263845ywp.13 for ; Fri, 07 Oct 2011 00:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=VIL/9DF9Be2lhZkSCJOtaaLoZbjFCO/RHvpw+7BnfWM=; b=NtMlUHz2YxQR2CZUl3sxDx83mcLtdhgrAzMUMVW6A9vT8k6kOxmY1xz2C57ivBkkuB 1K4TnJ4lJ+LYCXUgq9+HzVG5a3NO2RGh2RSWEwNM4LB6JmVuLkW5rtdKRfT5kMGcBX/l K03jjQFQUekqyo8bDt/VZrRyzDdFMWq8fkBKc= MIME-Version: 1.0 Received: by 10.150.238.18 with SMTP id l18mr1201198ybh.335.1317973271967; Fri, 07 Oct 2011 00:41:11 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Fri, 7 Oct 2011 00:41:11 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 00:41:11 -0700 Message-ID: From: Xin LI To: Sergey Gavrilov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 07:41:13 -0000 T24gRnJpLCBPY3QgNywgMjAxMSBhdCAxMjozNyBBTSwgU2VyZ2V5IEdhdnJpbG92IDxzcmcuZ2F2 cmlsb3ZAZ21haWwuY29tPiB3cm90ZToKPiBwb29sOiBwb29sMgo+IMKgc3RhdGU6IE9OTElORQo+ IMKgc2Nhbjogc2NydWIgcmVwYWlyZWQgMCBpbiA3aDI5bSB3aXRoIDAgZXJyb3JzIG9uIFRodSBP Y3QgwqA2IDAyOjE1OjUwIDIwMTEKPiBjb25maWc6Cj4KPiDCoCDCoE5BTUUgwqAgwqAgwqAgwqBT VEFURSDCoCDCoCBSRUFEIFdSSVRFIENLU1VNCj4gwqAgwqBwb29sMiDCoCDCoCDCoCBPTkxJTkUg wqAgwqAgwqAgMCDCoCDCoCAwIMKgIMKgIDAKPiDCoCDCoCDCoHJhaWR6Mi0wIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTkgwqAgwqAgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTEwIMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTExIMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTEyIMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTEzIMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTE0IMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTE1IMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4gwqAgwqAgwqAgwqBkYTE2IMKgIMKgT05MSU5FIMKg IMKgIMKgIDAgwqAgwqAgMCDCoCDCoCAwCj4KPiBlcnJvcnM6IE5vIGtub3duIGRhdGEgZXJyb3Jz CgpPa2F5Li4uIGNhbiB5b3UgYWxzbyBkbyAncHJvY3N0YXQgLWtrIC1hJyB3aXRoIHJvb3Qgd2hl biB6ZnMgcHJvY2VzcwpoYW5ncyB3aXRoIFt0eC0+dHhfc3luY19kb25lX2N2KV0/CgpDaGVlcnMs Ci0tIApYaW4gTEkgPGRlbHBoaWpAZGVscGhpai5uZXQ+IGh0dHBzOi8vd3d3LmRlbHBoaWoubmV0 LwpGcmVlQlNEIC0gVGhlIFBvd2VyIHRvIFNlcnZlISBMaXZlIGZyZWUgb3IgZGllCg== From owner-freebsd-fs@FreeBSD.ORG Fri Oct 7 07:52:35 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83C8D106566B for ; Fri, 7 Oct 2011 07:52:35 +0000 (UTC) (envelope-from srg.gavrilov@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4CD9C8FC13 for ; Fri, 7 Oct 2011 07:52:34 +0000 (UTC) Received: by iage36 with SMTP id e36so1836882iag.13 for ; Fri, 07 Oct 2011 00:52:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=x7GSoxDuAf0IHe6Tl44WgE2ViuDKAA2RDMkAt9ytkjM=; b=WUw/enVXltrWCZer1aMxBiPonTgcy0zud+3eaZz4g/KEjcrO6hpiTUIJKEZw9QG0jc V92GUZ7D3cSTlE6sMsuRdhgyFzdQo129u+T83mGzchlK48QTHqTB9H3qtnQGWqLdxEJG r0cXEblKN+Iufxwtt0ipFRh6fkkCRRxtfXGVc= MIME-Version: 1.0 Received: by 10.42.145.7 with SMTP id d7mr10577704icv.14.1317973953882; Fri, 07 Oct 2011 00:52:33 -0700 (PDT) Received: by 10.231.166.12 with HTTP; Fri, 7 Oct 2011 00:52:33 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 11:52:33 +0400 Message-ID: From: Sergey Gavrilov To: Xin LI Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 07:52:35 -0000 There are too many running processes right now. Will it be enough procstat for hanged process only? sudo zfs set mountpoint=/pool2/samba/NMO pool2/samba/NMO ps ax | grep zfs 29 ?? DL 88:26,87 [zfskern] 1273 ?? Is 0:00,00 /usr/sbin/mountd -r /etc/exports /etc/zfs/exports 11672 1 I+ 0:00,01 sudo zfs set mountpoint=/pool2/samba/NMO pool2/samba/NMO 11673 1 D+ 0:00,03 zfs set mountpoint=/pool2/samba/NMO pool2/samba/NMO 11817 2 S+ 0:00,00 grep zfs sudo procstat -kk 11673 PID TID COMM TDNAME KSTACK 11673 101295 zfs initial thread mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_wait_synced+0x85 zil_replay_log_record+0xe1 zil_parse+0x3d8 zil_replay+0xe5 zfsvfs_setup+0x117 zfs_mount+0x52f vfs_donmount+0xdc5 nmount+0x63 amd64_syscall+0x1f4 Xfast_syscall+0xfc And else. I'd sent dataset to another pool and it had mounted and works well. 2011/10/7 Xin LI > On Fri, Oct 7, 2011 at 12:37 AM, Sergey Gavrilov > wrote: > > pool: pool2 > > state: ONLINE > > scan: scrub repaired 0 in 7h29m with 0 errors on Thu Oct 6 02:15:50 > 2011 > > config: > > > > NAME STATE READ WRITE CKSUM > > pool2 ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > da9 ONLINE 0 0 0 > > da10 ONLINE 0 0 0 > > da11 ONLINE 0 0 0 > > da12 ONLINE 0 0 0 > > da13 ONLINE 0 0 0 > > da14 ONLINE 0 0 0 > > da15 ONLINE 0 0 0 > > da16 ONLINE 0 0 0 > > > > errors: No known data errors > > Okay... can you also do 'procstat -kk -a' with root when zfs process > hangs with [tx->tx_sync_done_cv)]? > > Cheers, > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -- Best regards, Sergey Gavrilov From owner-freebsd-fs@FreeBSD.ORG Fri Oct 7 08:11:51 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 081531065670 for ; Fri, 7 Oct 2011 08:11:51 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id AD7B78FC14 for ; Fri, 7 Oct 2011 08:11:48 +0000 (UTC) Received: by gyf2 with SMTP id 2so4315134gyf.13 for ; Fri, 07 Oct 2011 01:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SivBD2/JHfMtI6YbgZtucpLyUUmN2MHArACXMgF0dSM=; b=rFydecTwMIhwB3izpYL0W1B3iLyL/Ao50gNyeFZbcfeePpdgZe3s44CUl95UtPiE8E ggRS1WtJJ3PnbeqCzSNhn4RbDw0Hb3MqNZxXn2PbRFQ71/VfgBdu9T/VBEPvH4deR9vg M1Fov1RUt3zABr+Ty+yNBOBmyQj5hkXiMKQVU= MIME-Version: 1.0 Received: by 10.150.61.11 with SMTP id j11mr1233379yba.164.1317975107093; Fri, 07 Oct 2011 01:11:47 -0700 (PDT) Received: by 10.151.145.18 with HTTP; Fri, 7 Oct 2011 01:11:47 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 01:11:47 -0700 Message-ID: From: Xin LI To: Sergey Gavrilov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 08:11:51 -0000 Hi, On Fri, Oct 7, 2011 at 12:52 AM, Sergey Gavrilov wrote: > There are too many running processes right now. Will it be enough procstat > for hanged process only? No but output from procstat -kk 29 is likely to be helpful if you can't paste the full traces. Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-fs@FreeBSD.ORG Fri Oct 7 09:27:06 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 024D3106566B for ; Fri, 7 Oct 2011 09:27:06 +0000 (UTC) (envelope-from srg.gavrilov@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id BBDBE8FC08 for ; Fri, 7 Oct 2011 09:27:05 +0000 (UTC) Received: by iage36 with SMTP id e36so1948336iag.13 for ; Fri, 07 Oct 2011 02:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=E3OGPU2bQ97/ppu9g3fUXhXjJImS2cItkeOEGoskRyg=; b=A6AohTfknTdmR7ZWRNsB/ucBfTWwPYg8IWGQfvGc2UFQeUWBzs27Y5vGF4OmI/QEhq j4n939jDdTAitkqzLBfB+ARuEfBm5mlK6NnEMu/9ooLByvRVaF1tMjtX0tjZ5khzB+Ns Z7gm0nAbAWPxZGZXcjcmF3I7ZAHC9kutxQyHQ= MIME-Version: 1.0 Received: by 10.231.28.106 with SMTP id l42mr2897759ibc.66.1317979625373; Fri, 07 Oct 2011 02:27:05 -0700 (PDT) Received: by 10.231.166.12 with HTTP; Fri, 7 Oct 2011 02:27:05 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 13:27:05 +0400 Message-ID: From: Sergey Gavrilov To: Xin LI Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 09:27:06 -0000 PID TID COMM TDNAME KSTACK 29 100137 zfskern arc_reclaim_thre mi_switch+0x176 sleepq_timedwait+0x42 _cv_timedwait+0x134 arc_reclaim_thread+0x29d fork_exit+0x11f fork_trampoline+0xe 29 100138 zfskern l2arc_feed_threa mi_switch+0x176 sleepq_timedwait+0x42 _cv_timedwait+0x134 l2arc_feed_thread+0x1a8 fork_exit+0x11f fork_trampoline+0xe 29 100305 zfskern txg_thread_enter mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_thread_wait+0x79 txg_quiesce_thread+0xb5 fork_exit+0x11f fork_trampoline+0xe 29 100306 zfskern txg_thread_enter mi_switch+0x176 sleepq_timedwait+0x42 _cv_timedwait+0x134 txg_thread_wait+0x3c txg_sync_thread+0x269 fork_exit+0x11f fork_trampoline+0xe 29 100473 zfskern txg_thread_enter mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_quiesce_thread+0x1fb fork_exit+0x11f fork_trampoline+0xe 29 100474 zfskern txg_thread_enter mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_thread_wait+0x79 txg_sync_thread+0x2f0 fork_exit+0x11f fork_trampoline+0xe 29 100641 zfskern txg_thread_enter mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_thread_wait+0x79 txg_quiesce_thread+0xb5 fork_exit+0x11f fork_trampoline+0xe 29 100642 zfskern txg_thread_enter mi_switch+0x176 sleepq_timedwait+0x42 _cv_timedwait+0x134 txg_thread_wait+0x3c txg_sync_thread+0x269 fork_exit+0x11f fork_trampoline+0xe 2011/10/7 Xin LI > Hi, > > On Fri, Oct 7, 2011 at 12:52 AM, Sergey Gavrilov > wrote: > > There are too many running processes right now. Will it be enough > procstat > > for hanged process only? > > No but output from procstat -kk 29 is likely to be helpful if you > can't paste the full traces. > > Cheers, > -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -- Best regards, Sergey Gavrilov From owner-freebsd-fs@FreeBSD.ORG Sat Oct 8 08:15:49 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27F30106566C; Sat, 8 Oct 2011 08:15:49 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 005428FC12; Sat, 8 Oct 2011 08:15:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p988FmqW052440; Sat, 8 Oct 2011 08:15:48 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p988Fmm1052431; Sat, 8 Oct 2011 08:15:48 GMT (envelope-from gavin) Date: Sat, 8 Oct 2011 08:15:48 GMT Message-Id: <201110080815.p988Fmm1052431@freefall.freebsd.org> To: gavin@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/160860: Random UFS root filesystem corruption with SU+J [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 08:15:49 -0000 Old Synopsis: Random UFS root filesystem corruption! New Synopsis: Random UFS root filesystem corruption with SU+J [regression] Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: gavin Responsible-Changed-When: Sat Oct 8 08:13:45 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). To submitter: there have been several fixes to the SU+J code committed since 9.0-BETA2, which may well fix the issues you are seeing. Are you able to retest? http://www.freebsd.org/cgi/query-pr.cgi?pr=160860 From owner-freebsd-fs@FreeBSD.ORG Sat Oct 8 09:20:13 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E2B31065670; Sat, 8 Oct 2011 09:20:13 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (tignes.restart.be [IPv6:2001:41d0:2:56bf:0:1::]) by mx1.freebsd.org (Postfix) with ESMTP id 94CA88FC0C; Sat, 8 Oct 2011 09:20:12 +0000 (UTC) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:2:56bf:1:ffff::]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTPS id C7F6A1371A; Sat, 8 Oct 2011 11:20:11 +0200 (CEST) Received: from morzine.restart.bel (morzine.restart.be [IPv6:2001:41d0:2:56bf:1:2::]) (authenticated bits=0) by restart.be (8.14.5/8.14.5) with ESMTP id p989KAYn065029; Sat, 8 Oct 2011 11:20:10 +0200 (CEST) (envelope-from hlh@restart.be) X-DKIM: Sendmail DKIM Filter v2.8.3 restart.be p989KAYn065029 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1318065611; bh=yKdpnW2VTVjo35o4bRN6vA+zEZFzWbhk1dx+fcwQhLQ=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=CtZZYZIS5maQwXSrbg0EVI/Hn33StH7pH09nlCGswxPZH9c1QLhVZsx82A4fTRMls iE5xmlzl1ZXtYaFMcmHNg== X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 restart.be p989KAYn065029 DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to:cc: subject:references:in-reply-to:content-type:content-transfer-encoding; b=Fy6RatC3qFwdky9L7XvZavIALBgVqXb3iCxo9JnhlIvUx/Bgd7F2wcfR2H0MXfjFo jiwBZgMZABvz4VzqrevIg== Message-ID: <4E9015CA.30302@restart.be> Date: Sat, 08 Oct 2011 11:20:10 +0200 From: Henri Hennebert Organization: RestartSoft User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:7.0.1) Gecko/20111006 Thunderbird/7.0.1 MIME-Version: 1.0 To: Andriy Gapon References: <4E8D7406.4090302@restart.be> <4E8D86A2.1040508@FreeBSD.org> <4E8D9F57.70506@restart.be> <4E8DAEE5.4020004@FreeBSD.org> <4E8DB464.80202@restart.be> <4E8DB915.6000502@FreeBSD.org> In-Reply-To: <4E8DB915.6000502@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@FreeBSD.org Subject: Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 09:20:13 -0000 On 10/06/2011 16:20, Andriy Gapon wrote: > on 06/10/2011 17:00 Henri Hennebert said the following: >> On 10/06/2011 15:36, Andriy Gapon wrote: >>> on 06/10/2011 15:30 Henri Hennebert said the following: >>>> The pool is a mirror: >>>> >>>> [root@morzine ~]# zpool status rpool >>>> pool: rpool >>>> state: ONLINE >>>> scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 >>>> config: >>>> >>>> NAME STATE READ WRITE CKSUM >>>> rpool ONLINE 0 0 0 >>>> mirror-0 ONLINE 0 0 0 >>>> gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >>>> gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 >>>> >>>> errors: No known data errors >>>> >>>> and rpool/root is not compressed: >>>> >>>> [root@morzine ~]# zfs get compression rpool/root >>>> NAME PROPERTY VALUE SOURCE >>>> rpool/root compression off inherited from rpool >>>> >>>> pool is v28 and filesystems are v5 >>> >>> No particular recipes for this environment, just a general suggestion. >>> If you run into a situation like this again, please try to use >>> tools/tools/zfsboottest to diagnose where exactly an error originates. >>> >> I try [ please note _M_enu.rc ]: >> >> [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest >> /boot/Menu.rc /dev/da0p2 /dev/da1p2 >> ZFS: SPA version 28 >> pool: rpool >> config: >> >> NAME STATE >> rpool ONLINE >> mirror ONLINE >> gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE >> gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE >> \ Menu.rc >> \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ >> \ >> \ Load required Forth modules >> include /boot/version.4th >> include /boot/brand.4th >> include /boot/menu.4th >> include /boot/menu-commands.4th >> include /boot/shortcuts.4th >> >> \ Screen prep >> clear \ clear the screen (see `screen.4th') >> print_version \ print version string (bottom-right; see `version.4th') >> draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') >> draw-brand \ draw the FreeBSD title (top-left; see `brand.4th') >> menu-init \ initialize the menu area (see `menu.4th') >> >> \ Initialize main menu constructs (see `menu.4th') >> \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) >> >> set menu_timeout_command="boot" >> >> \ Display the main menu (see `menu.4th') >> menu-display >> [root@morzine ~] >> >> The line `ZFS: SPA version 28' >> >> come from my local patch: >> >> Index: sys/boot/zfs/zfsimpl.c >> =================================================================== >> --- sys/boot/zfs/zfsimpl.c (revision 225759) >> +++ sys/boot/zfs/zfsimpl.c (working copy) >> @@ -63,6 +63,8 @@ >> STAILQ_INIT(&zfs_vdevs); >> STAILQ_INIT(&zfs_pools); >> >> + printf("ZFS: SPA version %u\n", (unsigned) SPA_VERSION); >> + >> zfs_temp_buf = malloc(TEMP_SIZE); >> zfs_temp_end = zfs_temp_buf + TEMP_SIZE; >> zfs_temp_ptr = zfs_temp_buf; >> >> >> Is it what you sugest ? > > Yes. And this report indicates that the boot code (built from your source tree) > should be able to read that file. > I do: mv /boot/Menu.rc /boot/menu.rc and reboot. The /boot/menu.rc can be read by zfsloader so I conclude that it was the directory entry of /boot/menu.rc thas has a problem in the first place. Next time it happen I will directly use zfsboottest before any update to the pool. Thank for your time! Henri From owner-freebsd-fs@FreeBSD.ORG Sat Oct 8 21:28:58 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D6BF106567A for ; Sat, 8 Oct 2011 21:28:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 20C598FC18 for ; Sat, 8 Oct 2011 21:28:57 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq8EAIS/kE6DaFvO/2dsb2JhbABDhHWgdYM7gVMBAQEBAwEBASArIAsbDgMDAQIBAgINGQIjBgEJHggGCAcEARwEh2SlTZBSgSyBcoMRgRQEjlcBgwWCGooqh0M X-IronPort-AV: E=Sophos;i="4.68,508,1312171200"; d="scan'208";a="140764943" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 08 Oct 2011 17:28:57 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 0FD29B3F24; Sat, 8 Oct 2011 17:28:57 -0400 (EDT) Date: Sat, 8 Oct 2011 17:28:57 -0400 (EDT) From: Rick Macklem To: Mark Saad Message-ID: <593875119.2799381.1318109337022.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201109291600.p8TG0OI4040954@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@FreeBSD.org Subject: Re: kern/156168: [nfs] [panic] Kernel panic under concurrent access over NFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 21:28:58 -0000 Mark Saad wrote: > The following reply was made to PR kern/156168; it has been noted by > GNATS. > > From: Mark Saad > To: bug-followup@FreeBSD.org, niakrisn@gmail.com > Cc: > Subject: Re: kern/156168: [nfs] [panic] Kernel panic under concurrent > access > over NFS > Date: Thu, 29 Sep 2011 11:32:12 -0400 > > All > I am seeing a similar crash on 7.3-RELEASE-p2 amd64 when using > apache-1.3.34 with accf_httpd and a nfs docroot > The servers that have crashed are all FreeBSD 7.3-RELEASE amd64. > Hardware is HP Dl145 g2 > They have 2G of ram and 2G swap with one single core opteron cpu. > > > We are using the following sysctls . > > kern.ipc.maxsockbuf=2097152 > kern.ipc.nmbclusters=32768 > kern.ipc.somaxconn=1024 > kern.maxfiles=131072 > kern.maxfilesperproc=32768 > net.inet.tcp.inflight.enable=0 > net.inet.tcp.path_mtu_discovery=0 > net.inet.tcp.recvbuf_inc=524288 > net.inet.tcp.recvbuf_max=8388608 > net.inet.tcp.recvspace=32768 > net.inet.tcp.sendbuf_inc=16384 > net.inet.tcp.sendbuf_max=8388608 > net.inet.tcp.sendspace=32768 > net.inet.udp.recvspace=42080 > net.isr.direct=1 > vm.pmap.shpgperproc=600 > > > Up time prior to the crash was not the other system was up for 11 days > this one was 6 days. > > Here is the contents of my crash > > > [root@web29 /var/crash]# kgdb /boot/kernel/kernel /var/crash/vmcore.0 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x258 > fault code = supervisor read data, page not present > instruction pointer = 0x8:0xffffffff8051a66d > stack pointer = 0x10:0xffffff803e69b1c0 > frame pointer = 0x10:0xffffff0001b50ae0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 9336 (libhttpd.ep) > trap number = 12 > panic: page fault > cpuid = 0 > Uptime: 6d5h18m39s > Physical memory: 2034 MB > Dumping 1451 MB: 1436 1420 1404 1388 1372 1356 1340 1324 1308 1292 > 1276 1260 1244 1228 1212 1196 1180 1164 1148 1132 1116 1100 1084 1068 > 1052 1036 1020 1004 988 972 956 940 924 908 892 876 860 844 828 812 > 796 780 764 748 732 716 700 684 668 652 636 620 604 588 572 556 540 > 524 508 492 476 460 444 428 412 396 380 364 348 332 316 300 284 268 > 252 236 220 204 188 172 156 140 124 108 92 76 60 44 28 12 > > Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from > /boot/kernel/accf_http.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/accf_http.ko > #0 doadump () at pcpu.h:195 > 195 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:195 > #1 0x0000000000000004 in ?? () > #2 0xffffffff805285f9 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:418 > #3 0xffffffff80528a02 in panic (fmt=0x104
bounds>) at /usr/src/sys/kern/kern_shutdown.c:574 > #4 0xffffffff807ec813 in trap_fatal (frame=0xffffff0001b50ae0, > eva=Variable "eva" is not available. > ) at /usr/src/sys/amd64/amd64/trap.c:777 > #5 0xffffffff807ecbe5 in trap_pfault (frame=0xffffff803e69b110, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:693 > #6 0xffffffff807ed50c in trap (frame=0xffffff803e69b110) at > /usr/src/sys/amd64/amd64/trap.c:464 > #7 0xffffffff807d614e in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:218 > #8 0xffffffff8051a66d in _mtx_lock_sleep (m=0xffffff002f3d7a80, > tid=18446742974226565856, opts=Variable "opts" is not available. > ) > at /usr/src/sys/kern/kern_mutex.c:339 > #9 0xffffffff80701f60 in clnt_dg_create (so=0xffffff00017755a0, > svcaddr=0xffffff803e69b310, program=100000, version=4, sendsz=Variable > "sendsz" is not available. > ) > at /usr/src/sys/rpc/clnt_dg.c:259 > #10 0xffffffff806e97c9 in nlm_get_rpc (sa=Variable "sa" is not > available. > ) at /usr/src/sys/nlm/nlm_prot_impl.c:327 > #11 0xffffffff806e9d39 in nlm_host_get_rpc (host=0xffffff0001705000) > at /usr/src/sys/nlm/nlm_prot_impl.c:1199 > #12 0xffffffff806e680f in nlm_clearlock (host=0xffffff0001705000, > ext=0xffffff803e69b9a0, vers=4, timo=0xffffff803e69b9d0, > retries=2147483647, vp=0xffffff004881edc8, op=2, > fl=0xffffff803e69bac0, flags=64, svid=9336, fhlen=32, > fh=0xffffff803e69b750, > size=689) at /usr/src/sys/nlm/nlm_advlock.c:943 > #13 0xffffffff806e7801 in nlm_advlock_internal (vp=0xffffff004881edc8, > id=Variable "id" is not available. > ) at /usr/src/sys/nlm/nlm_advlock.c:355 > #14 0xffffffff806e8166 in nlm_advlock (ap=Variable "ap" is not > available. > ) at /usr/src/sys/nlm/nlm_advlock.c:392 > #15 0xffffffff806ced28 in nfs_advlock (ap=0xffffff803e69ba90) at > /usr/src/sys/nfsclient/nfs_vnops.c:3153 > #16 0xffffffff804f40e2 in closef (fp=0xffffff0073716d80, > td=0xffffff0001b50ae0) at vnode_if.h:1036 > #17 0xffffffff804f462b in kern_close (td=0xffffff0001b50ae0, > fd=Variable "fd" is not available. > ) at /usr/src/sys/kern/kern_descrip.c:1125 > #18 0xffffffff807ece67 in syscall (frame=0xffffff803e69bc80) at > /usr/src/sys/amd64/amd64/trap.c:920 > #19 0xffffffff807d635b in Xfast_syscall () at > /usr/src/sys/amd64/amd64/exception.S:339 > #20 0x00000008009c5b1c in ?? () > Previous frame inner to this frame (corrupt stack?) > I believe your crash is fixed in 8/stable or later systems. Unfortunately, it is not a trivial patch to backport to 7.n. It's r193437, which isn't too hard to put into the old code. The hard part is I'm not sure if you also need r193272, which could be really ugly to backport. I don't even have a 7.n system to test with at this point, so I can't try the backport, but you can try putting r193437 in 7.n if you feel up to it and see how it works. Btw, I don't think this is the same crash as reported in by kern/156168. I don't have any insight w.r.t. a fix for that one at this time. rick > -- > mark saad | nonesuch@longcount.org > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"