From owner-freebsd-current@FreeBSD.ORG Tue Feb 11 14:38:26 2014 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0A995DB; Tue, 11 Feb 2014 14:38:26 +0000 (UTC) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7A84518C4; Tue, 11 Feb 2014 14:38:26 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1WDETf-000BQT-VI ; Tue, 11 Feb 2014 16:38:23 +0200 Date: Tue, 11 Feb 2014 16:38:23 +0200 From: Vitalij Satanivskij To: Vitalij Satanivskij Subject: Re: ARC "pressured out", how to control/stabilize ? (reformatted to text/plain) Message-ID: <20140211143823.GB59461@hell.ukr.net> References: <20140131182637.GA82526@hell.ukr.net> <20140204100823.GA95709@hell.ukr.net> <52F0F687.6050307@FreeBSD.org> <20140204171040.GA82996@hell.ukr.net> <52F12210.10604@FreeBSD.org> <20140205090449.GA9341@hell.ukr.net> <20140205122241.GA38114@hell.ukr.net> <52F4A35E.1080902@FreeBSD.org> <52FA2CA2.5000508@FreeBSD.org> <20140211140746.GA59461@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140211140746.GA59461@hell.ukr.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Vladimir Sharun , Current FreeBSD , Andriy Gapon X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 14:38:26 -0000 Get first result's while testing l2 without compression Memory leak is not seen for now ( system working only 20 hours) but zfs stats saying that l2 degraded output of zfs-stats -L: ------------------------------------------------------------------------ ZFS Subsystem Report Tue Feb 11 16:34:43 2014 ------------------------------------------------------------------------ L2 ARC Summary: (DEGRADED) Passed Headroom: 3.81m Tried Lock Failures: 79.52m IO In Progress: 9 Low Memory Aborts: 235 Free on Write: 54.37k Writes While Full: 9.68k R/W Clashes: 2.82k Bad Checksums: 211.94k IO Errors: 0 SPA Mismatch: 58.33m L2 ARC Size: (Adaptive) 243.32 GiB Header Size: 0.36% 895.11 MiB L2 ARC Evicts: Lock Retries: 45 Upon Reading: 0 L2 ARC Breakdown: 38.15m Hit Ratio: 17.79% 6.79m Miss Ratio: 82.21% 31.36m Feeds: 88.88k L2 ARC Buffer: Bytes Scanned: 292.58 TiB Buffer Iterations: 88.88k List Iterations: 5.63m NULL List Iterations: 17.26k L2 ARC Writes: Writes Sent: (FAULTED) 77.95k Done Ratio: 100.00% 77.95k Error Ratio: 0.00% 0 ------------------------------------------------------------------------ As you can see we have Bad Checksums: 211.94k and growing and also Writes Sent: (FAULTED) 77.95k Done Ratio: 100.00% 77.95k Another question: Please provide revision number of arc.c against which was diff created (http://people.freebsd.org/~avg/l2arc-b_tmp_cdata-diag.2.patch) Because in version in head have some small diferent's and I need manualy aply patch. Thank you. Vitalij Satanivskij wrote: VS> Dear Andriy and FreeBSD community, VS> VS> VS> For now I begin testing l2 cache without compression (with you path provided in last messages) in production. VS> VS> I will test the new patch on the test server first, and then if all is ok on one of the production servers. VS> VS> VS> Andriy Gapon wrote: VS> AG> on 07/02/2014 11:11 Andriy Gapon said the following: VS> AG> > on 05/02/2014 14:22 Vitalij Satanivskij said the following: VS> AG> >> Dear Andriy and FreeBSD community, VS> AG> >> VS> AG> >> Ok. I'm get coredump on panic. VS> AG> >> VS> AG> >> What else i need to do? VS> AG> > VS> AG> > VS> AG> > Vitalij, Vladimir, VS> AG> > VS> AG> > I have been able to reproduce the leak at work, so now I have full access to all VS> AG> > debugging information that I need. Thank you for your testing and reports. VS> AG> > VS> AG> > I have reported my observations to OpenZFS developers. It looks like the author VS> AG> > of L2ARC compression code is too busy right now to produce a fix. VS> AG> > Unfortunately, I am not very familiar with the L2ARC code, so I can not promise VS> AG> > to produce a patch soon. VS> AG> VS> AG> I've been able to spend some time on this issue. VS> AG> Could you please try the following patch? VS> AG> http://people.freebsd.org/~avg/l2arc-b_tmp_cdata-diag.2.patch VS> AG> It obsoletes all previous patches from me. VS> AG> VS> AG> -- VS> AG> Andriy Gapon VS> AG> _______________________________________________ VS> AG> freebsd-current@freebsd.org mailing list VS> AG> http://lists.freebsd.org/mailman/listinfo/freebsd-current VS> AG> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" VS> _______________________________________________ VS> freebsd-current@freebsd.org mailing list VS> http://lists.freebsd.org/mailman/listinfo/freebsd-current VS> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"