From owner-freebsd-fs@FreeBSD.ORG Wed Aug 11 21:43:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B308106566C for ; Wed, 11 Aug 2010 21:43:06 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id 92AB78FC0C for ; Wed, 11 Aug 2010 21:43:05 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id o7BLh2C0046753 for ; Wed, 11 Aug 2010 21:43:02 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.1 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id o7BLh27S046752 for freebsd-fs@freebsd.org; Wed, 11 Aug 2010 21:43:02 GMT (envelope-from marco) Date: Wed, 11 Aug 2010 21:43:02 +0000 From: Marco van Tol To: freebsd-fs@freebsd.org Message-ID: <20100811214302.GB44635@tolstoy.tols.org> Mail-Followup-To: freebsd-fs@freebsd.org References: <20100810214418.GA28288@tolstoy.tols.org> <20100811014919.GA52992@icarus.home.lan> <20100811192537.GA44635@tolstoy.tols.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: zfs arc - just take it all and be good to me X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2010 21:43:06 -0000 On Wed, Aug 11, 2010 at 01:08:51PM -0700, Artem Belevich wrote: > > The bottom line of my original mail was: > > - Hey, I set the arc to just take all my RAM > > - It grows on reads and writes > > - It shrinks when programs need the memory it has. > > - It behaves well in a scenario of combined read/write/other programs. > > - This is good, really good, what is dangerous about my settings that > >  not everybody just sets the arc to nearly all their physical memory on > >  FreeBSD? > > The issue is that ARC will give up memory even if there's plenty of it > available and sitting in inactive/cache queues. That's particularly > nasty in case your system uses some other filesystem besides ZFS. For > example try tarring up few gigabytes worth of data from UFS filesystem > and see how far your ARC size shrinks. It could be mitigated by > setting minimum ARC size to be large enough so ZFS performance does > not degrade. The downside is that ARC will not give up memory below > its minimum no matter what, so if your APP really needs it, it would > have to go to swap. > > There's a hack floating around that attempts to force kernel into > freeing up memory from inactive/cache lists before draining ARC. It > does help a bit with this issue, but it's still a hack. That makes sense Artem, thanks. I think you mean the posts with the perl one-liner I used in my tests as well. (perl variable assignment of 1.5GB in the posts their version) I had seen the posts that mentioned that one and decided to remember the perl hack. :) What I understand from it: - In a UFS/ZFS mixed system - In a scenario where UFS "page cache" took (almost) all available memory - Run a perl one-liner to throw out the UFS active/inactive usage - Kind-off hope you do enough relevant ZFS accesses that you get a good new situation. So, if my worries can shift from fighting with kmem_size and arc_max to fighting with arc_min, that's a fight I like a lot better. Especially on zfs-only systems, I have to admit. Thanks a lot! Marco -- It's fried rice, you plick. -- Lethal Weapon 4