From owner-freebsd-fs@FreeBSD.ORG Sun Sep 5 23:57:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 670431065697 for ; Sun, 5 Sep 2010 23:57:51 +0000 (UTC) (envelope-from prvs=18641c54ab=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id EBD9B8FC17 for ; Sun, 5 Sep 2010 23:57:50 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Mon, 06 Sep 2010 00:57:45 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 06 Sep 2010 00:57:45 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 by mail1.multiplay.co.uk (MDaemon PRO v10.0.4) with ESMTP id md50011185136.msg for ; Mon, 06 Sep 2010 00:57:45 +0100 X-Authenticated-Sender: Killing@multiplay.co.uk X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=18641c54ab=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org Message-ID: <330B5DB2215F43899ABAEC2CF71C2EE0@multiplay.co.uk> From: "Steven Hartland" To: "jhell" References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C825D65.3040004@DataIX.net> <7EA7AD058C0143B2BF2471CC121C1687@multiplay.co.uk> <1F64110BFBD5468B8B26879A9D8C94EF@multiplay.co.uk> <4C83A214.1080204@DataIX.net> <06B9D23F202D4DB88D69B7C4507986B7@multiplay.co.uk> <4C842905.2080602@DataIX.net> Date: Mon, 6 Sep 2010 00:57:46 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5931 Cc: freebsd-fs@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Sep 2010 23:57:51 -0000 ----- Original Message ----- From: "jhell" > On 09/05/2010 16:13, Steven Hartland wrote: >>> 3656: uint64_t available_memory = ptoa((uintmax_t)cnt.v_free_count >>> 3657: + cnt.v_cache_count); > >> earlier at 3614 I have what I think your after which is: >> uint64_t available_memory = ptoa((uintmax_t)cnt.v_free_count); > > Alright change this to the above, recompile and re-run your tests. > Effectively before this change that apparently still needs to be MFC'd > or MFS'd would not allow ZFS to look at or use cnt.v_cache_count. Pretty > much to sum it up "available mem = cache + free" > > This possibly could cause what your seeing but there might be other > changes still yet TBD. Ill look into what else has changed from RELEASE > -> STABLE. > > Also do you check out your sources with svn(1) or csup(1) ? Based on Jeremy's comments I'm updating the box the stable. Its building now but will be the morning before I can reboot to activate changes as I need to deactivate the stream instance and wait for all active connections to finish. That said the problem doesn't seem to be cache + free but more cache + free + inactive with inactive being the large chunk, so not sure this change would make any difference? How does ufs deal with this, does it take inactive into account? Seems a bit silly for inactive pages to prevent reuse for extended periods when the memory could be better used as cache. As an experiment I compiled a little app which malloced a large block of memory, 1.3G in this case and then freed it. This does indeed pull the memory out of inactive and back into the free pool where zfs is which happy to re-expand arc and once again cache large files. Seems a bit extreme to have to do this though. Will see what happens with stable tomorrow though :) Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.