From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 21:51:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F032F106564A; Mon, 3 Jan 2011 21:51:33 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id 888528FC0A; Mon, 3 Jan 2011 21:51:33 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.13.8+Sun/8.13.8) with ESMTP id p03LZLt0019766; Mon, 3 Jan 2011 15:35:22 -0600 (CST) Date: Mon, 3 Jan 2011 15:35:21 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Attila Nagy In-Reply-To: <4D222B7B.1090902@fsn.hu> Message-ID: References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> <4D222B7B.1090902@fsn.hu> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Mon, 03 Jan 2011 15:35:22 -0600 (CST) Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Artem Belevich Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 21:51:34 -0000 >> > After four days, the L2 hit rate is still hovering around 10-20 percents (was > between 60-90), so I think it's clearly a regression in the ZFSv28 patch... > And the massive growth in CPU usage can also very nicely be seen... > > I've updated the graphs at (switch time can be checked on the zfs-mem graph): > http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ > > There is a new phenomenom: the large IOPS peaks. I use this munin script on a > lot of machines and never seen anything like this... I'm not sure whether > it's related or not. It is not so clear that there is a problem. I am not sure what you are using this server for but it is wise to consider that this is the funny time when a new year starts, SPAM delivery goes through the roof, and employees and customers behave differently. You chose the worst time of the year to implement the change and observe behavior. CPU use is indeed increased somewhat. A lower loading of the l2arc is not necessarily a problem. The l2arc is usually bandwidth limited compared with main store so if bulk data can not be cached in RAM, then it is best left in main store. A smarter l2arc algorithm could put only the data producing the expensive IOPS (the ones requiring a seek) in the l2arc, lessening the amount of data cached on the device. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/