From owner-freebsd-questions@FreeBSD.ORG Thu Apr 21 13:53:34 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36D3D1065677 for ; Thu, 21 Apr 2011 13:53:34 +0000 (UTC) (envelope-from freebsd-questions@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id D12078FC23 for ; Thu, 21 Apr 2011 13:53:33 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QCuK0-0002WZ-E1 for freebsd-questions@freebsd.org; Thu, 21 Apr 2011 15:53:28 +0200 Received: from 85.183.14.177 ([85.183.14.177]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Apr 2011 15:53:28 +0200 Received: from lw by 85.183.14.177 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Apr 2011 15:53:28 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-questions@freebsd.org From: Lars Wilke Date: Thu, 21 Apr 2011 15:51:42 +0200 Lines: 45 Message-ID: References: <4DA710AA.2030501@my.gd> X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 85.183.14.177 User-Agent: slrn/pre1.0.0-18 (Linux) Subject: Re: ZFS performance strangeness X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Apr 2011 13:53:34 -0000 Hi, thanks to all who answered. * Damien Fleuriot wrote: > I refer you to this post by Jeremy Chadwick with tuning values *AND* > their actual explanation. > > http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061642.html Good post indeed, but i think i found my problem elsewhere, though i have to admit that i don't really understand what is happening. The Citrix XenServers 5.6 boxes are connected with 2x1GB, using bonding with balance-slb, to an HP ProCurve 2910al-24G Switch. It seems this balance-slb mode is patch from Citrix and i could not come up with some documentation for it. Only thing i know it is a modified form of balance-alb. The FreeBSD box is also connected to this switch with two NICs, one em and one igb driven NIC. If i do not use lagg and assign just a single address to one of the NICs on FBSD, everything works well. I do only set vm.kmem_size_scale and vfs.zfs.arc_max and get r/w speed of a little bit above 100MB/s when doing: dd bs=1M count=4096 ... and remounting before reading or writting. But if i do use lagg in loadbalance or in lacp mode (here also enabled lacp on the necessary switch ports) i see via a tcpdump, that ip packets seem to get lost!? On top of that this only happens when reading data via NFS, writting is fine. Then again lagg in failover mode seems to work ok. The NICs themself look fine and are automatically set to 1000baseT full duplex. There are also no errors reported via netstat. I am not sure if the switch does not like playing with the arp addresses on its ports or if it has something to do with the slb implementation from citrix. When i have a downtime for the xenserver i will reconfigure it to use lacp, then lets see if that works. cheers --lars