From owner-freebsd-stable@FreeBSD.ORG Mon Nov 19 13:26:02 2012 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 883403FB for ; Mon, 19 Nov 2012 13:26:02 +0000 (UTC) (envelope-from peter@pean.org) Received: from lyra.its.uu.se (lyra.its.uu.se [130.238.7.73]) by mx1.freebsd.org (Postfix) with ESMTP id 157938FC08 for ; Mon, 19 Nov 2012 13:26:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at uu.se Received: from nyx.uppmax.uu.se (nyx.uppmax.uu.se [130.238.137.40]) by lyra.its.uu.se (Postfix) with ESMTP id BAFA7393B1 for ; Mon, 19 Nov 2012 14:17:01 +0100 (CET) Message-ID: <50AA314E.2090806@pean.org> Date: Mon, 19 Nov 2012 14:17:02 +0100 From: =?ISO-8859-1?Q?Peter_Ankerst=E5l?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.10) Gecko/20121030 Thunderbird/10.0.10 MIME-Version: 1.0 To: freebsd-stable@FreeBSD.org Subject: Poor iSCSI performance with istgt. Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Nov 2012 13:26:02 -0000 Hi! Im using two FreeBSD machines trying to get iSCSI to work and it works allright, but extremley slow. It peaks at about 50MB/s if I have multiple dd's runningt against the target. I have two separate networks but I get no difference in speed whichever I use. One regular 1G network connected to the same switch with Intel cards. I get close to maximum performance with both ftp and iperf. The other network is a IPoverIB network where I get 560MB/s with iperf and 250MB/s with ftp.. (hmm). Both machines run FreeBSD 9.1-PRE with OFED enabled: FreeBSD zfs1 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #0: Tue Oct 16 16:07:39 CEST 2012 peter@zfs1:/usr/obj/usr/src/sys/OFED amd64 Im only fiddeling with writes right now and like I said I get about 50MB/s with multiple dds running, around 30 with a single thread. (dd if=/dev/zero of=/dev/daX bs=1M count=n) Writing to local disk at the target gives me like 300MB/s I have also been experimenting with different burst lengths, queue depths and so no with no noticeable difference. I have also tried iozone and dd on both ZFS and UFS, similar results, 40MB/s and peaks at 50MB/s. So I figure it has to have something to do with my iscsi setup. istgt.conf: ------------------------------------------------ [Global] Comment "Global section" NodeBase "dc3" PidFile /var/run/istgt.pid AuthFile /usr/local/etc/istgt/auth.conf LogFacility "local7" Timeout 30 NopInInterval 20 DiscoveryAuthGroup AuthGroup9999 MaxSessions 16 MaxConnections 4 FirstBurstLength 262144 MaxBurstLength 1048576 MaxRecvDataSegmentLength 262144 [UnitControl] Comment "Internal Logical Unit Controller" AuthMethod CHAP Mutual AuthGroup AuthGroup10000 Portal UC1 127.0.0.1:3261 Netmask 127.0.0.1 [PortalGroup1] Comment "T1 portal" Portal DA1 ip:3260 [InitiatorGroup1] Comment "V1 group" InitiatorName "dc1" Netmask ip/24 [LogicalUnit1] TargetName V1 Mapping PortalGroup1 InitiatorGroup1 AuthGroup AuthGroup1 UseDigest Auto UnitType Disk QueueDepth 64 LUN0 Storage /dev/zvol/store/v1 Auto ------------------------------------------------ iscsi.conf ------------------------------------------------ dc3 { authmethod = CHAP chapIName = id chapSecret = pass Initiatorname = dc1 TargetName = dc3:V1 TargetAddress = ip:3260,1 MaxConnections = 4 } ------------------------------------------------