From owner-freebsd-stable@FreeBSD.ORG Tue Mar 11 22:32:18 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48D983B5 for ; Tue, 11 Mar 2014 22:32:18 +0000 (UTC) Received: from aussmtpmrkpc120.us.dell.com (aussmtpmrkpc120.us.dell.com [143.166.82.159]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB6E4C1C for ; Tue, 11 Mar 2014 22:32:16 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.97,633,1389765600"; d="scan'208";a="77984515" Message-ID: <531F8EAA.1020107@vangyzen.net> Date: Tue, 11 Mar 2014 17:31:06 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Danny Schales , Subject: Re: ZFS UNMAP performance References: <531F2BA0.6000105@LaTech.edu> <531F3503.8090403@LaTech.edu> <531F767C.3040105@LaTech.edu> In-Reply-To: <531F767C.3040105@LaTech.edu> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 22:32:18 -0000 On 03/11/2014 15:47, Danny Schales wrote: > On 03/11/2014 11:08, Danny Schales wrote: >> On 03/11/2014 10:48, Tom Evans wrote: >>> On Tue, Mar 11, 2014 at 3:28 PM, Danny Schales wrote: >>>> I'm seeing very slow performance with certain operations on a ZFS >>>> filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI >>>> implementation). The issue appears to be with BIO_DELETE operations. >>>> Monitoring the system with gstat shows expected times for read and write >>>> operations, but deletes are in the multiple hundreds of milliseconds >>>> under normal operation. Destroying a snapshot sends the times to >>>> astronomical levels. sysctl says the system is using UNMAP for deletes: >>>> >>>> kern.cam.da.0.delete_method: UNMAP >>>> >>>> I searched and found where Oracle issued a performance alert for Solaris >>>> 11.1 where ZFS using UNMAP was in use. Here's a link to a blog >>>> discussing it: >>>> >>>> http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html >>>> >>>> >>>> Is FreeBSD also impacted? If so, is there a fix or a workaround? >> The backend is an ISCSI LUN from a SAN (Dell Compellent). From >> research, it seems the SAN *does* support SCSI UNMAP requests for >> deletes, but the performance is horrible. I don't know if this is a >> FreeBSD issue or a Compellent issue. I haven't seen the problem with >> other devices, but I don't think anything else is using UNMAP (yet). >> >> Danny >> >> > Replying to myself...I note that the system is reporting that TRIM is > being used. Is this normal for non-SSD systems? There *is* SSD in the > system, but I'm pretty sure the system can't tell it's SSD (it's hidden > behind a Dell PERC card). The number of trim.successes is roughly > equivalent to the number of deletes reported by gstat for the ISCSI LUN > devices. Should the system be using TRIM for ISCSI LUNs? Sure, if the LUN (i.e. the storage controller) reports that it supports TRIM/UNMAP. Note that this is completely unrelated to the type of disks that provide the LUN's backing store. > kstat.zfs.misc.zio_trim.bytes: 232845656064 > kstat.zfs.misc.zio_trim.success: 30810983 > kstat.zfs.misc.zio_trim.unsupported: 809 > kstat.zfs.misc.zio_trim.failed: 0 > > Danny >