From owner-freebsd-scsi@FreeBSD.ORG Mon Nov 25 11:06:56 2013 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E3215CE for ; Mon, 25 Nov 2013 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85F7C2F58 for ; Mon, 25 Nov 2013 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rAPB6use089995 for ; Mon, 25 Nov 2013 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rAPB6u92089993 for freebsd-scsi@FreeBSD.org; Mon, 25 Nov 2013 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 25 Nov 2013 11:06:56 GMT Message-Id: <201311251106.rAPB6u92089993@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2013 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/184059 scsi [mps] mps SCSI driver causes FreeBSD to hang during bo o kern/179932 scsi [ciss] ciss i/o stall problem with HP Bl Gen8 (and HP o kern/178795 scsi [mps] MSI for mps driver doesn't work under vmware o kern/165982 scsi [mpt] mpt instability, drive resets, and losses on Fre o kern/165740 scsi [cam] SCSI code must drain callbacks before free f kern/162256 scsi [mpt] QUEUE FULL EVENT and 'mpt_cam_event: 0x0' o docs/151336 scsi Missing documentation of scsi_ and ata_ functions in c o kern/148083 scsi [aac] Strange device reporting o kern/144648 scsi [aac] Strange values of speed and bus width in dmesg o kern/142351 scsi [mpt] LSILogic driver performance problems o kern/134488 scsi [mpt] MPT SCSI driver probes max. 8 LUNs per device o kern/130621 scsi [mpt] tranfer rate is inscrutable slow when use lsi213 f kern/129602 scsi [ahd] ahd(4) gets confused and wedges SCSI bus f kern/123674 scsi [ahc] ahc driver dumping o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc 15 problems total. From owner-freebsd-scsi@FreeBSD.ORG Mon Nov 25 16:50:02 2013 Return-Path: Delivered-To: freebsd-scsi@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0B1D8FA for ; Mon, 25 Nov 2013 16:50:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BFD3E2793 for ; Mon, 25 Nov 2013 16:50:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rAPGo2ZX066881 for ; Mon, 25 Nov 2013 16:50:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rAPGo2og066880; Mon, 25 Nov 2013 16:50:02 GMT (envelope-from gnats) Date: Mon, 25 Nov 2013 16:50:02 GMT Message-Id: <201311251650.rAPGo2og066880@freefall.freebsd.org> To: freebsd-scsi@FreeBSD.org Cc: From: Sean Bruno Subject: Re: kern/179932: [ciss] ciss i/o stall problem with HP Bl Gen8 (and HP Bl Gen7 + Storage Blade) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list Reply-To: Sean Bruno List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2013 16:50:02 -0000 The following reply was made to PR kern/179932; it has been noted by GNATS. From: Sean Bruno To: bug-followup@FreeBSD.org, philipp.maechler@hostpoint.ch Cc: Subject: Re: kern/179932: [ciss] ciss i/o stall problem with HP Bl Gen8 (and HP Bl Gen7 + Storage Blade) Date: Mon, 25 Nov 2013 08:45:07 -0800 --=-pQztALPEYNid4Hq+my6z Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I have a test box for this now with an external array. Currently, its configured with an onboard p420i with a single raid0 of one disk and a P222 handling a 40TB array external to the chassis. I'll see if I can do something to make this die in my Gen8 box. sean --=-pQztALPEYNid4Hq+my6z Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSk36LAAoJEBkJRdwI6BaHXGsIAJIZ9KfADM6d3zs8ndaRL5Wp 6tQjCxVKehzX1CLgEvmG4GRq7FRUXMskvs7G+/EQMJFS9egOcz+CcXm5TocZp+Qu 21vt/7XYk8VRkCsb5I5GA+Jlbo1k8MiFI7waBPFeWzS3VfciEQ/qGk1H1m3IE1ax ZNVSmClQFGDsw9dlZSGEgrU6h/yoJM254dvIezbDC9opJ7cdUi9GDPwc00+38Y1o y8xOQYYXgREnDp1pX5SYjSeyWIjUJxOT/nSlMOvMGb79DBEqlf/dO7fxbn5fmyBV b37gX+xBa8uz+WJSzL8IUxdoMblRMTj2MATQkpHZ2xmfw9+HagBrb3re+BwD8qY= =TQRM -----END PGP SIGNATURE----- --=-pQztALPEYNid4Hq+my6z-- From owner-freebsd-scsi@FreeBSD.ORG Wed Nov 27 09:42:33 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6CFAE50 for ; Wed, 27 Nov 2013 09:42:33 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 429ED2400 for ; Wed, 27 Nov 2013 09:42:33 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 776A910A7 for ; Wed, 27 Nov 2013 10:42:32 +0100 (CET) MIME-Version: 1.0 Date: Wed, 27 Nov 2013 10:42:29 +0100 From: krichy@cflinux.hu To: freebsd-scsi@freebsd.org Subject: Fwd: ssd for zfs Message-ID: X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.16 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 09:42:34 -0000 Dear SCSI dev team, I bought an SSD for my ZFS filesystem to use it as a ZIL. I've tested it under linux, and found that it can handle around 1400 random synchronized write IOPS. Then I placed it into my freebsd 9.2 box, and after attaching it as a ZIL, my zpool only performs 100 (!) write iops. I've attached it to an AHCI controller and to an LSI 1068 controller, on both it behaves the same. So I expect that something in the scsi layer is different, FreeBSD is handling this device slower, but actually it can handle the 1400 iops as tested under linux. I've attached the simple script I used to do the benchmark. basically, on linux and bsd also I've added the SSD as a LOG device to an existing pool, and run the test on a filesystem in that pool. Please give some advice where to go, how to debug, and how to improve FreeBSD's performance with this drive. The device is: # camcontrol identify ada3 pass4: ATA-8 SATA 2.x device pass4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes) protocol ATA/ATAPI-8 SATA 2.x device model STEC MACH16 M16SD2S-50UI firmware revision 00000299 serial number STM0001680E8 WWN 5000a7203006f8e5 media serial number STEC MACH16 M16SD2S-50UI STM00 cylinders 16383 heads 15 sectors/track 63 sector size logical 512, physical 512, offset 0 LBA supported 97696368 sectors LBA48 supported 97696368 sectors PIO supported PIO4 DMA supported WDMA2 UDMA6 media RPM non-rotating Feature Support Enabled Value Vendor read ahead yes yes write cache yes yes flush cache yes yes overlap no Tagged Command Queuing (TCQ) no no Native Command Queuing (NCQ) yes 32 tags SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management no no automatic acoustic management no no media status notification no no power-up in Standby yes no write-read-verify no no unload no yes free-fall no no Data Set Management (DSM/TRIM) yes DSM - max 512byte blocks yes 8 DSM - deterministic read yes any value Host Protected Area (HPA) yes no 97696368/97696368 HPA - Security no Regards, Kojedzinszky Richard From owner-freebsd-scsi@FreeBSD.ORG Wed Nov 27 14:14:41 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 406321C1 for ; Wed, 27 Nov 2013 14:14:41 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 0736622D3 for ; Wed, 27 Nov 2013 14:14:41 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 3E66F10E7 for ; Wed, 27 Nov 2013 15:14:40 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 27 Nov 2013 15:14:38 +0100 From: krichy@cflinux.hu To: freebsd-scsi@freebsd.org Subject: Fwd: Re: ssd for zfs Message-ID: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 14:14:41 -0000 -------- Eredeti üzenet -------- Tárgy: Re: ssd for zfs Dátum: 2013-11-27 14:07 Feladó: Richard Kojedzinszky Címzett: Tom Evans Másolat: FreeBSD FS Dear FS devs, After some investigation, it turned out that when I turn write-cache off under linux, the performance drops to 100 on that OS also. But when enabled, 1400 IOPS (synchronous) can be achieved. So I would like to see the same on FreeBSD as well. Using camcontrol shows that the write cache is enabled, but I may assume that something around this is causing the performance degradation. But unfortunately I cannot step forward right now. Regards, Kojedzinszky Richard On Wed, 27 Nov 2013, Tom Evans wrote: > On Wed, Nov 27, 2013 at 8:51 AM, Richard Kojedzinszky > wrote: >> Dear fs developers, >> >> Probably this is not the best list to report my issue, but please >> forward it >> to where it should get. >> >> I bought an SSD for my ZFS filesystem to use it as a ZIL. I've tested >> it >> under linux, and found that it can handle around 1400 random >> synchronized >> write IOPS. Then I placed it into my freebsd 9.2 box, and after >> attaching it >> as a ZIL, my zpool only performs 100 (!) write iops. I've attached it >> to an >> AHCI controller and to an LSI 1068 controller, on both it behaves the >> same. >> So I expect that something in the scsi layer is different, FreeBSD is >> handling this device slower, but actually it can handle the 1400 iops >> as >> tested under linux. >> >> Please give some advice where to go, how to debug, and how to improve >> FreeBSD's performance with this drive. >> > > The ZIL is only used for synchronous writes. The majority of writes > are asynchronous, and the ZIL is not used at all. Plus, a ZIL can only > increase iops by bundling writes - if your underlying pool is write > saturated already, then a ZIL can't help - any data written to the ZIL > has to end up on the pool. > > Test the SSD by itself under FreeBSD to rule out FreeBSD not working > correctly on the SSD (I doubt this though). > > Cheers > > Tom > From owner-freebsd-scsi@FreeBSD.ORG Wed Nov 27 21:35:31 2013 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE5FFF9D for ; Wed, 27 Nov 2013 21:35:31 +0000 (UTC) Received: from kolbaba.stable.cz (zapmail.stable.cz [88.86.120.103]) by mx1.freebsd.org (Postfix) with ESMTP id 832AA2C47 for ; Wed, 27 Nov 2013 21:31:44 +0000 (UTC) Received: from localhost (kolbaba.miton.cz [127.0.0.1]) by kolbaba.stable.cz (Postfix) with ESMTP id C846D202A4 for ; Wed, 27 Nov 2013 22:31:09 +0100 (CET) X-Virus-Scanned: amavisd-new at kolbaba.stable.cz Received: from kolbaba.stable.cz ([127.0.0.1]) by localhost (kolbaba.stable.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OHyRbIzM8jlD for ; Wed, 27 Nov 2013 22:31:09 +0100 (CET) Received: from [172.27.2.129] (11.25.broadband6.iol.cz [88.101.25.11]) by kolbaba.stable.cz (Postfix) with ESMTPSA id 9DAD82029B for ; Wed, 27 Nov 2013 22:31:09 +0100 (CET) Message-ID: <52966449.8080102@ITSMexpert.EU> Date: Wed, 27 Nov 2013 22:29:45 +0100 From: info@ITSMexpert.EU User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: freebsd-scsi@FreeBSD.org Subject: kern/184154: [cam] QUIRK: SYNC_CACHE not supported on IBM ServeRAID 8k X-Priority: 2 (High) References: <5295153F.2010508@ITSMexpert.EU> In-Reply-To: <5295153F.2010508@ITSMexpert.EU> X-Forwarded-Message-Id: <5295153F.2010508@ITSMexpert.EU> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 21:35:31 -0000 Dear developers, last week I submitted PR kern/184154 containing request for adding a QUIRK to scsi_da driver. The problem is well described, fix found and tested, so that it should be simple to implement it. The severity is "Serious" from my perspective. It is still assigned to freebsd-bugs. What is the procedure? Is someone reviewing the PR? How long it can last? It would be nice to see the patch in upcoming FreeBSD 10.0 . Thanks for your help, Petr Cibulka On 21/11/2013 23:20,FreeBSD-gnats-submit@FreeBSD.org wrote: > Thank you very much for your problem report. > It has the internal identification `kern/184154'. > The individual assigned to look at your > report is: freebsd-bugs. > > You can access the state of your problem report at any time > via this link: > >http://www.freebsd.org/cgi/query-pr.cgi?pr=184154 > >> Category: kern >> Responsible: freebsd-bugs >> Synopsis: [cam] QUIRK: SYNC_CACHE not supported on IBM ServeRAID 8k >> Arrival-Date: Thu Nov 21 22:20:00 UTC 2013 From owner-freebsd-scsi@FreeBSD.ORG Thu Nov 28 01:02:39 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E927AD9C for ; Thu, 28 Nov 2013 01:02:38 +0000 (UTC) Received: from mail3.transactionware.com (mail3.transactionware.com [202.68.173.211]) by mx1.freebsd.org (Postfix) with SMTP id 509B0A17 for ; Thu, 28 Nov 2013 01:02:36 +0000 (UTC) Received: (qmail 16359 invoked by uid 907); 28 Nov 2013 00:55:53 -0000 Received: from eth222.nsw.adsl.internode.on.net (HELO [192.168.1.32]) (150.101.196.221) (smtp-auth username janm, mechanism plain) by mail3.transactionware.com (qpsmtpd/0.84) with (AES128-SHA encrypted) ESMTPSA; Thu, 28 Nov 2013 11:55:53 +1100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1812\)) Subject: Re: ssd for zfs From: Jan Mikkelsen In-Reply-To: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> Date: Thu, 28 Nov 2013 11:55:51 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> To: krichy@cflinux.hu X-Mailer: Apple Mail (2.1812) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2013 01:02:39 -0000 Hi, Using the drive write cache seems like a really bad idea for a ZIL. The purpose of a ZIL is to keep a log of actions for recovery after a = system crash. The drive write cache lets the drive lie to the operating = system about whether or not a write has been made durable. If you have a = power failure while you have 1400 writes outstanding to the drive you = might find that you have data loss on restart. For a ZIL you are best off with a drive that has a supercapacitor to = ensure all outstanding writes can be completed on power loss. For = example, the Intel S3700 series. Performance on your zpool is probably limited by the number of vdevs you = have. More vdevs give more I/O parallelism. If you only have one vdev = you will be limited to single drive throughput. Depending on the number = of drives you have and what you need, you will either need a bunch of = mirrored vdevs or a bunch of raidz2 vdevs (if you have enough drives). I made this mistake early on, thinking a raidz2 vdev alone would give = parallelism. You need multiple vdevs. Regards, Jan Mikkelsen janm@transactionware.com On 28 Nov 2013, at 1:14 am, krichy@cflinux.hu wrote: >=20 >=20 > -------- Eredeti =FCzenet -------- > T=E1rgy: Re: ssd for zfs > D=E1tum: 2013-11-27 14:07 > Felad=F3: Richard Kojedzinszky > C=EDmzett: Tom Evans > M=E1solat: FreeBSD FS >=20 > Dear FS devs, >=20 > After some investigation, it turned out that when I turn write-cache = off under linux, the performance drops to 100 on that OS also. But when = enabled, 1400 IOPS (synchronous) can be achieved. So I would like to see = the same on FreeBSD as well. Using camcontrol shows that the write cache = is enabled, but I may assume that something around this is causing the = performance degradation. But unfortunately I cannot step forward right = now. >=20 > Regards, >=20 > Kojedzinszky Richard >=20 > On Wed, 27 Nov 2013, Tom Evans wrote: >=20 >> On Wed, Nov 27, 2013 at 8:51 AM, Richard Kojedzinszky = wrote: >>> Dear fs developers, >>> Probably this is not the best list to report my issue, but please = forward it >>> to where it should get. >>> I bought an SSD for my ZFS filesystem to use it as a ZIL. I've = tested it >>> under linux, and found that it can handle around 1400 random = synchronized >>> write IOPS. Then I placed it into my freebsd 9.2 box, and after = attaching it >>> as a ZIL, my zpool only performs 100 (!) write iops. I've attached = it to an >>> AHCI controller and to an LSI 1068 controller, on both it behaves = the same. >>> So I expect that something in the scsi layer is different, FreeBSD = is >>> handling this device slower, but actually it can handle the 1400 = iops as >>> tested under linux. >>> Please give some advice where to go, how to debug, and how to = improve >>> FreeBSD's performance with this drive. >> The ZIL is only used for synchronous writes. The majority of writes >> are asynchronous, and the ZIL is not used at all. Plus, a ZIL can = only >> increase iops by bundling writes - if your underlying pool is write >> saturated already, then a ZIL can't help - any data written to the = ZIL >> has to end up on the pool. >> Test the SSD by itself under FreeBSD to rule out FreeBSD not working >> correctly on the SSD (I doubt this though). >> Cheers >> Tom > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 29 14:52:47 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 048F6DDB for ; Fri, 29 Nov 2013 14:52:47 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id B54C01AE6 for ; Fri, 29 Nov 2013 14:52:46 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 9BB311508 for ; Fri, 29 Nov 2013 15:52:38 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Fri, 29 Nov 2013 15:52:36 +0100 From: krichy@cflinux.hu To: freebsd-scsi@freebsd.org Subject: Re: Fwd: Re: ssd for zfs In-Reply-To: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> References: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> Message-ID: <81b87ca49aef7d88db1b4dbd5f7eb201@cflinux.hu> X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2013 14:52:47 -0000 Dear Devs, Regarding my problem, is that possible that the device honours the SYNCHRONIZE_CACHE command slowly? Just because linux and bsd detects its cache setting as disabled when attaching it to the system. Is that possible that when linux detects it has no write-cache, it skips SYNCHRONIZE_CACHE commands? By the way, the SSD still does not lose any data with this behaviour according to tests. Thanks in advance, 2013-11-27 15:14 időpontban krichy@cflinux.hu ezt írta: > -------- Eredeti üzenet -------- > Tárgy: Re: ssd for zfs > Dátum: 2013-11-27 14:07 > Feladó: Richard Kojedzinszky > Címzett: Tom Evans > Másolat: FreeBSD FS > > Dear FS devs, > > After some investigation, it turned out that when I turn write-cache > off under linux, the performance drops to 100 on that OS also. But > when enabled, 1400 IOPS (synchronous) can be achieved. So I would like > to see the same on FreeBSD as well. Using camcontrol shows that the > write cache is enabled, but I may assume that something around this is > causing the performance degradation. But unfortunately I cannot step > forward right now. > > Regards, > > Kojedzinszky Richard > > On Wed, 27 Nov 2013, Tom Evans wrote: > >> On Wed, Nov 27, 2013 at 8:51 AM, Richard Kojedzinszky >> wrote: >>> Dear fs developers, >>> >>> Probably this is not the best list to report my issue, but please >>> forward it >>> to where it should get. >>> >>> I bought an SSD for my ZFS filesystem to use it as a ZIL. I've tested >>> it >>> under linux, and found that it can handle around 1400 random >>> synchronized >>> write IOPS. Then I placed it into my freebsd 9.2 box, and after >>> attaching it >>> as a ZIL, my zpool only performs 100 (!) write iops. I've attached it >>> to an >>> AHCI controller and to an LSI 1068 controller, on both it behaves the >>> same. >>> So I expect that something in the scsi layer is different, FreeBSD is >>> handling this device slower, but actually it can handle the 1400 iops >>> as >>> tested under linux. >>> >>> Please give some advice where to go, how to debug, and how to improve >>> FreeBSD's performance with this drive. >>> >> >> The ZIL is only used for synchronous writes. The majority of writes >> are asynchronous, and the ZIL is not used at all. Plus, a ZIL can only >> increase iops by bundling writes - if your underlying pool is write >> saturated already, then a ZIL can't help - any data written to the ZIL >> has to end up on the pool. >> >> Test the SSD by itself under FreeBSD to rule out FreeBSD not working >> correctly on the SSD (I doubt this though). >> >> Cheers >> >> Tom >> From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 29 20:17:55 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9756FF55 for ; Fri, 29 Nov 2013 20:17:55 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id 17B6C1D56 for ; Fri, 29 Nov 2013 20:17:54 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 87F311745 for ; Fri, 29 Nov 2013 21:17:53 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Fri, 29 Nov 2013 21:17:50 +0100 From: krichy@cflinux.hu To: freebsd-scsi@freebsd.org Subject: Re: Fwd: Re: ssd for zfs In-Reply-To: <81b87ca49aef7d88db1b4dbd5f7eb201@cflinux.hu> References: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> <81b87ca49aef7d88db1b4dbd5f7eb201@cflinux.hu> Message-ID: <923bc6b4aa9141e5e9a49ab76b3a18b5@cflinux.hu> X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2013 20:17:55 -0000 Dear scsi-devs, It seems that my device handles the flush commands slowly. The linux code avoids issuing this if the device reports its write cache is turned off. That way my SSD works fast under linux, but infortunately slow under FreeBSD. Actually I dont understand why is it slow for the flush command, as it has power-loss-protection, maybe for such a command it really flushes everything out, dont know. But when I enable the write cache in linux also, the block layer gets knowledge of the write cache, it issues the flush commands, and it gets same slow. How could this be solved? Thanks in advance, 2013-11-29 15:52 időpontban krichy@cflinux.hu ezt írta: > Dear Devs, > > Regarding my problem, is that possible that the device honours the > SYNCHRONIZE_CACHE command slowly? Just because linux and bsd detects > its cache setting as disabled > when attaching it to the system. Is that possible that when linux > detects it has no write-cache, it skips SYNCHRONIZE_CACHE commands? By > the way, the SSD still does not lose any data > with this behaviour according to tests. > > Thanks in advance, > > 2013-11-27 15:14 időpontban krichy@cflinux.hu ezt írta: >> -------- Eredeti üzenet -------- >> Tárgy: Re: ssd for zfs >> Dátum: 2013-11-27 14:07 >> Feladó: Richard Kojedzinszky >> Címzett: Tom Evans >> Másolat: FreeBSD FS >> >> Dear FS devs, >> >> After some investigation, it turned out that when I turn write-cache >> off under linux, the performance drops to 100 on that OS also. But >> when enabled, 1400 IOPS (synchronous) can be achieved. So I would like >> to see the same on FreeBSD as well. Using camcontrol shows that the >> write cache is enabled, but I may assume that something around this is >> causing the performance degradation. But unfortunately I cannot step >> forward right now. >> >> Regards, >> >> Kojedzinszky Richard >> >> On Wed, 27 Nov 2013, Tom Evans wrote: >> >>> On Wed, Nov 27, 2013 at 8:51 AM, Richard Kojedzinszky >>> wrote: >>>> Dear fs developers, >>>> >>>> Probably this is not the best list to report my issue, but please >>>> forward it >>>> to where it should get. >>>> >>>> I bought an SSD for my ZFS filesystem to use it as a ZIL. I've >>>> tested it >>>> under linux, and found that it can handle around 1400 random >>>> synchronized >>>> write IOPS. Then I placed it into my freebsd 9.2 box, and after >>>> attaching it >>>> as a ZIL, my zpool only performs 100 (!) write iops. I've attached >>>> it to an >>>> AHCI controller and to an LSI 1068 controller, on both it behaves >>>> the same. >>>> So I expect that something in the scsi layer is different, FreeBSD >>>> is >>>> handling this device slower, but actually it can handle the 1400 >>>> iops as >>>> tested under linux. >>>> >>>> Please give some advice where to go, how to debug, and how to >>>> improve >>>> FreeBSD's performance with this drive. >>>> >>> >>> The ZIL is only used for synchronous writes. The majority of writes >>> are asynchronous, and the ZIL is not used at all. Plus, a ZIL can >>> only >>> increase iops by bundling writes - if your underlying pool is write >>> saturated already, then a ZIL can't help - any data written to the >>> ZIL >>> has to end up on the pool. >>> >>> Test the SSD by itself under FreeBSD to rule out FreeBSD not working >>> correctly on the SSD (I doubt this though). >>> >>> Cheers >>> >>> Tom >>> From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 29 20:35:41 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C384667 for ; Fri, 29 Nov 2013 20:35:41 +0000 (UTC) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1764E1E3B for ; Fri, 29 Nov 2013 20:35:28 +0000 (UTC) Received: from r2d2 ([82.69.179.241]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50006875075.msg for ; Fri, 29 Nov 2013 20:35:27 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 29 Nov 2013 20:35:27 +0000 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.179.241 X-Return-Path: prvs=10451ca0e0=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-scsi@freebsd.org Message-ID: From: "Steven Hartland" To: , References: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> <81b87ca49aef7d88db1b4dbd5f7eb201@cflinux.hu> <923bc6b4aa9141e5e9a49ab76b3a18b5@cflinux.hu> Subject: Re: Fwd: Re: ssd for zfs Date: Fri, 29 Nov 2013 20:35:20 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2013 20:35:41 -0000 You could add a DA_Q_NO_SYNC_CACHE quirk for you device which will prevent the cam layer attempting to call sync but this will only work if its attached to a SCSI controller like an LSI there's no such quirk currently implemented in ATA cam layer. Regards Steve ----- Original Message ----- From: To: Sent: Friday, November 29, 2013 8:17 PM Subject: Re: Fwd: Re: ssd for zfs > Dear scsi-devs, > > It seems that my device handles the flush commands slowly. The linux > code avoids issuing this if the device reports its write cache is turned > off. That way my SSD works fast under linux, but infortunately slow > under FreeBSD. Actually I dont understand why is it slow for the flush > command, as it has power-loss-protection, maybe for such a command it > really flushes everything out, dont know. But when I enable the write > cache in linux also, the block layer gets knowledge of the write cache, > it issues the flush commands, and it gets same slow. > > How could this be solved? ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 29 21:07:14 2013 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D06E39C3 for ; Fri, 29 Nov 2013 21:07:14 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id 8AC4D1F99 for ; Fri, 29 Nov 2013 21:07:14 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 487E9179B; Fri, 29 Nov 2013 22:07:13 +0100 (CET) Date: Fri, 29 Nov 2013 22:07:10 +0100 (CET) From: Richard Kojedzinszky X-X-Sender: krichy@pi.nmdps.net To: Steven Hartland Subject: Re: Fwd: Re: ssd for zfs In-Reply-To: Message-ID: References: <0b12c19b8832c72369ff7244d7231846@cflinux.hu> <81b87ca49aef7d88db1b4dbd5f7eb201@cflinux.hu> <923bc6b4aa9141e5e9a49ab76b3a18b5@cflinux.hu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2013 21:07:14 -0000 Dear Steve, Thanks, I've already did that, but as you've mentioned, that is not a possible solution for an ata controller. And what if one enables the write cache on a device, than probably flush commands should be issued to guarantee consistency. What if FreeBSD would handle flushing as linux, ie keep track of current write cache setting, and if that is disabled for a drive, simply ignore flush commands? This could be implemented for ata and scsi drives, or in the geom layer. Regards, Kojedzinszky Richard On Fri, 29 Nov 2013, Steven Hartland wrote: > You could add a DA_Q_NO_SYNC_CACHE quirk for you device > which will prevent the cam layer attempting to call sync but this > will only work if its attached to a SCSI controller like an LSI there's > no such quirk currently implemented in ATA cam layer. > > Regards > Steve > ----- Original Message ----- From: > To: > Sent: Friday, November 29, 2013 8:17 PM > Subject: Re: Fwd: Re: ssd for zfs > > >> Dear scsi-devs, >> >> It seems that my device handles the flush commands slowly. The linux code >> avoids issuing this if the device reports its write cache is turned off. >> That way my SSD works fast under linux, but infortunately slow under >> FreeBSD. Actually I dont understand why is it slow for the flush command, >> as it has power-loss-protection, maybe for such a command it really flushes >> everything out, dont know. But when I enable the write cache in linux also, >> the block layer gets knowledge of the write cache, it issues the flush >> commands, and it gets same slow. >> >> How could this be solved? > > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > >