From owner-freebsd-scsi@FreeBSD.ORG Tue Sep 23 08:11:37 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BAF3ABB; Tue, 23 Sep 2014 08:11:37 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD8ACE9B; Tue, 23 Sep 2014 08:11:36 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id r10so5872755pdi.26 for ; Tue, 23 Sep 2014 01:11:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=88zcngsJ6wBfkVu2BNiTx4qtAat/tz5iFJuUaPSdXDE=; b=soTAKZ/S1vLWBuB08iBxViJWJZE2GkRfYq0BEunKdnov3Idq2wnoxAM/IJMGZYM2Zg NBwENfgpDcanSiNBb0unCrQibRbxBeySoZmBEg3fRO/SJJln1ScdaZ40BwsNco6Bv8wX hl7ZejjFPYv52twy0/paV8LdMxiCdMUG/S/Cn+BeeIpGpQfA/Ca1NjJcUIAORP+d4DQ7 Q21Ecbzw5KeAETYMTeaVbCOh4mGM/u6sopwvVKZz3R235NVq6i4R/jg1vY2+Fuxl9msl c4sg79pDjN3fDNOWVUl+P8kKunGpBGbDn9rpYLQARWfbX4Def/qb56bzOIFtDTSGoHaD hhyg== X-Received: by 10.70.131.13 with SMTP id oi13mr35964706pdb.23.1411459896054; Tue, 23 Sep 2014 01:11:36 -0700 (PDT) Received: from Peters-MacAir.local ([106.39.255.55]) by mx.google.com with ESMTPSA id iq3sm11304345pbb.71.2014.09.23.01.11.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 23 Sep 2014 01:11:35 -0700 (PDT) Message-ID: <54212B31.7050608@gmail.com> Date: Tue, 23 Sep 2014 16:11:29 +0800 From: Xu Zhe User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Alexander Motin , d@delphij.net, freebsd-scsi@freebsd.org Subject: Re: How to disable hard disk write cache? References: <541804B0.7070407@gmail.com> <54184484.1070304@delphij.net> <5418FA4A.1050707@gmail.com> <541E974E.7060702@delphij.net> <541EA01E.6020600@ixsystems.com> In-Reply-To: <541EA01E.6020600@ixsystems.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2014 08:11:37 -0000 Hi, Alexandar, Xin, This is exactly what I am looking for. Thanks alot. :) Peter 于 14-9-21 17:53, Alexander Motin 写道: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 21.09.2014 12:15, Xin Li wrote: >> On 9/17/14 11:04 AM, Xu Zhe wrote: >>> I have googled about the "tagged command" but only found >>> something related to TCQ, which mainly talks about the queueing >>> but not anything related to cache sync. Any more hints? >> Hrm I don't have much thing off hand, but these are defined by the >> standards. Alexander knows much more than I do in this area. >> >>> I saw some pages that mentioned about SATA FUA command support >>> on Linux (Which I guess is what I am looking for): >>> https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt >>> However, I have not found useful information related to >>> Freebsd. :( >> Probably g_bio(9) but the documentation do not have much detail on >> driver implementation, and callers of the API expects the driver to >> do all the right things. This is if you want to implement a file >> system, where you would go. >> >> But back to your question, if you want to know e.g., "how to >> disable write cache on an AHCI connected drive", you would go to >> the individual driver's manual, e.g. ada(4). > I see there was several different topics touched in this threads, so > let me start from the beginning. There are several methods to control > disk caches, and respectively several approaches to maintaining the > data consistency: > > - Caches can be controlled globally > SCSI disks allow to control write cache via the Caching mode page. You > may do it with `camcontrol mode` command. ATA disks has specific > commands for that, and easiest way to do it is using sysctls/tunables > documented in ada(4) manual page. Disabling caches globally heavily > affects performance, and in most cases is overkill. It means that > _every_ request will go to the media before the operation complete. > For disks without command queuing (like legacy ATA) that usually meant > _one_ I/O request per platter revolution. Do you want disk doing 120 > IOPS peak? If you write huge file in 128K chunks, you will get limited > by 120/s * 128K = 15MB/s! Command queuing (NCQ for SATA) significantly > improved this situation since OS can now send more operations down to > the disk to give it more flexibility, in significant part compensating > disabled cache. But number of simultaneously active tags in disk, HBA > or application can be limited, creating delays. > > - Caches can be controlled per-command > SCSI disks may support FUA and DPO flags, that allow to do above cache > control on per-command basis. SATA disks got FUA flag just recently. > This approach has all properties of above, just can be controller per > data type or application. Those flags are equivalent of IO_SYNC and > IO_DIRECT flags on VFS layer of FreeBSD. Though FreeBSD block layer > never had their equivalents. I've heard that Windows NTFS used this > technique to keep metadata consistency up to some point, but moved > away from it. > > - Caches can be flushed to media with SYNCHRONIZE CACHE commands > Both SATA and SCSI disks provide ways to flush write caches to the > media on request. It allows disks to do many writes in any order they > prefer within one transaction group, but then reliably push everything > to the media before when the group being committed. FreeBSD supports > that on both VFS (fflush/VOP_FSYNC) and block layers (BIO_FLUSH). ZFS > on FreeBSD relies on it to keep transaction groups atomicity and so > metadata correctness. I've heard, this is what Windows NTFS is doing > now too. > > I hope that answered most of questions. > > - -- > Alexander Motin > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQF8BAEBCgBmBQJUHqAeXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w > ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4 > MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/E6UH+QF5AU3KNKjBLyNsgGTOn+UV > SZAj3V3JLUNM4Z+8Z+4rxY/26/nbfNPoEJD4SaOt74oEUA574XjVLkHmqZ5JKygV > dzOE2m8t0gyDIPurGx2CfRyG7UdJKbVsJ7Ebxafd8lRbwHGoObySUmew8t8WSIHA > zBsI3ZiRYzBX7NnejPlJ8UPh498PBl78U+Ak08q/0scdPWsCneCjqHHn0Rx2MEFp > 7sllphFh+EBXJXqetFRJfuWmm7yfIrzjO5UJ1mTjq5dCSeXrsIxBMBTSMEG34Yv6 > 9PqPWYPdVWQKgluKcaTKgrzPVTBgAqefx4gHRm/460a+7qohloCelMsgAsttYuA= > =De3m > -----END PGP SIGNATURE-----