Date: Sun, 21 Sep 2014 12:53:34 +0300 From: Alexander Motin <mav@FreeBSD.org> To: d@delphij.net, Xu Zhe <xzpeter@gmail.com>, freebsd-scsi@freebsd.org Subject: Re: How to disable hard disk write cache? Message-ID: <541EA01E.6020600@ixsystems.com> In-Reply-To: <541E974E.7060702@delphij.net> References: <541804B0.7070407@gmail.com> <54184484.1070304@delphij.net> <5418FA4A.1050707@gmail.com> <541E974E.7060702@delphij.net>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 21.09.2014 12:15, Xin Li wrote: > On 9/17/14 11:04 AM, Xu Zhe wrote: >> I have googled about the "tagged command" but only found >> something related to TCQ, which mainly talks about the queueing >> but not anything related to cache sync. Any more hints? > > Hrm I don't have much thing off hand, but these are defined by the > standards. Alexander knows much more than I do in this area. > >> I saw some pages that mentioned about SATA FUA command support >> on Linux (Which I guess is what I am looking for): > >> https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt > >> However, I have not found useful information related to >> Freebsd. :( > > Probably g_bio(9) but the documentation do not have much detail on > driver implementation, and callers of the API expects the driver to > do all the right things. This is if you want to implement a file > system, where you would go. > > But back to your question, if you want to know e.g., "how to > disable write cache on an AHCI connected drive", you would go to > the individual driver's manual, e.g. ada(4). I see there was several different topics touched in this threads, so let me start from the beginning. There are several methods to control disk caches, and respectively several approaches to maintaining the data consistency: - Caches can be controlled globally SCSI disks allow to control write cache via the Caching mode page. You may do it with `camcontrol mode` command. ATA disks has specific commands for that, and easiest way to do it is using sysctls/tunables documented in ada(4) manual page. Disabling caches globally heavily affects performance, and in most cases is overkill. It means that _every_ request will go to the media before the operation complete. For disks without command queuing (like legacy ATA) that usually meant _one_ I/O request per platter revolution. Do you want disk doing 120 IOPS peak? If you write huge file in 128K chunks, you will get limited by 120/s * 128K = 15MB/s! Command queuing (NCQ for SATA) significantly improved this situation since OS can now send more operations down to the disk to give it more flexibility, in significant part compensating disabled cache. But number of simultaneously active tags in disk, HBA or application can be limited, creating delays. - Caches can be controlled per-command SCSI disks may support FUA and DPO flags, that allow to do above cache control on per-command basis. SATA disks got FUA flag just recently. This approach has all properties of above, just can be controller per data type or application. Those flags are equivalent of IO_SYNC and IO_DIRECT flags on VFS layer of FreeBSD. Though FreeBSD block layer never had their equivalents. I've heard that Windows NTFS used this technique to keep metadata consistency up to some point, but moved away from it. - Caches can be flushed to media with SYNCHRONIZE CACHE commands Both SATA and SCSI disks provide ways to flush write caches to the media on request. It allows disks to do many writes in any order they prefer within one transaction group, but then reliably push everything to the media before when the group being committed. FreeBSD supports that on both VFS (fflush/VOP_FSYNC) and block layers (BIO_FLUSH). ZFS on FreeBSD relies on it to keep transaction groups atomicity and so metadata correctness. I've heard, this is what Windows NTFS is doing now too. I hope that answered most of questions. - -- Alexander Motin -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQF8BAEBCgBmBQJUHqAeXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4 MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/E6UH+QF5AU3KNKjBLyNsgGTOn+UV SZAj3V3JLUNM4Z+8Z+4rxY/26/nbfNPoEJD4SaOt74oEUA574XjVLkHmqZ5JKygV dzOE2m8t0gyDIPurGx2CfRyG7UdJKbVsJ7Ebxafd8lRbwHGoObySUmew8t8WSIHA zBsI3ZiRYzBX7NnejPlJ8UPh498PBl78U+Ak08q/0scdPWsCneCjqHHn0Rx2MEFp 7sllphFh+EBXJXqetFRJfuWmm7yfIrzjO5UJ1mTjq5dCSeXrsIxBMBTSMEG34Yv6 9PqPWYPdVWQKgluKcaTKgrzPVTBgAqefx4gHRm/460a+7qohloCelMsgAsttYuA= =De3m -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?541EA01E.6020600>