Date: Tue, 23 Sep 2014 16:11:29 +0800 From: Xu Zhe <xzpeter@gmail.com> To: Alexander Motin <mav@FreeBSD.org>, d@delphij.net, freebsd-scsi@freebsd.org Subject: Re: How to disable hard disk write cache? Message-ID: <54212B31.7050608@gmail.com> In-Reply-To: <541EA01E.6020600@ixsystems.com> References: <541804B0.7070407@gmail.com> <54184484.1070304@delphij.net> <5418FA4A.1050707@gmail.com> <541E974E.7060702@delphij.net> <541EA01E.6020600@ixsystems.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Alexandar, Xin, This is exactly what I am looking for. Thanks alot. :) Peter δΊ 14-9-21 17:53, Alexander Motin ει: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 21.09.2014 12:15, Xin Li wrote: >> On 9/17/14 11:04 AM, Xu Zhe wrote: >>> I have googled about the "tagged command" but only found >>> something related to TCQ, which mainly talks about the queueing >>> but not anything related to cache sync. Any more hints? >> Hrm I don't have much thing off hand, but these are defined by the >> standards. Alexander knows much more than I do in this area. >> >>> I saw some pages that mentioned about SATA FUA command support >>> on Linux (Which I guess is what I am looking for): >>> https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt >>> However, I have not found useful information related to >>> Freebsd. :( >> Probably g_bio(9) but the documentation do not have much detail on >> driver implementation, and callers of the API expects the driver to >> do all the right things. This is if you want to implement a file >> system, where you would go. >> >> But back to your question, if you want to know e.g., "how to >> disable write cache on an AHCI connected drive", you would go to >> the individual driver's manual, e.g. ada(4). > I see there was several different topics touched in this threads, so > let me start from the beginning. There are several methods to control > disk caches, and respectively several approaches to maintaining the > data consistency: > > - Caches can be controlled globally > SCSI disks allow to control write cache via the Caching mode page. You > may do it with `camcontrol mode` command. ATA disks has specific > commands for that, and easiest way to do it is using sysctls/tunables > documented in ada(4) manual page. Disabling caches globally heavily > affects performance, and in most cases is overkill. It means that > _every_ request will go to the media before the operation complete. > For disks without command queuing (like legacy ATA) that usually meant > _one_ I/O request per platter revolution. Do you want disk doing 120 > IOPS peak? If you write huge file in 128K chunks, you will get limited > by 120/s * 128K = 15MB/s! Command queuing (NCQ for SATA) significantly > improved this situation since OS can now send more operations down to > the disk to give it more flexibility, in significant part compensating > disabled cache. But number of simultaneously active tags in disk, HBA > or application can be limited, creating delays. > > - Caches can be controlled per-command > SCSI disks may support FUA and DPO flags, that allow to do above cache > control on per-command basis. SATA disks got FUA flag just recently. > This approach has all properties of above, just can be controller per > data type or application. Those flags are equivalent of IO_SYNC and > IO_DIRECT flags on VFS layer of FreeBSD. Though FreeBSD block layer > never had their equivalents. I've heard that Windows NTFS used this > technique to keep metadata consistency up to some point, but moved > away from it. > > - Caches can be flushed to media with SYNCHRONIZE CACHE commands > Both SATA and SCSI disks provide ways to flush write caches to the > media on request. It allows disks to do many writes in any order they > prefer within one transaction group, but then reliably push everything > to the media before when the group being committed. FreeBSD supports > that on both VFS (fflush/VOP_FSYNC) and block layers (BIO_FLUSH). ZFS > on FreeBSD relies on it to keep transaction groups atomicity and so > metadata correctness. I've heard, this is what Windows NTFS is doing > now too. > > I hope that answered most of questions. > > - -- > Alexander Motin > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQF8BAEBCgBmBQJUHqAeXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w > ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4 > MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/E6UH+QF5AU3KNKjBLyNsgGTOn+UV > SZAj3V3JLUNM4Z+8Z+4rxY/26/nbfNPoEJD4SaOt74oEUA574XjVLkHmqZ5JKygV > dzOE2m8t0gyDIPurGx2CfRyG7UdJKbVsJ7Ebxafd8lRbwHGoObySUmew8t8WSIHA > zBsI3ZiRYzBX7NnejPlJ8UPh498PBl78U+Ak08q/0scdPWsCneCjqHHn0Rx2MEFp > 7sllphFh+EBXJXqetFRJfuWmm7yfIrzjO5UJ1mTjq5dCSeXrsIxBMBTSMEG34Yv6 > 9PqPWYPdVWQKgluKcaTKgrzPVTBgAqefx4gHRm/460a+7qohloCelMsgAsttYuA= > =De3m > -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54212B31.7050608>