Date: Fri, 23 May 2014 08:39:19 +0100 From: krad <kraduk@gmail.com> To: Stefan Esser <se@freebsd.org> Cc: FreeBSD FS <freebsd-fs@freebsd.org> Subject: Re: Turn off RAID read and write caching with ZFS? Message-ID: <CALfReycEBCKS_VGxryE7EcpB2gNkfn6fRL%2BO0oPKDWBjVbXkNA@mail.gmail.com> In-Reply-To: <537DFD70.3010705@freebsd.org> References: <719056985.20140522033824@supranet.net> <537DF2F3.10604@denninger.net> <537DFD70.3010705@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
i think the general rule is correct in that you should turn off caching. Simply because you have two algorithms trying to be clever and most probably undermining each other. If you are worried about synchronous writes, then put some ssds in and put zil on it, even if its under the raid card, just make sure its a separate volume thats exported as a lun. There may be some gains to be made in certain scenarios with this hardware caching but that means you are going to have to test extensively to make sure its working in your case, which may or may not be worth while to you. On 22 May 2014 14:36, Stefan Esser <se@freebsd.org> wrote: > Am 22.05.2014 14:52, schrieb Karl Denninger: > [...] > > Modern drives typically try to compensate for their > > variable-geometryness through their own read-ahead cache, but the exact > > details of their algorithm are typically not exposed. > > > > What I would love to find is a "buffered" controller that recognizes all > > of this and works as follows: > > > > 1. Writes, when committed, are committed and no return is made until > > storage has written the data and claims it's on the disk. If the > > sector(s) written are in the buffer memory (from a previous read in 2 > > below) then the write physically alters both the disk AND the buffer. > > > > 2. Reads are always one full track in size and go into the buffer memory > > on a LRU basis. A read for a sector already in the buffer memory > > results in no physical I/O taking place. The controller does not store > > sectors per-se in the buffer, it stores tracks. This requires that the > > adapter be able to discern the *actual* underlying geometry of the drive > > so it knows where track boundaries are. Yes, I know drive caches > > themselves try to do this, but how well do they manage? Evidence > > suggests that it's not particularly effective. > > In the old times, controllers implemented read-ahead, either under > control of the host-adapter or the host OS (e.g. the based on the > detection of sequential access patterns). > > This changed, when large on-drive caches became practical. Drives > now do aggressive read-ahead caching, but without the penalty this > had, in the old times. I do not know, whether this applies to all > current drives, but since it is old technology, I assume so: > > The sector layout is reversed on each track - higher numbered > sectors come first. The drive starts reading data into its cache > as soon as the head receives stable data and it stops only when > the whole requested range of sectors has been read. > > E.g. if you request sectors 10 to 20, the drive may have the read > head positioned when sector 30 comes along. Starting at that sector, > data is read from sectors 30, 29, ..., 10 and stored in the drive's > cache. Only after sector 10 has been read, data is transferred to > the requesting host adapter, while the drive seeks to the next > track to operate on. This scheme offers opportunistic read-ahead, > which does not increase the random access seek times. > > The old method required the head to stay on the track for some > milliseconds to read sectors following the requested block on the > vague chance, that this data might later be requested. > > The new method just starts reading as soon as there is data under > the read head. This needs more cache on the drive, but does not add > latency for read-ahead. The disadvantage is, that you never know > how much read-ahead there will be, it depends on the rotational > position of the disk when the seek ends. And if the first sector > read from the track is in the middle of the requested range, the > drive needs to read the whole track to fulfil the request, but > that would happen with equal probability with the old sector > layout as well. > > > Without this read cache is a crapshoot that gets difficult to tune and > > is very workload-dependent in terms of what delivers best performance. > > All you can do is tune (if you're able with a given controller) and test. > > The read-ahead of reverse sectors as described above does not have > any negative side-effect. On average, you'll read half a track into > the drive's cache whenever you request a single sector. > > A controller that implements read-ahead does this by increasing the > amount of data requested from the drive. This leads to a higher > probability that a full track must be read to satisfy the request > and will thus increase latencies observed by the application. > > Rergards, STefan > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALfReycEBCKS_VGxryE7EcpB2gNkfn6fRL%2BO0oPKDWBjVbXkNA>