From owner-freebsd-current@FreeBSD.ORG Tue Mar 5 05:09:49 2013 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 20B392DF for ; Tue, 5 Mar 2013 05:09:49 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id 04FC6FB9 for ; Tue, 5 Mar 2013 05:09:48 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id r2559YCs012409; Mon, 4 Mar 2013 21:09:39 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201303050509.r2559YCs012409@gw.catspoiler.org> Date: Mon, 4 Mar 2013 21:09:34 -0800 (PST) From: Don Lewis Subject: Re: access to hard drives is "blocked" by writes to a flash drive To: peter@rulingia.com In-Reply-To: <20130304083802.GA44865@server.rulingia.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-current@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 05:09:49 -0000 On 4 Mar, Peter Jeremy wrote: > On 2013-Mar-03 23:12:40 -0800, Don Lewis wrote: >>On 4 Mar, Konstantin Belousov wrote: >>> It could be argued that the current typical value of 16MB for the >>> hirunningbufspace is too low, but experiments with increasing it did >>> not provided any measureable change in the throughput or latency for >>> some loads. >> >>The correct value is probably proportional to the write bandwidth >>available. > > The problem is that write bandwidth varies widely depending on the > workload. For spinning rust, this will vary between maybe 64KBps > (512B random writes) and 100-150MBps (single-theaded large sequential > writes). The (low-end) SSD in my Netbook also has about 100:1 variance > due to erase blocking. How do you tune hirunningbufspace in the face > of 2 or 3 orders of magnitude variance in throughput? Especially since > SSDs don't gradually degrade - they hit a brick wall. I think a latency target would be desirable. As a first cut, maybe a limit on the number of write ops per device instead of a limit on the amount of data. That should more or less do the correct thing for small random I/O vs. large sequential writes. It could be somewhat self tuning if we kept track of the average service time, which should probably be measured with write caching off. There might still need to be a higher size limit to avoid excessive memory consumption. If you only have a single drive and it is an SSD, then there's probably not much you can do about the erase blocking problem. About the best you could do is if it supports NCQ is to turn off write caching so that you can keep track of how many writes are outstanding so that you can limit the write load to something less than the maximum number of outstanding commands that it supports, and then hope that when you run into the erase blocking situation, that *maybe* it will still process reads. If you have multiple drives, then you don't want one backlogged drive to block writes to your other drive(s), which would require something other than a global hirunningbufspace limit.