From owner-freebsd-fs@freebsd.org Tue Jan 5 11:51:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9142A62383 for ; Tue, 5 Jan 2016 11:51:10 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8863D1528 for ; Tue, 5 Jan 2016 11:51:10 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by mail-ig0-x22b.google.com with SMTP id ik10so10547145igb.1 for ; Tue, 05 Jan 2016 03:51:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dE9mh7AqyC57Yhk6meGwqjMtchTYwYzFWh/Bgmg+ynw=; b=U/1lXbakm/5JYsWJM4S+Mm25W9cbA+yhNridrFSdhdsczQRbQpx6rBRKrHPr5ZsBwo tllXQ4a3HoymCKl2SiZNmDnRVyx1gLfzdvGo0PRcv8U9vvqDmGSPM4Vdp0yzfxYCEm6f nsu4ojfdNTNv4/AjZPdh6wtS4N1fC591FsujfHdUvEUTE0WnIJetw2uvk/iTOzG1EjrO M2/4PX79A06AgWi+ZiFgXPU48jjw/NG8pBSoZHTCibcUX4Eq60gzjMcPdO403wIlQU+w O/SvXtLMlqjttjoHsLulIOvP9GhGpfdWewpXVhvI+4jR2N78V8IwYDAqELy+EDWWsYTi IcrA== MIME-Version: 1.0 X-Received: by 10.50.50.238 with SMTP id f14mr3409194igo.37.1451994669929; Tue, 05 Jan 2016 03:51:09 -0800 (PST) Received: by 10.107.4.71 with HTTP; Tue, 5 Jan 2016 03:51:09 -0800 (PST) In-Reply-To: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> Date: Tue, 5 Jan 2016 06:51:09 -0500 Message-ID: Subject: Re: NFS reads vs. writes From: Tom Curry To: Paul Kraus Cc: FreeBSD Filesystems , "Mikhail T." Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 11:51:10 -0000 On Mon, Jan 4, 2016 at 10:06 PM, Paul Kraus wrote: > On Jan 4, 2016, at 18:58, Tom Curry wrote: > > > SSDs are so fast for three main reasons: low latency, large dram buffer= s, > > and parallel workloads. Only one of these is of any benefit (latency) a= s > a > > SLOG. Unfortunately that particular metric is not usually advertised in > > consumer SSDs where the benchmarks they use to tout 90,000 random write > > iops consist of massively concurrent, highly compressible, short lived > > bursts of data. Add that drive as a SLOG and the onboard dram may as we= ll > > not even exist, and queue depths count for nothing. It will be lucky to > > pull 2,000 IOPS. Once you start adding in ZFS features like checksums a= nd > > compression, or network latency in the case of NFS that 2,000 number > starts > > to drop even more. > > I have a file server that I am going through the task of optimizing for > NFS traffic (to store VM images). My first attempt, because I knew about > the need for an SSD based SLOG for the ZIL was using a pair of Intel 535 > series SSD=E2=80=99s. The performance with the SLOG/ZIL on the SSD was _w= orse_. > Turns out that those SSD=E2=80=99s have poor small block (8 KB) random wr= ite > performance (not well advertised). So I asked for advice for choosing a > _fast_ SSD on the OpenZFS list and had a number of people recommend the > Intel DC-Sxxxx series of SSDs. > > Based on the very thorough data sheets, I am going with a pair of DC-S3710 > 200 GB SSDs. Once I get them in and configured I=E2=80=99ll post results. > > That is an excellent choice, I'm looking forward to the results. They have capacitors for power loss protection which means its safe to allow them to use their on-board dram cache. Wouldn't it be great if we could tell ZFS to not flush the cache on log devices via sysctl or some other means? Change nothing else about how sync operations occur. Unless I'm missing something this would provide the same level of protection but allow for greatly improved performance of the SSD. > Note that my zpool consists of 5 top level vdevs each made up of a 3-way > mirror. So I am striping writes across 5 columns. I am using 500 GB WD RE > series drives leaving the ZIL on the primary vdevs was _faster_ than addi= ng > the consumer SSD as SLOG for NFS writes. > > -- > Paul Kraus > paul@kraus-haus.org > >