Date: Mon, 18 Mar 2013 18:08:44 +0100 From: Davide D'Amico <davide.damico@contactlab.com> To: kpneal@pobox.com Cc: freebsd-fs@freebsd.org Subject: Re: FreBSD 9.1 and ZFS v28 performances Message-ID: <51474A1C.7090604@contactlab.com> In-Reply-To: <20130318163833.GA11916@neutralgood.org> References: <514729BD.2000608@contactlab.com> <810E5C08C2D149DBAC94E30678234995@multiplay.co.uk> <20130318163833.GA11916@neutralgood.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Il 18/03/13 17:38, kpneal@pobox.com ha scritto: > On Mon, Mar 18, 2013 at 03:31:51PM -0000, Steven Hartland wrote: >> >> ----- Original Message ----- >> From: "Davide D'Amico" <davide.damico@contactlab.com> >> To: <freebsd-fs@freebsd.org> >> Sent: Monday, March 18, 2013 2:50 PM >> Subject: FreBSD 9.1 and ZFS v28 performances >> >> >>> Hi all, >>> I'm trying to use ZFS on a DELL R720 with 2x6-core, 32GB ram, H710 >>> controller (no JBOD) and 15K rpm SAS HD: I will use it for a mysql 5.6 >>> server, so I am trying to use ZFS to get L2ARC and ZIL benefits. >>> >>> I created a RAID10 and used zpool to create a pool on top: >>> >>> # zpool create DATA mfid3 >>> # zpool add DATA cache mfid1 log mfid2 >>> >>> I have a question on zfs performances. Using: >>> >>> dd if=/dev/zero of=file.out bs=16k count=1M >>> >>> I cannot go faster than 400MB/s so I think I'm missing something; I >>> tried removing zil, removing l2arc but everything is still the same. > > The ZIL only helps with synchronous writes. This is something apps must > request specifically typically and I would guess that dd would not do that. > So the ZIL doesn't affect your test. > > The L2ARC is a read cache. It does very little for writes. If the ZFS cache > working set fits all in memory then the L2ARC does nothing for you. Since > you are writing the only thing needed from the ARC is metadata. > >>> mfiutil show volumes: >>> mfi0 Volumes: >>> Id Size Level Stripe State Cache Name >>> mfid0 ( 278G) RAID-1 64k OPTIMAL Disabled <BASE> >>> mfid1 ( 118G) RAID-0 64k OPTIMAL Disabled <L2ARC0> >>> mfid2 ( 118G) RAID-0 64k OPTIMAL Disabled <ZIL0> >>> mfid3 ( 1116G) RAID-10 64k OPTIMAL Disabled <DATA> >>> >>> zpool status: >>> pool: DATA >>> state: ONLINE >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> DATA ONLINE 0 0 0 >>> mfid3 ONLINE 0 0 0 >>> logs >>> mfid2 ONLINE 0 0 0 >>> cache >>> mfid1 ONLINE 0 0 0 > > Warning: your ZIL should probably be mirrored. If it isn't, and the drive > fails, AND your machine takes a sudden dive (kernel panic, power outage, > etc) then you will lose data. I know: here is only one drive because i'm testing, yet. > >> DATA primarycache metadata local >> DATA secondarycache all default > > Is there a specific reason that you are making a point of not putting > regular data in the ARC? If you do that then reads of data will look in > the L2ARC, which is a normal 15k drive, before hitting the main pool drives > which also consists of normal 15k drives. Adding an extra set of spinning > rust before accessing your spinning rust doesn't sound helpful. Ok, good point. > >> HEAD has some significant changes for the mfi driver specifically:- >> http://svnweb.freebsd.org/base?view=revision&revision=247369 >> >> This fixes lots off bugs but also enables full queue support on TBOLT >> cards so if your mfi is a TBOLT card you may see some speed up in >> random IO, not that this would effect your test here. > > I believe the H710 is a TBOLT card. It was released with the 12G servers > like the R720. Anyway I cannot use -CURRENT on a production server, so I should use 9.1. My 'goal' is to understand if I can perform better than UFS on the same hardware setup. So, I removed the pool and formatted mfid3 using UFS: # mount | grep mfid3 /dev/mfid3 on /DATA (ufs, local, soft-updates) where I have: # dd if=/dev/zero of=file.out bs=8k count=1M 1048576+0 records in 1048576+0 records out 8589934592 bytes transferred in 21.406975 secs (401268025 bytes/sec) (executed 4-5 times) I have the same throughput than ZFS with or without ZIL. I don't know if this is "normal" or if I am missing something on my setup, that's why I'm asking if I can do something more or if with this setup this value is the best I can have. > > I don't believe the OP mentioned how many drives are in the RAID10. More > drives ~== more parallelism ~== better performance. So I too am wondering > how much performance is expected. > >> While having a separate ZIL disk is good, your benefits may well be >> limited if said disk is a traditional HD, better to look at enterprise >> SSD's for this. The same and them some applies to your L2ARC disks. > > Before purchasing SSD's check the H710 docs to make sure they are allowed. > The 6/i in my R610 specifically says that if an SSD is used it must be the > only drive. Your R720's H710 is much newer and thus may not have that > restriction. Still, checking the documentation is cheap. > Yes, in R620 SSD are allowed (and a possible choice in the online DELL configurator). Thanks, d.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51474A1C.7090604>