Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 14 Apr 2011 17:20:10 +0200
From:      Damien Fleuriot <ml@my.gd>
To:        freebsd-questions@freebsd.org
Subject:   Re: ZFS performance strangeness
Message-ID:  <4DA710AA.2030501@my.gd>
In-Reply-To: <espd78-2at.ln1@it-betrieb.de>
References:  <espd78-2at.ln1@it-betrieb.de>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help


On 4/12/11 1:33 PM, Lars Wilke wrote:
> Hi,
> 
> There are quite a few threads about ZFS and performance difficulties,
> but i did not find anything that really helped :)
> Therefor any advice would be highly appreciated.
> I started to use ZFS with 8.1R, only tuning i did was setting
> 
> vm.kmem_size_scale="1"
> vfs.zfs.arc_max="40000M"
> 
> The machines are supermicro boards with 48 GB ECC RAM and 15k RPM SAS
> drives. Local read/write performance was and is great.
> But exporting via NFS was a mixed bag in 8.1R.
> Generally r/w speed over NFS was ok, but large reads or writes took
> ages. Most of the reads and writes were small, so i did not bother.
> 
> Now i upgraded one machine to 8.2R and i get very good write performance
> over NFS but read performance drops to a ridiciously low value, around
> 1-2 MB/s. While writes are around 100MB/s. The network is a dedicated
> 1GB Ethernet. The zpool uses RAIDZ1 over 7 drives, one vdev.
> The filesystem has compression enabled. Turning it off made no
> difference AFAICT
> 
> Now i tried a few of the suggested tunables and my last try was this
> 
> vfs.zfs.txg.timeout="5"
> vfs.zfs.prefetch_disable="1"
> vfs.zfs.txg.synctime=2
> fs.zfs.vdev.min_pending="1"
> fs.zfs.vdev.max_pending="1"
> 
> still no luck. Writting is fast, reading is not. Even with enabled
> prefetching. The only thing i noticed is, that reading for example 10MB
> is fast (on a freshly mounted fs) but when reading larger amounts, i.e.
> couple hundred MBs, the performance drops and zpool iostat or iostat -x
> show that there is not much activity on the zpool/hdds.
> 
> It seems as if ZFS does not care that someone wants to read data, also idle
> time of the reading process happily ticks up and gets higher and higher!?
> When trying to access the file during this time, the process blocks and
> sometimes is difficult to kill, i.e. ls -la on the file.
> 
> I read and write with dd and before read tests i umount and mount the
> NFS share again.
> 
> dd if=/dev/zero of=/mnt/bla size=1M count=X
> dd if=/mnt/bla of=/dev/null size=1M count=Y
> 
> mount is done with this options from two centos 5 boxes:
> rw,noatime,tcp,bg,intr,hard,nfsvers=3,noacl,nocto
> 
> thanks
>    --lars
> 
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"



I refer you to this post by Jeremy Chadwick with tuning values *AND*
their actual explanation.

http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061642.html



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?4DA710AA.2030501>