Date: Thu, 7 Dec 2000 20:37:50 -0800 From: Guolin Cheng <Guolin@alexa.com> To: "'freebsd-questions@FreeBSD.ORG'" <freebsd-questions@FreeBSD.ORG> Subject: How to improve FreeBSD nfs performance? and please explain the k ernel parameters. Message-ID: <2A2825AC1334D411A922005004E0CD203F2BEB@exchange.alexa.com>
next in thread | raw e-mail | index | archive | help
Hi, all,
These days I encountered a nfs performance problem, Basically we use Intel
machines with 512M memory, Intel Pro100M Ethernet network interfaces and HP
switches,
But we find that the performance is too bad:
This is the performance test,. write from nfs client to nfs server mounted
through amd ("nfsiod -n 4" and "nfsd -t -u -n 16" are running to improve
performance already):
nfs server: nfs client: transfer speed (Mega bytes)
FreeBSD 4.1 FreeBSD 4.1 2-4 M
FreeBSD 4.1 RedHat Linux 6.2 5-7 M
Linux 6.2 Linux 6.2 10+ M
So it's really terrible, because I like FreeBSD and we install a lot of
FreeBSD boxes. So I tried to change nfs sync-write method, which is default
for FreeBSD, try to use async method, (sysctl -w vfs.nfs.async=1 ; sysctl
-w vfs.nfs.gatherdelay=0 ) and test the performance again, and the
performance is improved a lot
nfs server: nfs client: transfer speed (Mega bytes)
FreeBSD 4.1 FreeBSD 4.1 7-10 M
This is a good nfs performance, But I'm also concerned about the reliabity
and correctness of data, since write-behind is enabled (async write). Anyone
can show me if this is really terrible? From my test these 3 days, I found
that it is ok, no data lost at all.
Also, any one can do me a favor to explain the functions of the following
kernel parameters and their possible values?
1. vfs.nfs.commit_blks
2. vfs.nfs.commit_miss
3. vfs.nfs.realign_test
4. vfs.nfs.realign_count
5. vfs.nfs.bufpackets
6. vfs.nfs.gatherdelay
7. vfs.nfs.gatherdelay_v3
8. vfs.nfs.defect
9 . vfs.nfs.access_cache_timeout
10. vfs.nfs.nfsv3_commit_on_close
The following are collected on my test machine:
ops-test.alexa.com guolin 133% sysctl -a | grep nfs
vfs.nfs.nfs_privport: 0
vfs.nfs.async: 0
vfs.nfs.commit_blks: 34050
vfs.nfs.commit_miss: 19404
vfs.nfs.realign_test: 401468
vfs.nfs.realign_count: 0
vfs.nfs.bufpackets: 0
vfs.nfs.gatherdelay: 0
vfs.nfs.gatherdelay_v3: 0
vfs.nfs.defect: 0
vfs.nfs.diskless_valid: 0
vfs.nfs.diskless_rootpath:
vfs.nfs.diskless_swappath:
vfs.nfs.access_cache_timeout: 2
vfs.nfs.nfsv3_commit_on_close: 0
ops-test.alexa.com guolin 134% nfsstat -c
Client Info:
Rpc Counts:
Getattr Setattr Lookup Readlink Read Write Create
Remove
193672 38 95314 53 73 24454 17
32
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus
Access
0 16 0 0 0 2844 0
2073
Mknod Fsstat Fsinfo PathConf Commit GLease Vacate
Evict
0 1994 560 0 1505 0 0
0
Rpc Info:
TimedOut Invalid X Replies Retries Requests
0 0 1 693 322645
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits
Misses
103198 195218 97904 95314 691 73 3931
24454
BioRLHits Misses BioD Hits Misses DirE Hits Misses
96444 53 620 2844 613 0
ops-test.alexa.com guolin 135%
!!!!!!!!!!!!!!!!!! why there are so many misses? for nfs client
cashes? !!!!!!!!!!!!!!!!!!
ops-test2.alexa.com guolin 141% nfsstat -s
Server Info:
Getattr Setattr Lookup Readlink Read Write Create
Remove
0 5 19 2 0 73248 1
1
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus
Access
0 0 1 1 0 2 0
29
Mknod Fsstat Fsinfo PathConf Commit GLease Vacate
Evict
0 3 1 0 4388 0 0
0
Server Ret-Failed
6
Server Faults
0
Server Cache Stats:
Inprog Idem Non-idem Misses
0 0 0 1163
Server Lease Stats:
Leases PeakL GLeases
0 0 0
Server Write Gathering:
WriteOps WriteRPC Opsaved
73248 73248 0
ops-test2.alexa.com guolin 142%
!!!!!!!!!!!!! why the value of WriteOps and WriteRPC are equal?
!!!!!!!!!!!!!!!!!!!!!!!!!!
Thanks.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2A2825AC1334D411A922005004E0CD203F2BEB>