From owner-freebsd-questions Thu Dec 7 20:38: 1 2000 From owner-freebsd-questions@FreeBSD.ORG Thu Dec 7 20:37:57 2000 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from internal.alexa.com (unknown [209.247.40.13]) by hub.freebsd.org (Postfix) with ESMTP id E4B2D37B400 for ; Thu, 7 Dec 2000 20:37:56 -0800 (PST) Received: by exchange.alexa.com with Internet Mail Service (5.5.2650.21) id ; Thu, 7 Dec 2000 20:37:51 -0800 Message-ID: <2A2825AC1334D411A922005004E0CD203F2BEB@exchange.alexa.com> From: Guolin Cheng To: "'freebsd-questions@FreeBSD.ORG'" Subject: How to improve FreeBSD nfs performance? and please explain the k ernel parameters. Date: Thu, 7 Dec 2000 20:37:50 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, all, These days I encountered a nfs performance problem, Basically we use Intel machines with 512M memory, Intel Pro100M Ethernet network interfaces and HP switches, But we find that the performance is too bad: This is the performance test,. write from nfs client to nfs server mounted through amd ("nfsiod -n 4" and "nfsd -t -u -n 16" are running to improve performance already): nfs server: nfs client: transfer speed (Mega bytes) FreeBSD 4.1 FreeBSD 4.1 2-4 M FreeBSD 4.1 RedHat Linux 6.2 5-7 M Linux 6.2 Linux 6.2 10+ M So it's really terrible, because I like FreeBSD and we install a lot of FreeBSD boxes. So I tried to change nfs sync-write method, which is default for FreeBSD, try to use async method, (sysctl -w vfs.nfs.async=1 ; sysctl -w vfs.nfs.gatherdelay=0 ) and test the performance again, and the performance is improved a lot nfs server: nfs client: transfer speed (Mega bytes) FreeBSD 4.1 FreeBSD 4.1 7-10 M This is a good nfs performance, But I'm also concerned about the reliabity and correctness of data, since write-behind is enabled (async write). Anyone can show me if this is really terrible? From my test these 3 days, I found that it is ok, no data lost at all. Also, any one can do me a favor to explain the functions of the following kernel parameters and their possible values? 1. vfs.nfs.commit_blks 2. vfs.nfs.commit_miss 3. vfs.nfs.realign_test 4. vfs.nfs.realign_count 5. vfs.nfs.bufpackets 6. vfs.nfs.gatherdelay 7. vfs.nfs.gatherdelay_v3 8. vfs.nfs.defect 9 . vfs.nfs.access_cache_timeout 10. vfs.nfs.nfsv3_commit_on_close The following are collected on my test machine: ops-test.alexa.com guolin 133% sysctl -a | grep nfs vfs.nfs.nfs_privport: 0 vfs.nfs.async: 0 vfs.nfs.commit_blks: 34050 vfs.nfs.commit_miss: 19404 vfs.nfs.realign_test: 401468 vfs.nfs.realign_count: 0 vfs.nfs.bufpackets: 0 vfs.nfs.gatherdelay: 0 vfs.nfs.gatherdelay_v3: 0 vfs.nfs.defect: 0 vfs.nfs.diskless_valid: 0 vfs.nfs.diskless_rootpath: vfs.nfs.diskless_swappath: vfs.nfs.access_cache_timeout: 2 vfs.nfs.nfsv3_commit_on_close: 0 ops-test.alexa.com guolin 134% nfsstat -c Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 193672 38 95314 53 73 24454 17 32 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 16 0 0 0 2844 0 2073 Mknod Fsstat Fsinfo PathConf Commit GLease Vacate Evict 0 1994 560 0 1505 0 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 1 693 322645 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 103198 195218 97904 95314 691 73 3931 24454 BioRLHits Misses BioD Hits Misses DirE Hits Misses 96444 53 620 2844 613 0 ops-test.alexa.com guolin 135% !!!!!!!!!!!!!!!!!! why there are so many misses? for nfs client cashes? !!!!!!!!!!!!!!!!!! ops-test2.alexa.com guolin 141% nfsstat -s Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 5 19 2 0 73248 1 1 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 1 1 0 2 0 29 Mknod Fsstat Fsinfo PathConf Commit GLease Vacate Evict 0 3 1 0 4388 0 0 0 Server Ret-Failed 6 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 1163 Server Lease Stats: Leases PeakL GLeases 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 73248 73248 0 ops-test2.alexa.com guolin 142% !!!!!!!!!!!!! why the value of WriteOps and WriteRPC are equal? !!!!!!!!!!!!!!!!!!!!!!!!!! Thanks. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message