Date: Mon, 3 Mar 2003 15:04:02 -0500 (EST) From: Daniel Ellard <ellard@eecs.harvard.edu> To: FreeBSD-gnats-submit@FreeBSD.org Cc: Alfred Perlstein <bright@mu.org>, Matt Dillon <dillon@FreeBSD.org> Subject: kern/48894: Improvements to the NFS read-ahead heuristic code Message-ID: <20030303200402.6846572CB@ganon.eecs.harvard.edu>
next in thread | raw e-mail | index | archive | help
>Number: 48894
>Category: kern
>Synopsis: Improvements to the NFS read-ahead heuristic code
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Mon Mar 03 12:10:10 PST 2003
>Closed-Date:
>Last-Modified:
>Originator: Daniel Ellard
>Release: FreeBSD 4.6.2-RELEASE-p9 i386
>Organization:
Harvard University
>Environment:
System: FreeBSD ant.eecs.harvard.edu 4.6.2-RELEASE-p9 FreeBSD 4.6.2-RELEASE-p9 #12: Mon Feb 24 10:51:51 EST 2003 ellard@ant.eecs.harvard.edu:/usr/obj/usr/freebsd_4.6/src/sys/ANT i386
>Description:
There are several potential improvements to the NFS read-ahead
heuristics. First, the nfsheur table size is sized too small for
large servers. Second, the sequentiality hueristic can be
modified to recognize and handle read patterns that have a large
sequential component (such as stride reads) that are missed by
the current sequentiality metric. Third, the sequentiality metric
is fragile in the face of jitter in the request stream (which happens
when client nfsiods are stressed).
>How-To-Repeat:
Beat on your NFS server
>Fix:
The first problem (nfsheur too small) is easy to fix, and I recommend
this be done for all systems:
1. In sys/nfs/nfs_serv.c: change NUM_HEURISTIC from 64 to
1021. The exact value isn't important, but it should
be at least a few hundred, and prime.
2. Change all the calculations of the nfsheur hash bucket location
from
(something) & (NUM_HEURISTIC - 1);
to
(something) % NUM_HEURISTIC;
This is necessary because if NUM_HEURISTIC isn't a power
of two, then we can't use bit-fiddling to do the %.
2. In function nfsrv_read, file sys/nfs/nfs_serv.c: change the
number of probes from 4 to something larger:
/*
* Calculate seqcount for heuristic
*/
{
int try = 4;
In my experiments, 8 works well, but YMMV.
The downside of this fix is that nfsheur uses more memory (approx 20K)
and if you have a slow CPU then doing the % instead of & will hurt.
But if you have a busy NFS server, the reduction in hash table
collisions will be worth the extra computation.
Fixing the second and third problems involves much more extensive
changes to the code. The ideas are described in the latter half of a
paper I am working on for FreeNIX (URL below). I have implemented
these changes in a new file, using sysctl to control the parameters
and whether or not the new code is used at all. A snapshot of my code
and the paper is available at:
http://www.eecs.harvard.edu/~ellard/NFS/
ellard-freenix03.pdf - paper snapshot, with description of
the new algorithms.
nfs_serv.c - modified nfs_serv.c
nfs_serv.c.diff - listing of the diffs from the original nfs_serv.c
nfs_tweak.c - new source code
nfs_tweak.h - new header file
Please contact me if you need more information, or have suggestions
for how I can improve my code.
Thanks,
-Dan
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030303200402.6846572CB>