Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 19 Feb 2009 17:26:02 -0700
From:      Scott Long <scottl@samsco.org>
To:        John Baldwin <jhb@FreeBSD.org>
Cc:        svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org
Subject:   Re: svn commit: r188833 - in head/sys: kern nfsclient sys
Message-ID:  <499DF89A.2030703@samsco.org>
In-Reply-To: <200902191739.46808.jhb@freebsd.org>
References:  <200902192228.n1JMSn2Q009472@svn.freebsd.org> <200902191739.46808.jhb@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote:
> On Thursday 19 February 2009 5:28:49 pm John Baldwin wrote:
>> Author: jhb
>> Date: Thu Feb 19 22:28:48 2009
>> New Revision: 188833
>> URL: http://svn.freebsd.org/changeset/base/188833
>>
>> Log:
>>   Enable caching of negative pathname lookups in the NFS client.  To avoid
>>   stale entries, we save a copy of the directory's modification time when
>>   the first negative cache entry was added in the directory's NFS node.
>>   When a negative cache entry is hit during a pathname lookup, the parent
>>   directory's modification time is checked.  If it has changed, all of the
>>   negative cache entries for that parent are purged and the lookup falls
>>   back to using the RPC.  This required adding a new cache_purge_negative()
>>   method to the name cache to purge only negative cache entries for a given
>>   directory.
>>   
>>   Submitted by:	mohans, Rick Macklem, Ricardo Labiaga @ NetApp
>>   Reviewed by:	mohans
> 
> Together with the previous two changes I measured a 30% drop in the number of
> RPCs for a kernel build (no modules):
> 
> x nfs.old
> + nfs.new
> +------------------------------------------------------------------------------+
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          xx|
> | +                                                                          xx|
> | +                                                                          xx|
> |++                                                                          xx|
> |++                                                                          xx|
> |+++                                                                         xx|
> |+++                                                                         xx|
> |+++                                                                         xx|
> |+++                                                                        xxx|
> | A                                                                          A||
> +------------------------------------------------------------------------------+
>     N           Min           Max        Median           Avg        Stddev
> x  30        250515        252053        251393     251352.13     362.86492
> +  30        173990        175930        175117     175011.87     474.44092
> Difference at 95.0% confidence
>         -76340.3 +/- 218.321
>         -30.3718% +/- 0.0868584%
>         (Student's t, pooled s = 422.354)
> 
> The majority of the drop was in the number of LOOKUP RPCs performed:
> 
> x nfs.old
> + nfs.new
> +------------------------------------------------------------------------------+
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          x |
> | +                                                                          xx|
> | +                                                                          xx|
> | +                                                                          xx|
> | +                                                                          xx|
> |+++                                                                         xx|
> |+++                                                                         xx|
> |+++                                                                         xx|
> |+++                                                                         xx|
> |+++                                                                        xxx|
> | A|                                                                         A||
> +------------------------------------------------------------------------------+
>     N           Min           Max        Median           Avg        Stddev
> x  30        116997        118613        117898     117887.47     372.34733
> +  30         40791         42782         41975       41846.5     481.79805
> Difference at 95.0% confidence
>         -76041 +/- 222.565
>         -64.503% +/- 0.188794%
>         (Student's t, pooled s = 430.565)
> 
> The old client performed 0 GETATTR RPC calls during the test.  The new client
> performs a mix of GETATTR and ACCESS calls.  One of the worries was that the
> client might now perform more RPC calls due to not priming the access cache,
> but this is not true in this test.  This next histogram is for combined totals
> of ACCESS and GETATTR calls:
> 
> x nfs.attr.old
> + nfs.attr.new
> +------------------------------------------------------------------------------+
> |      + +                                                         x  x        |
> |    + + + + +                                                     x xxx  xx   |
> |    +++++++++ +                                                 x x xxx  xx   |
> |+  ++++++++++ +                                          +     xxxx xxxxxxxxxx|
> ||_______M_A________|                                              |__MA__|    |
> +------------------------------------------------------------------------------+
>     N           Min           Max        Median           Avg        Stddev
> x  30         90823         90898         90858     90859.633     19.496212
> +  30         90494         90792         90537     90543.467     50.121737
> Difference at 95.0% confidence
>         -316.167 +/- 19.6573
>         -0.347973% +/- 0.0216348%
>         (Student's t, pooled s = 38.0282)
> 
> With the new client, of the ~90k ACCESS + GETATTR calls are now split into
> approximately 94% GETATTR and 6% ACCESS versus 0% GETATTR and 100% ACCESS.
> 
> All of these changes are more the product of other folks who deserve the credit
> moreso than myself.
> 

These graphs are fun, but are useless without more information on the 
characteristics of the server and the NFS connections.

Scott




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?499DF89A.2030703>