From owner-freebsd-current@FreeBSD.ORG Sat Jun 29 23:12:27 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E3031F32 for ; Sat, 29 Jun 2013 23:12:27 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id AABCC1937 for ; Sat, 29 Jun 2013 23:12:27 +0000 (UTC) X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=u+Bwc9JL7tMNtl/i9xObSTPSFclN5AOtXcIZY5dPsHA= c=1 sm=2 a=ctSXsGKhotwA:10 a=FKkrIqjQGGEA:10 a=vEQvWETbpu8A:10 a=IkcTkHD0fZMA:10 a=6I5d2MoRAAAA:8 a=SSXDxoJpruQyTNpCM6UA:9 a=QEXdDO2ut3YA:10 a=SV7veod9ZcQA:10 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEABJoz1GDaFve/2dsb2JhbABZFoMlSYMHvAyBFHSCIwEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBGQMEh2gGDKl1kFOBJo0GfjQHglGBFgOVA4NukByDLSAygQM3 X-IronPort-AV: E=Sophos;i="4.87,967,1363147200"; d="scan'208";a="37889922" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 29 Jun 2013 19:12:25 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EF82979776; Sat, 29 Jun 2013 19:12:25 -0400 (EDT) Date: Sat, 29 Jun 2013 19:12:25 -0400 (EDT) From: Rick Macklem To: Lars Eggert Message-ID: <622716002.319507.1372547545943.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: NFSv4 console messages (locks lost etc.) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jun 2013 23:12:28 -0000 Lars Eggert wrote: > Hi, > > I should have mentioned that the server is FreeBSD -STABLE running > newnfs, and the network isn't partitioned (because I access the box > over SSH at the same time I see these messages.) > > They only appear under heavy NFS load (portmaster build of math/R in > this case.) > All I can think of that the server gets overloaded to the point where it doesn't process Renews within a reasonable timeframe. One thing you could try is the drc4 patch which wollman@ has had pretty good luck with for reducing CPU overhead and mutex contention in the server. The patch can be found at: http://people.freebsd.org/~rmacklem/drc4.patch Someday I'll get together with ivoras@ and come up with a version of this for head, rick > Lars > > On Jun 29, 2013, at 2:32, Rick Macklem wrote: > > > Lars Eggert wrote: > >> Hi, > >> > >> on a -CURRENT client, I get quite a number of console messages > >> under > >> heavy NFSv4 load, such as: > >> > >> nfsv4 expired locks lost > > Means the lease expired on the NFSv4 server somehow. Lease > > expiry is "bad news" and there is no way to recover locks > > lost because of it. > >> nfscl: never fnd open > > Usually, opens can be recovered after a lease expiry, but it > > might be broken. Since lease expiry should never happen during > > normal operation (see below), it doesn't get a lot of testing. > > > >> nfscl: never fnd open > >> nfscl: never fnd open > >> nfsv4 expired locks lost > >> nfscl: never fnd open > >> nfscl: never fnd open > >> nfsv4 expired locks lost > >> nfsv4 expired locks lost > >> nfsv4 expired locks lost > >> nfsv4 expired locks lost > >> nfsv4 expired locks lost > >> nfscl: never fnd open > >> > >> Can I ignore them? Can I turn them off? > >> > > Well, these should never happen during normal, correct operation. > > The > > "nfsv4 expired locks lost" implies lease expiry. This should only > > happen > > when the client is network partitioned from the server for more > > than > > a lease duration (chosen by the server, but typically about > > 1minute). > > The client does a Renew Op every 1/2 lease durations to avoid this. > > Also, > > any state related operation (open/lock/locku/close/etc) is supposed > > to > > renew the lease implicitly. > > > > If you are getting network partitions happening, then you really > > need > > to fix the network. > > > > If not, then if you watch network traffic with something like > > wireshark > > and see Renew Ops happening at regular intervals, then I can only > > suggest > > that the server is somehow broken for NFSv4. You should also look > > for > > NFS4ERR_EXPIRED error replies to operations related to state > > (open/lock/locku/close). > > That is the server reply which indicates the lease expiry. If the > > server is > > never returning this, I have no idea how the client would generate > > the above > > messages, but it does indicate a client NFSv4 bug if that is the > > case. > > > > Switching all mounts to NFSv3 will get rid of the above, although > > it is > > not exactly a fix;-) > > > > rick > > > >> Thanks, > >> Lars > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to > >> "freebsd-current-unsubscribe@freebsd.org" > >> > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" >