From owner-freebsd-fs@FreeBSD.ORG Fri Oct 3 22:10:24 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BE5348E for ; Fri, 3 Oct 2014 22:10:24 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 52FBDDE7 for ; Fri, 3 Oct 2014 22:10:23 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqwEAJ4dL1SDaFve/2dsb2JhbABgDoNTWASCfsgECoZ5VAKBJAF7hAMBAQEDAQEBASArIAsFFhgCAg0ZAikBCSYGCAcEARwEiBUIDakblgoBF4EsjjEBARs0B4J3gVQFljCCQYFJhGyUGIFxGYEZXCEvB4EIOYECAQEB X-IronPort-AV: E=Sophos;i="5.04,650,1406606400"; d="scan'208";a="159433771" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 03 Oct 2014 18:10:16 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D5608B409E; Fri, 3 Oct 2014 18:10:16 -0400 (EDT) Date: Fri, 3 Oct 2014 18:10:16 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1265026957.57440085.1412374216862.JavaMail.root@uoguelph.ca> In-Reply-To: <21551.6675.236161.476053@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Oct 2014 22:10:24 -0000 Garrett Wollman wrote: > I've been doing some NFS testing, and found some issues with the NFS > client in 9.3. This is pretty easily reproducible with bonnie++, but > I haven't tried to reduce the size of the workload to make it easier > to test or trace. > > ##### NFSv3 mount, 9.3 client from 9.3 server > # bonnie++ -s 0 -n 1024:65536:0:1023:16384 -u 4294967294 -D > Using uid:-2, gid:65533. > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...Bonnie: drastic I/O error (rmdir): > Directory not empty > Cleaning up test directory after error. > > Identical behavior is observed for a 9.3 client talking to a 9.1 > server, and for NFSv4 mounts, but NOT for an Ubuntu 14.04 client > talking to the 9.3 server using either NFSv3 or NFSv4. > A couple of things to try: 1 - I suspect you are using TCP, but you might want to check this. 2 - Try an "oldnfs" mount and see if the old client exhibits the same behaviour. 3 - If you could disable the "Cleaning up test..." after the failure, it would be nice to check and see if "ls" on the directory shows any files. Then, unmount/remount it and do the "ls" again. 4 - Capture the packets for wireshark. If you can just capture the "Delete files..." it would make the packet capture a lot smaller. rick ps: There is one difference between the old and new NFS client's readdir code that I am aware of. It was inherited from OpenBSD and I left it in, but it can be safely removed for FreeBSD. I have no reason to believe it would cause a problem, but I can easily generate a patch for you to test, if the "oldnfs" mount doesn't exhibit the problem. > I haven't tried using a 9.2 or 9.1 client (yes, I still have some > machines running 9.1 here) to see if the issue is a recently > introduced one, nor have I traced the client or server code to see > what's actually going on. > > Has anyone seen this before? > > -GAWollman > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >