From owner-freebsd-current@FreeBSD.ORG Sun Jul 1 00:53:20 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ADED91065670 for ; Sun, 1 Jul 2012 00:53:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-scalar.mail.uoguelph.ca (esa-scalar.mail.uoguelph.ca [66.199.40.18]) by mx1.freebsd.org (Postfix) with ESMTP id 7CACE8FC14 for ; Sun, 1 Jul 2012 00:53:20 +0000 (UTC) Received: from zcs3.mail.uoguelph.ca (new.mail.uoguelph.ca [131.104.93.37]) by esa-scalar.mail.uoguelph.ca (8.14.1/8.14.1) with ESMTP id q610r9KU009659; Sat, 30 Jun 2012 20:53:09 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7FA8AB4064; Sat, 30 Jun 2012 20:53:09 -0400 (EDT) Date: Sat, 30 Jun 2012 20:53:09 -0400 (EDT) From: Rick Macklem To: Vincent Hoffman Message-ID: <68594395.2439924.1341103989486.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4FECAD2C.8070402@unsane.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-current@freebsd.org Subject: Re: Occassional "permission denied" in the middle of a large transfer over NFS X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 00:53:20 -0000 Vincent Hoffman wrote: > Just a note to say I have tested this on -CURRENT with the new nfs > server and it is still the case. > > On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3 > #0: Tue Jun 12 00:39:29 UTC 2012 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64) > > [root@seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar > /var/nfsen/profiles-data/ ; date > Thu Jun 28 20:12:36 BST 2012 > tar: Removing leading '/' from member names > tar: Write error > Thu Jun 28 20:12:38 BST 2012 > [root@seaurchin /mnt/nfs/vm]# > > > > on the Server > [root@fbsd /var/tmp]# uname -a ; date ; while true ; do mount /dev/md0 > /mnt/tmp/ ; umount /mnt/tmp ; done > FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun 27 > 19:13:26 BST 2012 > toor@fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm amd64 > Thu Jun 28 20:12:38 BST 2012 > ^C > [root@fbsd /var/tmp]# > > any suggestions welcome. > > Vince > > > On 27/06/2012 16:27, Vincent Hoffman wrote: > > Hi, > > After only one off-list reply from the author of kern/136865 (see > > below) > > after asking -questions, I thought it worth asking -CURRENT. > > Basically: > > > > I seem to have run into the problems described in this old thread. > > http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html > > > > tl:dr mountd may give incorrect permission denied errors when it is > > refreshing the exports list due to non-atomic operations, > > /sbin/mount has code that sends SIGHUP to > > mountd on any mount operation, which implies that any manual mount > > request would cause the problem. > > > > Currently I have still only tested on 8.3-RELEASE but the svn log > > doesnt > > seem to mention a fix since then. I'm currently taking a VM up to > > -CURRENT to test. > > > > Looking though old PRs I see the following related. > > kern/131342 > > kern/136865 (with patch for 7.2 and links to > > http://nfse.sourceforge.net/ for -CURRENT ) > > > > Does anyone who is qualified (sadly not me) feel like looking at the > > code to see if its suitable for inclusion in part/whole as not > > having > > NFS transfers interrupted by local mount operations on the nfs > > server > > would be very handy :) > > I haven't looked at Andrey's patch, but conceptually it sounds like the best approach. As I understand it, the problem with replacing mountd with nfse (at least in the FreeBSD source tree) is that nfse is not 100% backwards compatible with /etc/exports and, as such, is a POLA violation. To modify mountd to use the kernel changes is more work than I have time for, in part because mountd.c is a very ugly old piece of C code, imho. I do have a patch that suspends/resumes execution of the nfsd threads (new, experimental for FreeBSD8.n only) when mountd reloads /etc/exports. This approach has certain disadvantages vs Andrey's transactional load/commit model, but it is simple and might be sufficient for your problem. If you want to try this patch, it is at: http://people.freebsd.org/~rmacklem/atomic-export.patch (The patch affects both the kernel and mountd.c.) Also, you could easily hack mount.c so that it doesn't send a SIGHUP to mountd (which causes the exports to be reloaded) every time a local fs is mounted. rick > > > > thanks, Vince > > > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org"