Date: Mon, 04 Apr 2011 18:44:21 -0700 From: Chuck Swiger <cswiger@mac.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: "Marc G. Fournier" <scrappy@hub.org>, freebsd-net@freebsd.org Subject: Re: 7-STABLE NFS: fatal: "select lock: Permission denied" Message-ID: <AB2EA7D5-2EBF-44E2-BEC6-DDFB0772F5C2@mac.com> In-Reply-To: <1359778820.2757108.1301963093210.JavaMail.root@erie.cs.uoguelph.ca> References: <1359778820.2757108.1301963093210.JavaMail.root@erie.cs.uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Rick--
On Apr 4, 2011, at 5:24 PM, Rick Macklem wrote:
>> On Apr 4, 2011, at 11:09 AM, Marc G. Fournier wrote:
>> Be careful; multiple access from different processes even on a single
>> host can still run into locking issues against NFS filesystems, or
>> data corruption if locking isn't available.
>
> From what I know of the implementation, I would have to disagree with
> this statement. When mounted with the "nolockd" option, file locking is
> still done within the client, using the same kernel function that is used
> for file locking for local file systems like UFS.
I'm certainly willing to believe you. :-) "nolockd" option to mount_nfs sets NFSMNT_NOLOCKD. In nfs_advlock() sys/nfsclient/nfs_vnops.c:
if ((VFSTONFS(vp->v_mount)->nm_flag & NFSMNT_NOLOCKD) != 0) {
size = VTONFS(vp)->n_size;
VOP_UNLOCK(vp, 0);
error = lf_advlock(ap, &(vp->v_lockf), size);
} else {
if (nfs_advlock_p)
error = nfs_advlock_p(ap);
else
error = ENOLCK;
}
lf_advlock() is kernel's lockf from kern/kern_lockf.c. Well, that's a lot better than most of the other implementations I've seen, frankly, which tend to simply return OK if you try to do a lock when rpc.lockd isn't available, and $DEITY help your data if multiple writes happen.
>> You're most at risk with local delivery to an mbox-style INBOX;
>> delivery to maildir-style INBOX is much safer even on NFS without locking.
>>
>
> Yes, but I think you are assuming that whatever is putting the email in
> the mbox (sendmail daemon or ???) is running on a different machine than
> the one that the imapd daemon (or whatever is reading the email) is
> running on?
Yes. It's fairly common to scale up a mail infrastructure from one box handling both SMTP and IMAP (or POP) to a SMTP-only box writing to NFS-mounted user mailboxes, and have one or more dedicated reader boxes which only run IMAP/POP daemons which access that same NFS filesystem holding the user mailboxes.
> In general, NFS mounting a mail spool can be problematic, since it will
> normally result in file(s) being accessed from more than one machine. As
> such, I believe your warning w.r.t. the "nolockd" option is approriate,
> I'm just not convinced that this is broken for the case where
> all processes (including all daemons) that access the file are running on
> the same NFS client (unlikely, but possible).
>
> Also, I believe your other advise is very appropriate, such as configuring
> a mail system to avoid using file locking primitives.
If you can, anyway-- but maildir is becoming more commonly used with the growing popularity of Cyrus and Dovecot compared with UWash IMAP (which did mbox and mbx).
Regards,
--
-Chuck
PS: Please accept my apologies if you felt I was throwing stones at FreeBSD's implementation of NFS locking, both because it does appear to be handling this better than other implementations I've checked, and because I believe quite a bit of this work was done by someone closely resembling you, Rick. :-)
But I've been burned by NFS locking (mis)adventures in the past, and I hate to see people depend on it if they have alternatives....
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AB2EA7D5-2EBF-44E2-BEC6-DDFB0772F5C2>