From owner-freebsd-net@FreeBSD.ORG Tue Apr 5 01:44:24 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3E3E106564A for ; Tue, 5 Apr 2011 01:44:24 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout021.mac.com (asmtpout021.mac.com [17.148.16.96]) by mx1.freebsd.org (Postfix) with ESMTP id AABFB8FC14 for ; Tue, 5 Apr 2011 01:44:22 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from cswiger1.apple.com ([17.209.4.71]) by asmtp021.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LJ500G9SOTX9W10@asmtp021.mac.com> for freebsd-net@freebsd.org; Mon, 04 Apr 2011 18:44:22 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15,1.0.148,0.0.0000 definitions=2011-04-05_01:2011-04-05, 2011-04-05, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1104040163 From: Chuck Swiger In-reply-to: <1359778820.2757108.1301963093210.JavaMail.root@erie.cs.uoguelph.ca> Date: Mon, 04 Apr 2011 18:44:21 -0700 Message-id: References: <1359778820.2757108.1301963093210.JavaMail.root@erie.cs.uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1084) Cc: "Marc G. Fournier" , freebsd-net@freebsd.org Subject: Re: 7-STABLE NFS: fatal: "select lock: Permission denied" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 01:44:24 -0000 Hi, Rick-- On Apr 4, 2011, at 5:24 PM, Rick Macklem wrote: >> On Apr 4, 2011, at 11:09 AM, Marc G. Fournier wrote: >> Be careful; multiple access from different processes even on a single >> host can still run into locking issues against NFS filesystems, or >> data corruption if locking isn't available. > > From what I know of the implementation, I would have to disagree with > this statement. When mounted with the "nolockd" option, file locking is > still done within the client, using the same kernel function that is used > for file locking for local file systems like UFS. I'm certainly willing to believe you. :-) "nolockd" option to mount_nfs sets NFSMNT_NOLOCKD. In nfs_advlock() sys/nfsclient/nfs_vnops.c: if ((VFSTONFS(vp->v_mount)->nm_flag & NFSMNT_NOLOCKD) != 0) { size = VTONFS(vp)->n_size; VOP_UNLOCK(vp, 0); error = lf_advlock(ap, &(vp->v_lockf), size); } else { if (nfs_advlock_p) error = nfs_advlock_p(ap); else error = ENOLCK; } lf_advlock() is kernel's lockf from kern/kern_lockf.c. Well, that's a lot better than most of the other implementations I've seen, frankly, which tend to simply return OK if you try to do a lock when rpc.lockd isn't available, and $DEITY help your data if multiple writes happen. >> You're most at risk with local delivery to an mbox-style INBOX; >> delivery to maildir-style INBOX is much safer even on NFS without locking. >> > > Yes, but I think you are assuming that whatever is putting the email in > the mbox (sendmail daemon or ???) is running on a different machine than > the one that the imapd daemon (or whatever is reading the email) is > running on? Yes. It's fairly common to scale up a mail infrastructure from one box handling both SMTP and IMAP (or POP) to a SMTP-only box writing to NFS-mounted user mailboxes, and have one or more dedicated reader boxes which only run IMAP/POP daemons which access that same NFS filesystem holding the user mailboxes. > In general, NFS mounting a mail spool can be problematic, since it will > normally result in file(s) being accessed from more than one machine. As > such, I believe your warning w.r.t. the "nolockd" option is approriate, > I'm just not convinced that this is broken for the case where > all processes (including all daemons) that access the file are running on > the same NFS client (unlikely, but possible). > > Also, I believe your other advise is very appropriate, such as configuring > a mail system to avoid using file locking primitives. If you can, anyway-- but maildir is becoming more commonly used with the growing popularity of Cyrus and Dovecot compared with UWash IMAP (which did mbox and mbx). Regards, -- -Chuck PS: Please accept my apologies if you felt I was throwing stones at FreeBSD's implementation of NFS locking, both because it does appear to be handling this better than other implementations I've checked, and because I believe quite a bit of this work was done by someone closely resembling you, Rick. :-) But I've been burned by NFS locking (mis)adventures in the past, and I hate to see people depend on it if they have alternatives....