Date: Fri, 4 Jul 1997 11:37:20 -0400 (EDT) From: Bill Paul <wpaul@skynet.ctr.columbia.edu> To: lederer@bonn-online.com (Sebastian Lederer) Cc: hackers@freebsd.org Subject: Re: NFS locking, was: Re: NFS V3 is it stable? Message-ID: <199707041537.LAA19945@skynet.ctr.columbia.edu> In-Reply-To: <33BCA373.41C67EA6@bonn-online.com> from "Sebastian Lederer" at Jul 4, 97 09:17:07 am
next in thread | previous in thread | raw e-mail | index | archive | help
Of all the gin joints in all the towns in all the world, Sebastian Lederer had to walk into mine and say: > > > Mmmnnn... I have to differ with you here. With SGI IRIX 6.2, the NFS > > lock manager is in fact implemented inside the kernel. There is a > > /usr/etc/rpc.lockd, but it's just a small program that launches the > > lock manager as a kernel-space process. Actually, it may really be a > > kernel-space thread: no lock manager process shows up when you do a > > 'ps -ef' however the 'nlockmgr' service is registered with the portmapper. > > (It's registered on port 2049 too, which is significant because that's > > where nfs is registered too.) There is no llockmgr service registered > > anymore since it's no longer required (the kernel can calk to the > > lock manager process/thread directly instead of needing to use RPCs.) > > Right, one could also implement the lockd as a kernel process, just > like the nfs server. But even then there would be still an nlockmgr rpc > service > running on the nfs client, as you have pointed out. The nlockmgr > protocol > cannot be handled just by the client code. > > It's interesting that the nlockmgr port is the same as the nfs port, > that would probably mean that the lock manager has been combined with > the in-kernel nfs service (maybe that's what nfssys(LM_SVC) is for, > but at least on Sun, nfssys is undocumented, so I'm only guessing) I can tell you at least one thing nfssys() does: if you look at the keylogin(1) and chkey(1) source code from the tirpcsrc-2.3 distribution (on playground.sun.com:/pub/rpc, among other places), you'll see that it's used for revoking Secure RPC credentials for use with 'Secure NFS' (nfssys(NFS_REVAUTH, &foo)). What I don't know is how Secure RPC credentials are established for secure NFS in the first place. I think the kernel may actually communicate with the keyserv daemon to do it, but I'm not certain. > It's also interesting that you mention a llockmgr protocol. Does that > mean there *was* a (user process) rpc.lockd on the nfs client > in earlier versions of IRIX, and that the llockmgr protocol was > used by the kernel to communicate with it? Yes: before IRIX 6.2 came out, previous versions of IRIX had both a user-space rpc.statd and rpc.lockd, and lockd contained both an nlockmgr and llockmgr service implementation. This is the usual way it's done on commercial UNIXes that have licensed Sun's NFS code: rpc.lockd processes communicate with each other over the network using the nlockmgr protocol, but each rpc.lockd communicates with the kernel on the local host using the llockmgr protocol. That's what the /usr/include/rpcsvc/klm_prot.x protocol is for. Here's how it works (I think): - A process on a client decides it wants to lock something so it uses the flock() or fcntl() system calls. - The kernel eventually works out that the filesystem on which the process is trying to establish the lock is an NFS filesystem. - The kernel uses the klm_prot (llockmgr) protocol to ask the local rpc.lockd to go out and talk to the rpc.lockd on the NFS server and ask it for a lock. - The rpc.lockd on the server gets the request, locks the file, then replies back to the client. - The rpc.lockd on the client gets the remote lockd's reply and replies back to the kernel to tell it what happened. In IRIX 6.2, the extra RPC between the kernel and the local rpc.lockd goes away since the kernel has the nlm_prot (nlockmgr) protocol already implemented inside it. > > I'm not certain if I like this approach or not; on the one hand, it > > makes the llockmgr protocol unnecessary, but on the other hand, if the > > lock manager kernel process/thread goes insane (as the NFS lock manager > > is wont to do, in my experience), there's no way to kill/restart it. > > Assuming that an in-kernel lockd implementation is much more difficult > than a user-process implementation, I would suggest that we should try > a user-process based implementation first. > If that works, maybe later we could move to a kernel process for > performance optimization. > But I think for now it's more important to have something working > than to have improved performance. > The overall performance of nfs locking is not / won't be > great anyway. What do you think? I think somebody should start working on a klm_prot implementation for FreeBSD so we can tie everything together. This implies writing the client-side NFS locking support first though. -Bill -- ============================================================================= -Bill Paul (212) 854-6020 | System Manager, Master of Unix-Fu Work: wpaul@ctr.columbia.edu | Center for Telecommunications Research Home: wpaul@skynet.ctr.columbia.edu | Columbia University, New York City ============================================================================= "It is not I who am crazy; it is I who am mad!" - Ren Hoek, "Space Madness" =============================================================================
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199707041537.LAA19945>