From owner-freebsd-current@FreeBSD.ORG Tue Feb 12 22:47:22 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 505A1B74 for ; Tue, 12 Feb 2013 22:47:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E048D89B for ; Tue, 12 Feb 2013 22:47:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAJXFGlGDaFvO/2dsb2JhbABEhk66M3OCHwEBBAEjBFIbGAICDQUUAlkGExkCh3EGDK0tgkCQAYEjjCAQgQQMgg+BEwOIZooEgzqBHY82gySBSwkXHg X-IronPort-AV: E=Sophos;i="4.84,652,1355115600"; d="scan'208";a="16348757" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 12 Feb 2013 17:47:20 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 55885B419E; Tue, 12 Feb 2013 17:47:20 -0500 (EST) Date: Tue, 12 Feb 2013 17:47:20 -0500 (EST) From: Rick Macklem To: =?utf-8?Q?Elias_M=C3=A5rtenson?= Message-ID: <1115347632.2956091.1360709240326.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: Possible bug in NFSv4 with krb5p security? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2013 22:47:22 -0000 Elias Martenson wrote: > On 12 February 2013 23:20, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > > > > There is (in case you missed it on google): > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup > (Nothing much has changed since FreeBSD8, except the name of the > client > side patch for host based initiator credentials in the keytab file.) > I was hoping others would add/update the wiki and it would eventually > become FreeBSD doc, but that hasn't happened. > > > > > Thank you for the link. I have indeed found that, and I have followed > it to the letter. > > > I have up the exact same thing from Ubuntu machines as well as from > Solaris, and I do have a fairly good understanding of Kerberos. > FreeBSD however, is pretty new to me. > > > Other than that, the various discussions in the archive on this list > may help. Unfortunately, I don't know of an easy way to figure out > what is busted. I always suggest looking at the packets in wireshark, > but for some reason, I get the impression that folk don't like > doing this? It is what I do first when I run into NFS issues. > > > > I've been looking at the dumps using Wireshark. Well, I had to drop > down the security since everything is encrypted when using keb5p. I do > get the same errors using sec=krb5. > > > When looking at this trace, I see a normal OPEN request followed by a > NFS4ERR_ACCESS as a reply. The Kerberos credentials are of course > encrypted, so I can't really say anything about that part. > Well, it sounds like you are doing all the right stuff, so I don't know why it is returning EACCES? I'm not a ZFS person, so I never test that. If you have a UFS file system you could export for testing, that might be worth a try. ZFS likes to do things differently;-) You can look at the authentication stuff in the RPC header: Actually, the credentials in the RPC header aren't encrypted, although they are binary data. It's been a while since I looked at the RFC, but the authenticator is basically: - an RPCSEC_GSS version (must be 1 for FreeBSD to be able to use it) - a type that will be DATA in this case - a credential handle (just a blob of bits the server uses as a shorthand for the principal) - a sequence# used to subvert replay attempts Then the authentication verifier is an encrypted checksum of the above, that the server uses to verify it. All the Kerberos stuff happens via NFS Null RPCs, where the GSSAPI tokens are passed as data (a Null RPC has no arguments) and the credential handle and a session key get established. (The Kerberos ticket is inside the GSSAPI token for the first of these Null RPCs.) > > Note that NFS4 with Kerberos security never uses the user ID numbers. > They purely use the Kerberos principals for authorisation. This is > different from the default "sys" security model that blindly uses user > ID's. > Yep, of course. The Kerberos user principal "name@REALM" is translated to a uid + gid list by the gssd via a lookup of "name" as the username. The uid + gid list is then associated with that credential handle I mentioned above. > > > > > nfscbd_enable="YES" > Needed for client side only, and only if delegations are > enabled at the server and you want the client to acquire > delegations. (Delegations are not enabled by default on the > FreeBSD NFSv4 server.) > > > > Noted. Thank you. I will change this. > > > > > rpc_lockd_enable="YES" > > rpc_statd_enable="YES" > You shouldn't need rpc_lockd or rpc_statd for NFSv4, > since they are only used for NFSv3. > > > > Good point. I'll disable those too. > > > > I don't know why a Linux client would mix NFSv3 RPCs with NFSv4 ones, > > > I was suggested to set vfs.nfsd.server_min_nfsvers to 4 in order to > completely disable NFS version below 4. I did this and got rid of the > stray NFS3 requests. It didn't solve the original problem though. > > > > > If anyone is able to confirm whether or not this actually has been > > tested > > in 9.1-CURRENT, I'd appreciate it. Also, if not, then I'd love to > > know > > where I should start looking for a solution. I'm experienced in > > system > > level programming (having worked on Solaris at Sun in a previous > > life), but > > a pointer where to start would be helpful. > > > Usually, when everything is being done by "nobody", it indicates that > the mapping between <-> name@domain isn't working > correctly. > (Looking at the packets in wireshark, you need to look at the > attributes > called Owner and Owner_Group to see what they are being set to.) > > > > I actually doubt this. First of all, I have the correct idmapd setup > working from FreeBSD to Ubuntu (I can see that since I can see the > correct user names in "ls" even though the user ID's differ). On OSX I > haven't got it to work yet. > > > But, the behaviour is the same on both systems. > > > This is actually expected, as the permission checks are orthogonal to > the ID mapping. > > > The most common problem (since you do have nfsuserd running on the > server) > is for the "domain" spec to be different between client and server. > FreeBSD's nfsuserd defaults to the domain part of the machine's > hostname. > Linux's rpc.idmapd sets the domain from /etc/idmapd.conf (at least I > think > that's what it is called) and many distros ship with it set to > "my.domain" > or something like that. > > > > Correct. I have set this correctly. I know this, since once I did, > "ls" started giving me the correct user names. > > > As such, I'd start by checking the Linux client and seeing what it has > for > the domain spec. in /etc/idmapd.conf. > > If you want to override the default for FreeBSD, there is a command > line > option for nfsuserd to do this. I can't remember what it is, but "man > nfsuserd" > will give you the answer and it can be set in /etc/rc.conf using > nfsuserd_flags. > > > > Thank you. I'm definitely willing to double-check this and I am not > going to claim to know exactly what's going on in the realms of NFS > security. :-) > > > If this is configured correctly, then looking at the packets in > wireshark is > the best starting point w.r.t. figuring out what is broken. I do > limited > testing of this and it works for me. I don't know how many others use > it, > although some definitely have fun getting it working. (Usually it is > the > Kerberos part on the client side that causes the most grief.) > > > > It certainly is fun. But it gets frustrating when one fights a single > problem for weeks on end. > > > Far too few shops use Kerberos though. > Or maybe too many;-) A public key system might not be as secure, but it would be far easier to manage imho. > > > Regards, > Elias