From owner-freebsd-current@FreeBSD.ORG Wed Jun 17 15:33:35 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AA5E106566C for ; Wed, 17 Jun 2009 15:33:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id BF35C8FC1A for ; Wed, 17 Jun 2009 15:33:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAMepOEqDaFvI/2dsb2JhbADVc4QIBQ X-IronPort-AV: E=Sophos;i="4.42,236,1243828800"; d="scan'208";a="38648054" Received: from darling.cs.uoguelph.ca ([131.104.91.200]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 17 Jun 2009 11:33:33 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id A38AE94006C; Wed, 17 Jun 2009 11:33:33 -0400 (EDT) X-Virus-Scanned: amavisd-new at darling.cs.uoguelph.ca Received: from darling.cs.uoguelph.ca ([127.0.0.1]) by localhost (darling.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zwDmV65xW-Ym; Wed, 17 Jun 2009 11:33:32 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id 725CD940062; Wed, 17 Jun 2009 11:33:31 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n5HFZJS14076; Wed, 17 Jun 2009 11:35:19 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Wed, 17 Jun 2009 11:35:19 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Martin In-Reply-To: <20090616215803.4a3aa748@zelda.local> Message-ID: References: <4A2504AA.1020406@zedat.fu-berlin.de> <20090603235227.GB15659@hades.panopticon> <20090615173315.1cdb39e1@zelda.local> <20090616000758.714912e6@zelda.local> <20090616215803.4a3aa748@zelda.local> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org Subject: Re: RPCPROG_MNT: RPC: Timed out / receiving NFS error when trying to mount NFS file system after make world X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2009 15:33:35 -0000 On Tue, 16 Jun 2009, Martin wrote: [assorted bits deleted for brevity] >> If you could post to -current again with the last version that worked >> vs doesn't work now, hopefully someone will spot the problem, rick > > I've looked up the date that is encoded in my kernel.old build. > FreeBSD 8.0-CURRENT #0: Fri May 15 10:57:37 CEST 2009 > > I'm sure it worked at this point. Also nfs filesystems that have > been already mounted survived the change, but when I started to > remount them, the problems have begun to appear, so I suppose that nfsd > itself is functioning well, but mountd or rpcbind are somehow affected. > Yep, mountd only gets involved at mount time. (It does suggest that the /etc/exports stuff is ok, since mountd does push that down into the kernel and it gets used by nfsd.) > The version I'm using now, that shows the issue, is of Jun 14 19:50 > CET. > > > Fortunatelly there are not many changes in libc/rpc in this > time interval. > One that might be worth trying is a pre-r192913 svc_dg.c. I'll email you a copy of that, in case you don't have an easy way to get one. rick ps: This is related to Martin's problem posted recently, as follows: # ls -l /lib/libc.so* -r--r--r-- 1 root wheel 1234432 Jun 15 01:04 /lib/libc.so.7 # ls -l /boot/kernel/kernel -r-xr-xr-x 1 root wheel 12010608 Jun 15 01:38 /boot/kernel/kernel Further info to diagnose the problem: I've got 3 NICs on 35, 135 and 235 (different subnets). Netmasks are: - xx.xx.xx.35/25 - xx.xx.xx.135/26 - xx.xx.xx.235/26 Client is xx.xx.xx.150/26. But you can try it with a single server. A mount on 127.0.0.1 won't work either. Here relevant part of rc.conf: nfs_server_flags="-t -n 4 -h xx.xx.xx.35 -h xx.xx.xx.135" mountd_flags="-l -r -h xx.xx.xx.35 -h xx.xx.xx.135" /etc/exports: /usr/export/src -maproot=root -ro -network xx.xx.xx.128 -mask 255.255.255.192 And here what I changed (rpcbind_flags) and the effects. Notice, I executed rpcinfo each time before restarting nfsd and mountd. Maybe I don't understand the rpcinfo output, because it differs from sockstat. ------------------------------ rpcbind_flags="-h xx.xx.xx.35 -h xx.xx.xx.135" # rpcinfo program version netid address service owner 100000 4 tcp xx.xx.xx.35.0.111 rpcbind superuser 100000 3 tcp xx.xx.xx.35.0.111 rpcbind superuser 100000 2 tcp xx.xx.xx.35.0.111 rpcbind superuser 100000 4 udp xx.xx.xx.35.0.111 rpcbind superuser 100000 3 udp xx.xx.xx.35.0.111 rpcbind superuser 100000 2 udp xx.xx.xx.35.0.111 rpcbind superuser 100000 4 tcp6 ::1.0.111 rpcbind superuser 100000 3 tcp6 ::1.0.111 rpcbind superuser 100000 4 udp6 ::1.0.111 rpcbind superuser 100000 3 udp6 ::1.0.111 rpcbind superuser 100000 4 local /var/run/rpcbind.sock rpcbind superuser 100000 3 local /var/run/rpcbind.sock rpcbind superuser 100000 2 local /var/run/rpcbind.sock rpcbind superuser # sockstat | grep rpcbind root rpcbind 28763 4 udp6 *:* *:* root rpcbind 28763 5 stream /var/run/rpcbind.sock root rpcbind 28763 6 udp6 ::1:111 *:* root rpcbind 28763 7 udp6 *:1008 *:* root rpcbind 28763 8 tcp6 ::1:111 *:* root rpcbind 28763 9 udp4 127.0.0.1:111 *:* root rpcbind 28763 10 udp4 xx.xx.xx.135:111 *:* root rpcbind 28763 11 udp4 xx.xx.xx.35:111 *:* root rpcbind 28763 12 udp4 *:842 *:* root rpcbind 28763 13 tcp4 127.0.0.1:111 *:* root rpcbind 28763 14 tcp4 xx.xx.xx.135:111 *:* root rpcbind 28763 15 tcp4 xx.xx.xx.35:111 *:* client# mount_nfs -o ro,tcp,intr,soft,bg,nfsv3 xx.xx.xx.35:/usr/export/src /usr/src [tcp] xx.xx.xx.35:/usr/export/src: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out mount_nfs: Cannot immediately mount xx.xx.xx.35:/usr/export/src, backgrounding -------------------------------------- rpcbind_flags="-h xx.xx.xx.135 -h xx.xx.xx.35" # rpcinfo program version netid address service owner 100000 4 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 3 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 2 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 4 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 3 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 2 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 4 tcp6 ::1.0.111 rpcbind superuser 100000 3 tcp6 ::1.0.111 rpcbind superuser 100000 4 udp6 ::1.0.111 rpcbind superuser 100000 3 udp6 ::1.0.111 rpcbind superuser 100000 4 local /var/run/rpcbind.sock rpcbind superuser 100000 3 local /var/run/rpcbind.sock rpcbind superuser 100000 2 local /var/run/rpcbind.sock rpcbind superuser # sockstat | grep rpcbind root rpcbind 28591 4 udp6 *:* *:* root rpcbind 28591 5 stream /var/run/rpcbind.sock root rpcbind 28591 6 udp6 ::1:111 *:* root rpcbind 28591 7 udp6 *:825 *:* root rpcbind 28591 8 tcp6 ::1:111 *:* root rpcbind 28591 9 udp4 127.0.0.1:111 *:* root rpcbind 28591 10 udp4 xx.xx.xx.35:111 *:* root rpcbind 28591 11 udp4 xx.xx.xx.135:111 *:* root rpcbind 28591 12 udp4 *:1009 *:* root rpcbind 28591 13 tcp4 127.0.0.1:111 *:* root rpcbind 28591 14 tcp4 xx.xx.xx.35:111 *:* root rpcbind 28591 15 tcp4 xx.xx.xx.135:111 *:* client# mount_nfs -o ro,tcp,intr,soft,bg,nfsv3 xx.xx.xx.35:/usr/export/src /usr/src [tcp] xx.xx.xx.35:/usr/export/src: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out mount_nfs: Cannot immediately mount xx.xx.xx.35:/usr/export/src, backgrounding -------------------------------------- rpcbind_flags="-h xx.xx.xx.135 -h xx.xx.xx.35 -h xx.xx.xx.235" # rpcinfo program version netid address service owner 100000 4 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 3 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 2 tcp xx.xx.xx.135.0.111 rpcbind superuser 100000 4 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 3 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 2 udp xx.xx.xx.135.0.111 rpcbind superuser 100000 4 tcp6 ::1.0.111 rpcbind superuser 100000 3 tcp6 ::1.0.111 rpcbind superuser 100000 4 udp6 ::1.0.111 rpcbind superuser 100000 3 udp6 ::1.0.111 rpcbind superuser 100000 4 local /var/run/rpcbind.sock rpcbind superuser 100000 3 local /var/run/rpcbind.sock rpcbind superuser 100000 2 local /var/run/rpcbind.sock rpcbind superuser # sockstat |grep rpcbind root rpcbind 28564 4 udp6 *:* *:* root rpcbind 28564 5 stream /var/run/rpcbind.sock root rpcbind 28564 6 udp6 ::1:111 *:* root rpcbind 28564 7 udp6 *:892 *:* root rpcbind 28564 8 tcp6 ::1:111 *:* root rpcbind 28564 9 udp4 127.0.0.1:111 *:* root rpcbind 28564 10 udp4 xx.xx.xx.235:111 *:* root rpcbind 28564 11 udp4 xx.xx.xx.35:111 *:* root rpcbind 28564 12 udp4 xx.xx.xx.135:111 *:* root rpcbind 28564 13 udp4 *:630 *:* root rpcbind 28564 14 tcp4 127.0.0.1:111 *:* root rpcbind 28564 15 tcp4 xx.xx.xx.235:111 *:* root rpcbind 28564 16 tcp4 xx.xx.xx.35:111 *:* root rpcbind 28564 17 tcp4 xx.xx.xx.135:111 *:* client# mount_nfs -o ro,tcp,intr,soft,bg,nfsv3 xx.xx.xx.35:/usr/export/src /usr/src [tcp] xx.xx.xx.35:/usr/export/src: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out mount_nfs: Cannot immediately mount xx.xx.xx.35:/usr/export/src, backgrounding -------------------------------------- rpcbind_flags="" # rpcinfo program version netid address service owner 100000 4 tcp 0.0.0.0.0.111 rpcbind superuser 100000 3 tcp 0.0.0.0.0.111 rpcbind superuser 100000 2 tcp 0.0.0.0.0.111 rpcbind superuser 100000 4 udp 0.0.0.0.0.111 rpcbind superuser 100000 3 udp 0.0.0.0.0.111 rpcbind superuser 100000 2 udp 0.0.0.0.0.111 rpcbind superuser 100000 4 tcp6 ::.0.111 rpcbind superuser 100000 3 tcp6 ::.0.111 rpcbind superuser 100000 4 udp6 ::.0.111 rpcbind superuser 100000 3 udp6 ::.0.111 rpcbind superuser 100000 4 local /var/run/rpcbind.sock rpcbind superuser 100000 3 local /var/run/rpcbind.sock rpcbind superuser 100000 2 local /var/run/rpcbind.sock rpcbind superuser # sockstat | grep rpcbind root rpcbind 28735 4 udp6 *:* *:* root rpcbind 28735 5 stream /var/run/rpcbind.sock root rpcbind 28735 6 udp6 *:111 *:* root rpcbind 28735 7 udp6 *:718 *:* root rpcbind 28735 8 tcp6 *:111 *:* root rpcbind 28735 9 udp4 *:111 *:* root rpcbind 28735 10 udp4 *:870 *:* root rpcbind 28735 11 tcp4 *:111 *:* client# mount_nfs -o ro,tcp,intr,soft,bg,nfsv3 xx.xx.xx.35:/usr/export/src /usr/src client# umount /usr/src umount: xx.xx.xx.35: RPCMNT_UMOUNT: RPC: Timed out --------------------------------- Hmm... the mount has been successful, but I wonder why umount still gets a time out... Need more info?