From owner-freebsd-stable@freebsd.org Sat Aug 29 23:34:49 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B8E69C65BB for ; Sat, 29 Aug 2015 23:34:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3020B9A7 for ; Sat, 29 Aug 2015 23:34:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:uFF7hxJNcDOSWuGpjtmcpTZWNBhigK39O0sv0rFitYgUIvnxwZ3uMQTl6Ol3ixeRBMOAu6kC1bqd6vC8EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35/xibD5os2PKyxzxxODIppKZC2sqgvQssREyaBDEY0WjiXzn31TZu5NznlpL1/A1zz158O34YIxu38I46Fp34d6XK77Z6U1S6BDRHRjajhtpZ6jiR6WByWV730fGmkb2jBPH0CF6BDhQoyw+GOusutmxDKyJsD/V7s0SHKk4rs9GzHyjyJSDT8y8ynyg8dziK9e6Ea7ohV0wIrZZamIM/Vjc6fFfZURTDwSDY5qSyVdD9bkPMM0BO0bMLMd9tGlqg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AnAgCYQOJV/61jaINeg29pBoMdumgBCYFuCoUxSgKBWxQBAQEBAQEBAYEJgh2CBgEBAQMBAQEBICsgCwULAgEIGAICDRkCAicBCSYCBAgHBAEcBIgFCA2oVpRCAQEBAQEBAQECAQEBAQEBAQEWBIEihU6Ee4Q5AQEFFzQHgmmBQwWVQYUHhQuELJU0g2oCJoQbIjMHgQc6gQUBAQE X-IronPort-AV: E=Sophos;i="5.15,746,1432612800"; d="scan'208";a="233734536" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 29 Aug 2015 19:34:41 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A386F15F565; Sat, 29 Aug 2015 19:34:41 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MKea0l736_S2; Sat, 29 Aug 2015 19:34:40 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id E397415F56D; Sat, 29 Aug 2015 19:34:40 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id o42EAjSL6Gvk; Sat, 29 Aug 2015 19:34:40 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id C7A4715F565; Sat, 29 Aug 2015 19:34:40 -0400 (EDT) Date: Sat, 29 Aug 2015 19:34:40 -0400 (EDT) From: Rick Macklem To: "Frank de Bot (lists)" Cc: freebsd-stable@freebsd.org Message-ID: <569931445.35926615.1440891280752.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55E224BE.5000900@searchy.net> References: <55E224BE.5000900@searchy.net> Subject: Re: NFS Client changing it's source address? (FreeBSD 10.2) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: NFS Client changing it's source address? (FreeBSD 10.2) Thread-Index: 7mAhIv4+FuIA9DyYEV+ghUYfvB1jrw== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Aug 2015 23:34:49 -0000 Frank de Bot wrote: > Hello, > > I have a server running FreeBSD 10.2. It has several NFS mounts. > Frequently my NFS mount hang (v3). After a little investigation it looks > like FreeBSD has chosen a wrong source address for it's connections and > all packets are departing from the wrong interface. > > Sockstat output: > > [root@host004 ~]# sockstat -4 | grep 2049 > root ssh 14689 3 tcp4 10.4.2.4:20499 10.4.2.5:22 > ? ? ? ? tcp4 10.13.37.4:672 10.13.37.2:2049 > ? ? ? ? tcp4 79.x.x.210:905 10.13.37.2:2049 > ? ? ? ? tcp4 79.x.x.210:992 10.13.37.2:2049 > > tcpdump confirms nfs connection are trying to get out via the 79.x.x.x > interface > > My fstab for the nfs mounts look like: > > 10.13.37.2:/tank/hostingbase /opt/jails/hostingbase nfs > nfsv3,ro,noatime,async,noauto 0 0 > > /opt/jails/hostingbase /opt/jails/test01 nullfs > ro,noatime,noauto 0 0 > 10.13.37.2:/tank/hosting/test /opt/jails/test01/opt nfs > nfsv3,noatime,async,rw,noauto 0 0 > tmpfs /opt/jails/test01/shm tmpfs > rw,size=512000000,noauto 0 0 > > /opt/jails/hostingbase /opt/jails/test2 nullfs > ro,noatime,noauto 0 0 > 10.13.37.2:/tank/hosting/test2 /opt/jails/test2/opt nfs > nfsv3,noatime,async,rw,noauto 0 0 > tmpfs /opt/jails/test2/shm tmpfs > rw,size=512000000,noauto 0 0 > > > The change of source address looks to be happening after a nfs > connection is re-established. At first everything works, I leave the > server idling (it's a test server) and after that the mounts are hanging > If the client side of the kernel RPC needs a new TCP connection to the server, it will do a soconnect(). It will be done by whatever process/thread is doing the NFS I/O syscall at the time that the connection isn't available. I'd guess that a process/thread that is running in a jail that can't see the 10.13.37 network does this soconnect() and then it is broken. I am not conversant w.r.t. jails and I don't know of any clever way around this. A couple of "hackish" workarounds that might work: - Reconfigure your NFS server so that it never drops idle TCP connections. (FreeBSD never drops an NFS TCP connection until umount, but some others like Solaris NFS servers drop idle connections.) - Make your NFS server accessible through the 79.n.n network instead of 10.13.37. - Write a daemon that does a stat() of a file on the NFS mount once per minute, so the NFS connection is never idle long enough to be disconnected. rick > 10.2-RELEASE #0 r286666 is the current running version. > > Regards, > > Frank de Bot > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >