Skip site navigation (1)Skip section navigation (2) 
Date:      Sun, 30 May 2004 14:07:39 +0400
From:      Gleb Smirnoff <glebius@cell.sick.ru>
To:        freebsd-net@freebsd.org
Subject:   incorrect connect() behavior
Message-ID:  <20040530100739.GA58477@cell.sick.ru>
next in thread | raw e-mail | index | archive | help 

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline

  Dear networkers,

  there is a problem in connect() syscall, which can be reproduced
on a box running without default route.

According to POSIX, connect() must return if ENETUNREACH, if a route to
destination was not found.

http://www.opengroup.org/onlinepubs/000095399/functions/connect.html

  In case of SOCK_STREAM it works this way. But in case of SOCK_DGRAM connect()
does not return error. And it picks up first available local IP address for
local side of socket. In some cases this address may appear to be 127.0.0.1.
Later, when a route to destination shows up, datagrams will fail to send,
since 127.0.0.1 can not appear on wire.

Affected installations are:
 - BGP routers without default route
 - localnet routers running some IGP

Affected applications are:
 - ntpd. ntpd starts before routing daemon have established all adjacencies,
   connect() binds to 127.0.0.1. Later when routing show up, ntpd fails to
   send dgrams to server.
 - net-snmpd. It is difficult to reproduce, but after some route flapping
   snmpd hangs, and does not respond to requests. This can be workarounded with
   a static route to source of queries.
 - ng_ksocket. If node is of type inet/dgram/udp and a connect message is sent
   to it, it does not return an error. Later it fails to send packets with EPERM.

Here is attached a test case for this problem no-route-test.c. To test, one needs
to delete default route, compile no-route-test and run it. If connect() picks up
non-localhost address, then you are lucky :), some of your interfaces was
ifconfiged before lo0. To reproduce problem with 100 % guarantee, one needs
to have lo0 first one in list ${network_interfaces} var in /etc/rc.conf.
Then you should add default route, and look into what is typed by no-route-test,
which was started before this route was added.

I have written two patches to deal with this problem. The first one clings to
POSIX behavior - it returns ENETUNREACH. I have tested ntpd with it - it works well.
But there is no guarantee that anything else would be broken. The second patch is
a POLA-patch, it makes connect() to take first non-localhost address for local side
of socket. Code was obtained directly from NetBSD. This patch is considered not
to break anything. Both patches are attached.

I'd be happy if one of commiters fixes this problem. I'm really tired of routers with
lost time synchronisation. Thanks in advance.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=koi8-r
Content-Disposition: attachment; filename="no-route-test.c"

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <errno.h>

#define	SOMEHOST	"10.0.0.1"
#define MSG		"test"

int main() {
  int s, rtn;
  struct sockaddr_in saddr;
  int saddrlen = sizeof(saddr);
  

  saddr.sin_family = AF_INET;
  saddr.sin_addr.s_addr = inet_addr(SOMEHOST);
  saddr.sin_port = htons(2000);

  s = socket(AF_INET, SOCK_DGRAM, 0);
  if (s < 0) {
	printf("Error from socket()\n");
	return -1;
  }

  rtn = connect(s, (struct sockaddr *)&saddr, sizeof(saddr));
  if (rtn < 0) {
	printf("Error from connect(): %s\n", strerror(errno));
	return -1;
  }

  rtn = getsockname(s, (struct sockaddr *)&saddr, &saddrlen);
  if (rtn < 0) {
	printf("Error from getsockname(): %s\n", strerror(errno));
	return -1;
  }
  printf("Addr is %s\n", inet_ntoa(saddr.sin_addr));

  for (;;) {
	rtn = send(s, (void *)MSG, strlen(MSG), 0);
	if (rtn != strlen(MSG))
		printf("Error from send(): %s\n", strerror(errno));
	sleep(1);
  }

  close(s);
}

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=koi8-r
Content-Disposition: attachment; filename="in_pcb.c.connect.POLA.diff"

Index: in_pcb.c
===================================================================
RCS file: /home/ncvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.147
diff -u -r1.147 in_pcb.c
--- in_pcb.c	20 May 2004 06:35:02 -0000	1.147
+++ in_pcb.c	29 May 2004 21:58:55 -0000
@@ -611,8 +611,13 @@
 			ia = ifatoia(ifa_ifwithdstaddr(sintosa(&sa)));
 			if (ia == 0)
 				ia = ifatoia(ifa_ifwithnet(sintosa(&sa)));
-			if (ia == 0)
-				ia = TAILQ_FIRST(&in_ifaddrhead);
+			if (ia == 0) {
+				/* Find 1st non-loopback AF_INET address */
+				TAILQ_FOREACH(ia, &in_ifaddrhead, ia_link) {
+					if (!(ia->ia_ifp->if_flags & IFF_LOOPBACK))
+						break;
+				}
+			}
 			if (ia == 0)
 				return (EADDRNOTAVAIL);
 		}

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=koi8-r
Content-Disposition: attachment; filename="in_pcb.c.connect.POSIX.diff"

Index: in_pcb.c
===================================================================
RCS file: /home/ncvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.147
diff -u -r1.147 in_pcb.c
--- in_pcb.c	20 May 2004 06:35:02 -0000	1.147
+++ in_pcb.c	29 May 2004 21:12:40 -0000
@@ -612,9 +612,7 @@
 			if (ia == 0)
 				ia = ifatoia(ifa_ifwithnet(sintosa(&sa)));
 			if (ia == 0)
-				ia = TAILQ_FIRST(&in_ifaddrhead);
-			if (ia == 0)
-				return (EADDRNOTAVAIL);
+				return (ENETUNREACH);
 		}
 		/*
 		 * If the destination address is multicast and an outgoing

--C7zPtVaVf+AK4Oqc--

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040530100739.GA58477>