Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 17 Oct 2011 12:50:44 -0700
From:      Devin Teske <devin.teske@fisglobal.com>
To:        "'Sergey Kandaurov'" <pluknet@gmail.com>
Cc:        'Dave Robison' <Dave.Robison@fisglobal.com>, 'FreeBSD amd64' <freebsd-amd64@freebsd.org>, devin.teske@fisglobal.com
Subject:   RE: 32-bit route(8) on amd64 host and jails
Message-ID:  <102d01cc8d06$0fd26a70$2f773f50$@fisglobal.com>
In-Reply-To: <CAE-mSO%2B_hc%2BGJHYUyX6dBS6NckV=iiHRrGMYbc=AxQeqKwK2Zg@mail.gmail.com>
References:  <714EF3C9-33B0-4EF5-B52C-1E95F7F432F9@fisglobal.com>	<CAE-mSO%2B4KKWeC6-Z3UAi70bxxEUJL5gUD0phEfNzzkDMXrA3pg@mail.gmail.com> <CAE-mSO%2B_hc%2BGJHYUyX6dBS6NckV=iiHRrGMYbc=AxQeqKwK2Zg@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
------=_NextPart_000_102E_01CC8CCB.63739270
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"

> -----Original Message-----
> From: Sergey Kandaurov [mailto:pluknet@gmail.com]
> Sent: Friday, October 14, 2011 2:40 PM
> To: Devin Teske
> Cc: FreeBSD amd64; Dave Robison
> Subject: Re: 32-bit route(8) on amd64 host and jails
>=20
> On 21 September 2011 16:47, Sergey Kandaurov <pluknet@gmail.com> wrote:
> > On 21 September 2011 06:40, Devin Teske <devin.teske@fisglobal.com> wro=
te:
> >> I'm noticing that a 32-bit route(8) shows strange behaviour while runn=
ing
> under an amd64 kernel (regardless of whether in the base-host -- utilizing
> /usr/lib32/libc.so.7 -- or in a jail and/or vimage -- 32-bit in nature;
results are
> same).
> >>
> >> Executable runs fine, but you can't (a) set the default route or (b) v=
iew
the
> default route (after successfully setting it with the amd64 build, of cou=
rse).
> >>
> >> ASIDE: This is under 8.1-RELEASE.
> >>
> >> When attempting to set the default route, you get the following...
> >>
> >> root@kps0a / # route add -net default 10.10.125.99
> >> route: writing to routing socket: Invalid argument add net default:
> >> gateway 10.10.125.99: Invalid argument
> >>
> >> Meanwhile, using the amd64 version, no issues.
> >>
> >> When attempting to view the default route, you get the following...
> >>
> >> root@kps0a / # route -n get default
> >> =A0 route to: default
> >> destination: default
> >> =A0 =A0 =A0 mask: default
> >> =A0 =A0gateway: default
> >> =A0 =A0 =A0flags: <UP,GATEWAY,DONE,STATIC>
> >> =A0recvpipe =A0sendpipe =A0ssthresh =A0rtt,msec =A0 =A0mtu =A0 =A0 =A0=
 =A0weight
> >> expire
> >> =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =
=A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0
> >> -1316570637
> >>
> >> It's the "gateway: default" that's out of place.
> >>
> >
> > Currently, FreeBSD has a poor shape of rtsocket freebsd32
> > compatibility, AFAIK only sysctl layer (sysctl_rtsock) is aware of it.
> > That means when a 32bit app writes some data to a routing socket,
> > kernel expects to receive it in the native ABI, and it doesn't work.
>=20
> Can you please try this preliminary patch and report back?
> You only need to rebuild kernel and reboot with the attached patch to ena=
ble
use
> of e.g. 32-bit route(8) command on amd64 kernel.
> No any userland utility needs to rebuild to enable the compatibility.
> Tested on 8.2 amd64 only with route get default and route delete/add.
>=20

First-off, let me say thank you. The patch works as-expected (for amd64).

ASIDE: attached is a patch that applies cleanly to RELENG_8_1 -- in case an=
yone
is interested.

However -- as you probably already know -- the patch should not be applied =
to
i386, else the following compiler error arises:

#################### BEGIN COMPILER OUTPUT ####################
cc -c -O -pipe  -std=3Dc99 -g -Wall -Wredundant-decls -Wnested-externs
-Wstrict-prototypes  -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-q=
ual
-Wundef -Wno-pointer-sign -fformat-extensions -nostdinc  -I. -I../../..
-I../../../contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include
opt_global.h -fno-common -finline-limit=3D8000 --param inline-unit-growth=
=3D100
--param large-function-growth=3D1000  -mno-align-long-strings
-mpreferred-stack-boundary=3D2  -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno=
-sse3
-ffreestanding -fstack-protector -Werror  ../../../net/rtsock.c
../../../net/rtsock.c: In function 'route_output':
../../../net/rtsock.c:1136: error: 'rp' undeclared (first use in this funct=
ion)
../../../net/rtsock.c:1136: error: (Each undeclared identifier is reported =
only
once
../../../net/rtsock.c:1136: error: for each function it appears in.)
../../../net/rtsock.c: At top level:
../../../net/rtsock.c:1156: error: expected identifier or '(' before 'retur=
n'
../../../net/rtsock.c:1158: error: expected identifier or '(' before '}' to=
ken
*** Error code 1
#################### END COMPILER OUTPUT ####################

Again, thank you for the patch!
--=20
Devin

P.S. +1 vote to get this cleaned-up and committed HEAD and then MFC'd to
RELENG_8

P.P.S. Any chance there's a patch in the works to allow 32-bit "netstat -nr"
under adm64 kernel? Currently comes back with "no name list".

______________________________________________________________________
The information contained in this message is proprietary and/or confidentia=
l. If you are not the intended recipient, please: (i) delete the message an=
d all copies; (ii) do not disclose, distribute or use the message in any ma=
nner; and (iii) notify the sender immediately. In addition, please be aware=
 that any message addressed to our domain is subject to archiving and revie=
w by persons other than the intended recipient. Thank you.

------=_NextPart_000_102E_01CC8CCB.63739270
Content-Type: application/octet-stream; name="rtmsg32.RELENG_8_1-20111017.diff"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="rtmsg32.RELENG_8_1-20111017.diff"

--- sys/net/rtsock.c.orig	2011-10-17 11:44:27.000000000 -0700=0A=
+++ sys/net/rtsock.c	2011-10-17 11:52:07.000000000 -0700=0A=
@@ -75,6 +75,7 @@=0A=
 =0A=
 #ifdef COMPAT_FREEBSD32=0A=
 #include <sys/mount.h>=0A=
+#include <sys/sysent.h>=0A=
 #include <compat/freebsd32/freebsd32.h>=0A=
 =0A=
 struct if_data32 {=0A=
@@ -114,6 +115,42 @@=0A=
 	uint16_t ifm_index;=0A=
 	struct	if_data32 ifm_data;=0A=
 };=0A=
+=0A=
+struct rt_metrics32 {=0A=
+	uint32_t	rmx_locks;=0A=
+	uint32_t	rmx_mtu;=0A=
+	uint32_t	rmx_hopcount;=0A=
+	uint32_t	rmx_expire;=0A=
+	uint32_t	rmx_recvpipe;=0A=
+	uint32_t	rmx_sendpipe;=0A=
+	uint32_t	rmx_ssthresh;=0A=
+	uint32_t	rmx_rtt;=0A=
+	uint32_t	rmx_rttvar;=0A=
+	uint32_t	rmx_pksent;=0A=
+	uint32_t	rmx_weight;=0A=
+	uint32_t	rmx_filler[3];=0A=
+};=0A=
+=0A=
+struct rt_msghdr32 {=0A=
+	uint16_t	rtm_msglen;=0A=
+	uint8_t		rtm_version;=0A=
+	uint8_t		rtm_type;=0A=
+	uint16_t	rtm_index;=0A=
+	uint16_t	rtm_hole1;=0A=
+	uint32_t	rtm_flags;=0A=
+	uint32_t	rtm_addrs;=0A=
+	uint32_t	rtm_pid;=0A=
+	uint32_t	rtm_seq;=0A=
+	uint32_t	rtm_errno;=0A=
+	uint32_t	rtm_fmask;=0A=
+	uint32_t	rtm_inits;=0A=
+	struct rt_metrics32 rtm_rmx;=0A=
+};=0A=
+=0A=
+#define	SA_SIZE32(sa)						\=0A=
+    (  (!(sa) || ((struct sockaddr *)(sa))->sa_len =3D=3D 0) ?	\=0A=
+	sizeof(int32_t)		:				\=0A=
+	1 + ( (((struct sockaddr *)(sa))->sa_len - 1) | (sizeof(int32_t) - 1)))=0A=
 #endif=0A=
 =0A=
 MALLOC_DEFINE(M_RTABLE, "routetbl", "routing tables");=0A=
@@ -501,6 +538,138 @@=0A=
 	return (0);=0A=
 }=0A=
 =0A=
+#ifdef COMPAT_FREEBSD32=0A=
+static void=0A=
+freebsd32_rt_metrics_in(struct rt_metrics32 *src, struct rt_metrics =
*dst)=0A=
+{=0A=
+=0A=
+	bzero(dst, sizeof(*dst));=0A=
+	CP(*src, *dst, rmx_mtu);=0A=
+	CP(*src, *dst, rmx_expire);=0A=
+	CP(*src, *dst, rmx_pksent);=0A=
+	CP(*src, *dst, rmx_weight);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_metrics_out(struct rt_metrics *src, struct rt_metrics32 =
*dst)=0A=
+{=0A=
+=0A=
+	bzero(dst, sizeof(*dst));=0A=
+	CP(*src, *dst, rmx_mtu);=0A=
+	CP(*src, *dst, rmx_expire);=0A=
+	CP(*src, *dst, rmx_pksent);=0A=
+	CP(*src, *dst, rmx_weight);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_msghdr_in(struct rt_msghdr32 *src, struct rt_msghdr *dst)=0A=
+{=0A=
+=0A=
+	bzero(dst, sizeof(*dst));=0A=
+	/* CP(*src, *dst, rtm_msglen); */	/* updated separately */=0A=
+	CP(*src, *dst, rtm_version);=0A=
+	CP(*src, *dst, rtm_type);=0A=
+	CP(*src, *dst, rtm_index);=0A=
+	CP(*src, *dst, rtm_flags);=0A=
+	CP(*src, *dst, rtm_addrs);=0A=
+	CP(*src, *dst, rtm_pid);=0A=
+	CP(*src, *dst, rtm_seq);=0A=
+	CP(*src, *dst, rtm_errno);=0A=
+	CP(*src, *dst, rtm_fmask);=0A=
+	CP(*src, *dst, rtm_inits);=0A=
+	freebsd32_rt_metrics_in(&src->rtm_rmx, &dst->rtm_rmx);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_msghdr_out(struct rt_msghdr *src, struct rt_msghdr32 *dst)=0A=
+{=0A=
+=0A=
+	bzero(dst, sizeof(*dst));=0A=
+	/* CP(*src, *dst, rtm_msglen); */	/* updated separately */=0A=
+	CP(*src, *dst, rtm_version);=0A=
+	CP(*src, *dst, rtm_type);=0A=
+	CP(*src, *dst, rtm_index);=0A=
+	CP(*src, *dst, rtm_flags);=0A=
+	CP(*src, *dst, rtm_addrs);=0A=
+	CP(*src, *dst, rtm_pid);=0A=
+	CP(*src, *dst, rtm_seq);=0A=
+	CP(*src, *dst, rtm_errno);=0A=
+	CP(*src, *dst, rtm_fmask);=0A=
+	CP(*src, *dst, rtm_inits);=0A=
+	freebsd32_rt_metrics_out(&src->rtm_rmx, &dst->rtm_rmx);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_len_in(caddr_t cp, caddr_t cplim, int *buflen)=0A=
+{=0A=
+	struct sockaddr *sa;=0A=
+	int i;=0A=
+=0A=
+	for (i =3D 0, *buflen =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+		sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+		if (cp + sa->sa_len > cplim)=0A=
+			return (EINVAL);=0A=
+		cp +=3D SA_SIZE32(sa);=0A=
+		*buflen +=3D SA_SIZE(sa);=0A=
+	}=0A=
+	return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_len_out(caddr_t cp, caddr_t cplim, int *buflen)=0A=
+{=0A=
+	struct sockaddr *sa;=0A=
+	int i;=0A=
+=0A=
+	for (i =3D 0, *buflen =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+		sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+		if (cp + sa->sa_len > cplim)=0A=
+			return (EINVAL);=0A=
+		cp +=3D SA_SIZE(sa);=0A=
+		*buflen +=3D SA_SIZE32(sa);=0A=
+	}=0A=
+	return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_in(caddr_t cp, caddr_t cp64, caddr_t cplim)=0A=
+{=0A=
+	struct sockaddr *sa;=0A=
+	int i;=0A=
+=0A=
+	for (i =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+		sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+		if (cp + sa->sa_len > cplim)=0A=
+			return (EINVAL);=0A=
+		memcpy(cp64, cp, SA_SIZE32(sa));=0A=
+		cp +=3D SA_SIZE32(sa);=0A=
+		cp64 +=3D SA_SIZE(sa);=0A=
+	}=0A=
+	return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_out(caddr_t cp, caddr_t cp32, caddr_t cplim)=0A=
+{=0A=
+	struct sockaddr *sa;=0A=
+	int i;=0A=
+=0A=
+	for (i =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+		sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+		if (cp + sa->sa_len > cplim)=0A=
+			return (EINVAL);=0A=
+		memcpy(cp32, cp, SA_SIZE(sa));=0A=
+		cp +=3D SA_SIZE(sa);=0A=
+		cp32 +=3D SA_SIZE32(sa);=0A=
+	}=0A=
+	return (0);=0A=
+}=0A=
+#endif=0A=
+=0A=
 /*ARGSUSED*/=0A=
 static int=0A=
 route_output(struct mbuf *m, struct socket *so)=0A=
@@ -513,7 +682,14 @@=0A=
 	int len, error =3D 0;=0A=
 	struct ifnet *ifp =3D NULL;=0A=
 	union sockaddr_union saun;=0A=
+	size_t rtmlen;=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+	struct rt_msghdr32 *rtm32 =3D NULL;=0A=
+	int len32 =3D 0, rt_mspace_len =3D 0, wrap32 =3D 0;=0A=
 =0A=
+	if (SV_CURPROC_FLAG(SV_ILP32))=0A=
+		wrap32 =3D 1;=0A=
+#endif=0A=
 #define senderr(e) { error =3D e; goto flush;}=0A=
 	if (m =3D=3D NULL || ((m->m_len < sizeof(long)) &&=0A=
 		       (m =3D m_pullup(m, sizeof(long))) =3D=3D NULL))=0A=
@@ -521,17 +697,50 @@=0A=
 	if ((m->m_flags & M_PKTHDR) =3D=3D 0)=0A=
 		panic("route_output");=0A=
 	len =3D m->m_pkthdr.len;=0A=
-	if (len < sizeof(*rtm) ||=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+	if (wrap32) {=0A=
+		rtmlen =3D sizeof(*rtm32);=0A=
+		len32 =3D len;=0A=
+	} else=0A=
+#endif=0A=
+		rtmlen =3D sizeof(*rtm);=0A=
+=0A=
+	if (len < rtmlen ||=0A=
 	    len !=3D mtod(m, struct rt_msghdr *)->rtm_msglen) {=0A=
 		info.rti_info[RTAX_DST] =3D NULL;=0A=
 		senderr(EINVAL);=0A=
 	}=0A=
-	R_Malloc(rtm, struct rt_msghdr *, len);=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+	if (wrap32) {=0A=
+		R_Malloc(rtm32, struct rt_msghdr32 *, len32);=0A=
+		if (rtm32 =3D=3D NULL) {=0A=
+			info.rti_info[RTAX_DST] =3D NULL;=0A=
+			senderr(ENOBUFS);=0A=
+		}=0A=
+		m_copydata(m, 0, len32, (caddr_t)rtm32);=0A=
+		freebsd32_rt_mspace_len_in((caddr_t)(rtm32 + 1), len32 + =
(caddr_t)rtm32, &rt_mspace_len);=0A=
+		/* fixup len to alloc rtm of native size */=0A=
+		len =3D rt_mspace_len + sizeof(*rtm);=0A=
+	}=0A=
+#endif=0A=
+	R_Zalloc(rtm, struct rt_msghdr *, len);=0A=
 	if (rtm =3D=3D NULL) {=0A=
 		info.rti_info[RTAX_DST] =3D NULL;=0A=
 		senderr(ENOBUFS);=0A=
 	}=0A=
-	m_copydata(m, 0, len, (caddr_t)rtm);=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+	if (wrap32) {=0A=
+		freebsd32_rt_msghdr_in(rtm32, rtm);=0A=
+		freebsd32_rt_mspace_in((caddr_t)(rtm32 + 1),=0A=
+		    (caddr_t)(rtm + 1), (caddr_t)rtm32 + len32);=0A=
+		rtm->rtm_msglen =3D len;=0A=
+	} else=0A=
+#endif=0A=
+		m_copydata(m, 0, len, (caddr_t)rtm);=0A=
+=0A=
 	if (rtm->rtm_version !=3D RTM_VERSION) {=0A=
 		info.rti_info[RTAX_DST] =3D NULL;=0A=
 		senderr(EPROTONOSUPPORT);=0A=
@@ -543,6 +752,16 @@=0A=
 		info.rti_info[RTAX_DST] =3D NULL;=0A=
 		senderr(EINVAL);=0A=
 	}=0A=
+/*=0A=
+        int i; uint16_t *j;=0A=
+=0A=
+        for (i =3D 0; i * (int)sizeof(uint16_t) < len; i++) {=0A=
+                  j =3D (uint16_t *)rtm;=0A=
+                  if (i % 8 =3D=3D 0)=0A=
+                         printf("\n0x%04jx ", =
(uintmax_t)i*sizeof(uint16_t));=0A=
+                  printf("%04x ", bswap16(j[i]));=0A=
+        }; printf("\n");=0A=
+*/=0A=
 	info.rti_flags =3D rtm->rtm_flags;=0A=
 	if (info.rti_info[RTAX_DST] =3D=3D NULL ||=0A=
 	    info.rti_info[RTAX_DST]->sa_family >=3D AF_MAX ||=0A=
@@ -877,13 +1096,41 @@=0A=
 		rp =3D sotorawcb(so);=0A=
 	}=0A=
 	if (rtm) {=0A=
-		m_copyback(m, 0, rtm->rtm_msglen, (caddr_t)rtm);=0A=
-		if (m->m_pkthdr.len < rtm->rtm_msglen) {=0A=
-			m_freem(m);=0A=
-			m =3D NULL;=0A=
-		} else if (m->m_pkthdr.len > rtm->rtm_msglen)=0A=
-			m_adj(m, rtm->rtm_msglen - m->m_pkthdr.len);=0A=
-		Free(rtm);=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+		if (wrap32) {=0A=
+			int rt_mspace_len32;=0A=
+=0A=
+			freebsd32_rt_mspace_len_out((caddr_t)(rtm + 1),=0A=
+			    len + (caddr_t)rtm, &rt_mspace_len32);=0A=
+			len32 =3D rt_mspace_len32 + sizeof(*rtm32);=0A=
+			if (len32 > rtm32->rtm_msglen) {=0A=
+				struct rt_msghdr32 *new_rtm32;=0A=
+				R_Malloc(new_rtm32, struct rt_msghdr32 *, len32);=0A=
+				if (new_rtm32 =3D=3D NULL)=0A=
+					senderr(ENOBUFS);=0A=
+				bcopy(rtm32, new_rtm32, rtm32->rtm_msglen);=0A=
+				Free(rtm32); rtm32 =3D new_rtm32;=0A=
+			}=0A=
+			freebsd32_rt_msghdr_out(rtm, rtm32);=0A=
+			rtm32->rtm_msglen =3D len32;=0A=
+			freebsd32_rt_mspace_out((caddr_t)(rtm + 1),=0A=
+			    (caddr_t)(rtm32 + 1), (caddr_t)rtm + len);=0A=
+=0A=
+			m_copyback(m, 0, rtm32->rtm_msglen, (caddr_t)rtm32);=0A=
+			if (m->m_pkthdr.len < rtm32->rtm_msglen) {=0A=
+				m_freem(m);=0A=
+				m =3D NULL;=0A=
+			} else if (m->m_pkthdr.len > rtm32->rtm_msglen)=0A=
+				m_adj(m, rtm32->rtm_msglen - m->m_pkthdr.len);=0A=
+		} else {=0A=
+#endif=0A=
+			m_copyback(m, 0, rtm->rtm_msglen, (caddr_t)rtm);=0A=
+			if (m->m_pkthdr.len < rtm->rtm_msglen) {=0A=
+				m_freem(m);=0A=
+				m =3D NULL;=0A=
+			} else if (m->m_pkthdr.len > rtm->rtm_msglen)=0A=
+				m_adj(m, rtm->rtm_msglen - m->m_pkthdr.len);=0A=
+		}=0A=
 	}=0A=
 	if (m) {=0A=
 		if (rp) {=0A=
@@ -898,6 +1145,13 @@=0A=
 		} else=0A=
 			rt_dispatch(m, info.rti_info[RTAX_DST]);=0A=
 	}=0A=
+	if (rtm) {=0A=
+		Free(rtm);=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+		if (wrap32)=0A=
+			Free(rtm32);=0A=
+#endif=0A=
+	}=0A=
     }=0A=
 	return (error);=0A=
 #undef	sa_equal=0A=

------=_NextPart_000_102E_01CC8CCB.63739270--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?102d01cc8d06$0fd26a70$2f773f50$>