Date: Mon, 17 Oct 2011 12:50:44 -0700 From: Devin Teske <devin.teske@fisglobal.com> To: "'Sergey Kandaurov'" <pluknet@gmail.com> Cc: 'Dave Robison' <Dave.Robison@fisglobal.com>, 'FreeBSD amd64' <freebsd-amd64@freebsd.org>, devin.teske@fisglobal.com Subject: RE: 32-bit route(8) on amd64 host and jails Message-ID: <102d01cc8d06$0fd26a70$2f773f50$@fisglobal.com> In-Reply-To: <CAE-mSO%2B_hc%2BGJHYUyX6dBS6NckV=iiHRrGMYbc=AxQeqKwK2Zg@mail.gmail.com> References: <714EF3C9-33B0-4EF5-B52C-1E95F7F432F9@fisglobal.com> <CAE-mSO%2B4KKWeC6-Z3UAi70bxxEUJL5gUD0phEfNzzkDMXrA3pg@mail.gmail.com> <CAE-mSO%2B_hc%2BGJHYUyX6dBS6NckV=iiHRrGMYbc=AxQeqKwK2Zg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_NextPart_000_102E_01CC8CCB.63739270
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"
> -----Original Message-----
> From: Sergey Kandaurov [mailto:pluknet@gmail.com]
> Sent: Friday, October 14, 2011 2:40 PM
> To: Devin Teske
> Cc: FreeBSD amd64; Dave Robison
> Subject: Re: 32-bit route(8) on amd64 host and jails
>=20
> On 21 September 2011 16:47, Sergey Kandaurov <pluknet@gmail.com> wrote:
> > On 21 September 2011 06:40, Devin Teske <devin.teske@fisglobal.com> wro=
te:
> >> I'm noticing that a 32-bit route(8) shows strange behaviour while runn=
ing
> under an amd64 kernel (regardless of whether in the base-host -- utilizing
> /usr/lib32/libc.so.7 -- or in a jail and/or vimage -- 32-bit in nature;
results are
> same).
> >>
> >> Executable runs fine, but you can't (a) set the default route or (b) v=
iew
the
> default route (after successfully setting it with the amd64 build, of cou=
rse).
> >>
> >> ASIDE: This is under 8.1-RELEASE.
> >>
> >> When attempting to set the default route, you get the following...
> >>
> >> root@kps0a / # route add -net default 10.10.125.99
> >> route: writing to routing socket: Invalid argument add net default:
> >> gateway 10.10.125.99: Invalid argument
> >>
> >> Meanwhile, using the amd64 version, no issues.
> >>
> >> When attempting to view the default route, you get the following...
> >>
> >> root@kps0a / # route -n get default
> >> =A0 route to: default
> >> destination: default
> >> =A0 =A0 =A0 mask: default
> >> =A0 =A0gateway: default
> >> =A0 =A0 =A0flags: <UP,GATEWAY,DONE,STATIC>
> >> =A0recvpipe =A0sendpipe =A0ssthresh =A0rtt,msec =A0 =A0mtu =A0 =A0 =A0=
=A0weight
> >> expire
> >> =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0 =
=A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 0
> >> -1316570637
> >>
> >> It's the "gateway: default" that's out of place.
> >>
> >
> > Currently, FreeBSD has a poor shape of rtsocket freebsd32
> > compatibility, AFAIK only sysctl layer (sysctl_rtsock) is aware of it.
> > That means when a 32bit app writes some data to a routing socket,
> > kernel expects to receive it in the native ABI, and it doesn't work.
>=20
> Can you please try this preliminary patch and report back?
> You only need to rebuild kernel and reboot with the attached patch to ena=
ble
use
> of e.g. 32-bit route(8) command on amd64 kernel.
> No any userland utility needs to rebuild to enable the compatibility.
> Tested on 8.2 amd64 only with route get default and route delete/add.
>=20
First-off, let me say thank you. The patch works as-expected (for amd64).
ASIDE: attached is a patch that applies cleanly to RELENG_8_1 -- in case an=
yone
is interested.
However -- as you probably already know -- the patch should not be applied =
to
i386, else the following compiler error arises:
#################### BEGIN COMPILER OUTPUT ####################
cc -c -O -pipe -std=3Dc99 -g -Wall -Wredundant-decls -Wnested-externs
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-q=
ual
-Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I../../..
-I../../../contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include
opt_global.h -fno-common -finline-limit=3D8000 --param inline-unit-growth=
=3D100
--param large-function-growth=3D1000 -mno-align-long-strings
-mpreferred-stack-boundary=3D2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno=
-sse3
-ffreestanding -fstack-protector -Werror ../../../net/rtsock.c
../../../net/rtsock.c: In function 'route_output':
../../../net/rtsock.c:1136: error: 'rp' undeclared (first use in this funct=
ion)
../../../net/rtsock.c:1136: error: (Each undeclared identifier is reported =
only
once
../../../net/rtsock.c:1136: error: for each function it appears in.)
../../../net/rtsock.c: At top level:
../../../net/rtsock.c:1156: error: expected identifier or '(' before 'retur=
n'
../../../net/rtsock.c:1158: error: expected identifier or '(' before '}' to=
ken
*** Error code 1
#################### END COMPILER OUTPUT ####################
Again, thank you for the patch!
--=20
Devin
P.S. +1 vote to get this cleaned-up and committed HEAD and then MFC'd to
RELENG_8
P.P.S. Any chance there's a patch in the works to allow 32-bit "netstat -nr"
under adm64 kernel? Currently comes back with "no name list".
______________________________________________________________________
The information contained in this message is proprietary and/or confidentia=
l. If you are not the intended recipient, please: (i) delete the message an=
d all copies; (ii) do not disclose, distribute or use the message in any ma=
nner; and (iii) notify the sender immediately. In addition, please be aware=
that any message addressed to our domain is subject to archiving and revie=
w by persons other than the intended recipient. Thank you.
------=_NextPart_000_102E_01CC8CCB.63739270
Content-Type: application/octet-stream; name="rtmsg32.RELENG_8_1-20111017.diff"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="rtmsg32.RELENG_8_1-20111017.diff"
--- sys/net/rtsock.c.orig 2011-10-17 11:44:27.000000000 -0700=0A=
+++ sys/net/rtsock.c 2011-10-17 11:52:07.000000000 -0700=0A=
@@ -75,6 +75,7 @@=0A=
=0A=
#ifdef COMPAT_FREEBSD32=0A=
#include <sys/mount.h>=0A=
+#include <sys/sysent.h>=0A=
#include <compat/freebsd32/freebsd32.h>=0A=
=0A=
struct if_data32 {=0A=
@@ -114,6 +115,42 @@=0A=
uint16_t ifm_index;=0A=
struct if_data32 ifm_data;=0A=
};=0A=
+=0A=
+struct rt_metrics32 {=0A=
+ uint32_t rmx_locks;=0A=
+ uint32_t rmx_mtu;=0A=
+ uint32_t rmx_hopcount;=0A=
+ uint32_t rmx_expire;=0A=
+ uint32_t rmx_recvpipe;=0A=
+ uint32_t rmx_sendpipe;=0A=
+ uint32_t rmx_ssthresh;=0A=
+ uint32_t rmx_rtt;=0A=
+ uint32_t rmx_rttvar;=0A=
+ uint32_t rmx_pksent;=0A=
+ uint32_t rmx_weight;=0A=
+ uint32_t rmx_filler[3];=0A=
+};=0A=
+=0A=
+struct rt_msghdr32 {=0A=
+ uint16_t rtm_msglen;=0A=
+ uint8_t rtm_version;=0A=
+ uint8_t rtm_type;=0A=
+ uint16_t rtm_index;=0A=
+ uint16_t rtm_hole1;=0A=
+ uint32_t rtm_flags;=0A=
+ uint32_t rtm_addrs;=0A=
+ uint32_t rtm_pid;=0A=
+ uint32_t rtm_seq;=0A=
+ uint32_t rtm_errno;=0A=
+ uint32_t rtm_fmask;=0A=
+ uint32_t rtm_inits;=0A=
+ struct rt_metrics32 rtm_rmx;=0A=
+};=0A=
+=0A=
+#define SA_SIZE32(sa) \=0A=
+ ( (!(sa) || ((struct sockaddr *)(sa))->sa_len =3D=3D 0) ? \=0A=
+ sizeof(int32_t) : \=0A=
+ 1 + ( (((struct sockaddr *)(sa))->sa_len - 1) | (sizeof(int32_t) - 1)))=0A=
#endif=0A=
=0A=
MALLOC_DEFINE(M_RTABLE, "routetbl", "routing tables");=0A=
@@ -501,6 +538,138 @@=0A=
return (0);=0A=
}=0A=
=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+static void=0A=
+freebsd32_rt_metrics_in(struct rt_metrics32 *src, struct rt_metrics =
*dst)=0A=
+{=0A=
+=0A=
+ bzero(dst, sizeof(*dst));=0A=
+ CP(*src, *dst, rmx_mtu);=0A=
+ CP(*src, *dst, rmx_expire);=0A=
+ CP(*src, *dst, rmx_pksent);=0A=
+ CP(*src, *dst, rmx_weight);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_metrics_out(struct rt_metrics *src, struct rt_metrics32 =
*dst)=0A=
+{=0A=
+=0A=
+ bzero(dst, sizeof(*dst));=0A=
+ CP(*src, *dst, rmx_mtu);=0A=
+ CP(*src, *dst, rmx_expire);=0A=
+ CP(*src, *dst, rmx_pksent);=0A=
+ CP(*src, *dst, rmx_weight);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_msghdr_in(struct rt_msghdr32 *src, struct rt_msghdr *dst)=0A=
+{=0A=
+=0A=
+ bzero(dst, sizeof(*dst));=0A=
+ /* CP(*src, *dst, rtm_msglen); */ /* updated separately */=0A=
+ CP(*src, *dst, rtm_version);=0A=
+ CP(*src, *dst, rtm_type);=0A=
+ CP(*src, *dst, rtm_index);=0A=
+ CP(*src, *dst, rtm_flags);=0A=
+ CP(*src, *dst, rtm_addrs);=0A=
+ CP(*src, *dst, rtm_pid);=0A=
+ CP(*src, *dst, rtm_seq);=0A=
+ CP(*src, *dst, rtm_errno);=0A=
+ CP(*src, *dst, rtm_fmask);=0A=
+ CP(*src, *dst, rtm_inits);=0A=
+ freebsd32_rt_metrics_in(&src->rtm_rmx, &dst->rtm_rmx);=0A=
+}=0A=
+=0A=
+static void=0A=
+freebsd32_rt_msghdr_out(struct rt_msghdr *src, struct rt_msghdr32 *dst)=0A=
+{=0A=
+=0A=
+ bzero(dst, sizeof(*dst));=0A=
+ /* CP(*src, *dst, rtm_msglen); */ /* updated separately */=0A=
+ CP(*src, *dst, rtm_version);=0A=
+ CP(*src, *dst, rtm_type);=0A=
+ CP(*src, *dst, rtm_index);=0A=
+ CP(*src, *dst, rtm_flags);=0A=
+ CP(*src, *dst, rtm_addrs);=0A=
+ CP(*src, *dst, rtm_pid);=0A=
+ CP(*src, *dst, rtm_seq);=0A=
+ CP(*src, *dst, rtm_errno);=0A=
+ CP(*src, *dst, rtm_fmask);=0A=
+ CP(*src, *dst, rtm_inits);=0A=
+ freebsd32_rt_metrics_out(&src->rtm_rmx, &dst->rtm_rmx);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_len_in(caddr_t cp, caddr_t cplim, int *buflen)=0A=
+{=0A=
+ struct sockaddr *sa;=0A=
+ int i;=0A=
+=0A=
+ for (i =3D 0, *buflen =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+ sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+ if (cp + sa->sa_len > cplim)=0A=
+ return (EINVAL);=0A=
+ cp +=3D SA_SIZE32(sa);=0A=
+ *buflen +=3D SA_SIZE(sa);=0A=
+ }=0A=
+ return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_len_out(caddr_t cp, caddr_t cplim, int *buflen)=0A=
+{=0A=
+ struct sockaddr *sa;=0A=
+ int i;=0A=
+=0A=
+ for (i =3D 0, *buflen =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+ sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+ if (cp + sa->sa_len > cplim)=0A=
+ return (EINVAL);=0A=
+ cp +=3D SA_SIZE(sa);=0A=
+ *buflen +=3D SA_SIZE32(sa);=0A=
+ }=0A=
+ return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_in(caddr_t cp, caddr_t cp64, caddr_t cplim)=0A=
+{=0A=
+ struct sockaddr *sa;=0A=
+ int i;=0A=
+=0A=
+ for (i =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+ sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+ if (cp + sa->sa_len > cplim)=0A=
+ return (EINVAL);=0A=
+ memcpy(cp64, cp, SA_SIZE32(sa));=0A=
+ cp +=3D SA_SIZE32(sa);=0A=
+ cp64 +=3D SA_SIZE(sa);=0A=
+ }=0A=
+ return (0);=0A=
+}=0A=
+=0A=
+static int=0A=
+freebsd32_rt_mspace_out(caddr_t cp, caddr_t cp32, caddr_t cplim)=0A=
+{=0A=
+ struct sockaddr *sa;=0A=
+ int i;=0A=
+=0A=
+ for (i =3D 0; i < RTAX_MAX && cp < cplim; i++) {=0A=
+ sa =3D (struct sockaddr *)cp;=0A=
+=0A=
+ if (cp + sa->sa_len > cplim)=0A=
+ return (EINVAL);=0A=
+ memcpy(cp32, cp, SA_SIZE(sa));=0A=
+ cp +=3D SA_SIZE(sa);=0A=
+ cp32 +=3D SA_SIZE32(sa);=0A=
+ }=0A=
+ return (0);=0A=
+}=0A=
+#endif=0A=
+=0A=
/*ARGSUSED*/=0A=
static int=0A=
route_output(struct mbuf *m, struct socket *so)=0A=
@@ -513,7 +682,14 @@=0A=
int len, error =3D 0;=0A=
struct ifnet *ifp =3D NULL;=0A=
union sockaddr_union saun;=0A=
+ size_t rtmlen;=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ struct rt_msghdr32 *rtm32 =3D NULL;=0A=
+ int len32 =3D 0, rt_mspace_len =3D 0, wrap32 =3D 0;=0A=
=0A=
+ if (SV_CURPROC_FLAG(SV_ILP32))=0A=
+ wrap32 =3D 1;=0A=
+#endif=0A=
#define senderr(e) { error =3D e; goto flush;}=0A=
if (m =3D=3D NULL || ((m->m_len < sizeof(long)) &&=0A=
(m =3D m_pullup(m, sizeof(long))) =3D=3D NULL))=0A=
@@ -521,17 +697,50 @@=0A=
if ((m->m_flags & M_PKTHDR) =3D=3D 0)=0A=
panic("route_output");=0A=
len =3D m->m_pkthdr.len;=0A=
- if (len < sizeof(*rtm) ||=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ if (wrap32) {=0A=
+ rtmlen =3D sizeof(*rtm32);=0A=
+ len32 =3D len;=0A=
+ } else=0A=
+#endif=0A=
+ rtmlen =3D sizeof(*rtm);=0A=
+=0A=
+ if (len < rtmlen ||=0A=
len !=3D mtod(m, struct rt_msghdr *)->rtm_msglen) {=0A=
info.rti_info[RTAX_DST] =3D NULL;=0A=
senderr(EINVAL);=0A=
}=0A=
- R_Malloc(rtm, struct rt_msghdr *, len);=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ if (wrap32) {=0A=
+ R_Malloc(rtm32, struct rt_msghdr32 *, len32);=0A=
+ if (rtm32 =3D=3D NULL) {=0A=
+ info.rti_info[RTAX_DST] =3D NULL;=0A=
+ senderr(ENOBUFS);=0A=
+ }=0A=
+ m_copydata(m, 0, len32, (caddr_t)rtm32);=0A=
+ freebsd32_rt_mspace_len_in((caddr_t)(rtm32 + 1), len32 + =
(caddr_t)rtm32, &rt_mspace_len);=0A=
+ /* fixup len to alloc rtm of native size */=0A=
+ len =3D rt_mspace_len + sizeof(*rtm);=0A=
+ }=0A=
+#endif=0A=
+ R_Zalloc(rtm, struct rt_msghdr *, len);=0A=
if (rtm =3D=3D NULL) {=0A=
info.rti_info[RTAX_DST] =3D NULL;=0A=
senderr(ENOBUFS);=0A=
}=0A=
- m_copydata(m, 0, len, (caddr_t)rtm);=0A=
+=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ if (wrap32) {=0A=
+ freebsd32_rt_msghdr_in(rtm32, rtm);=0A=
+ freebsd32_rt_mspace_in((caddr_t)(rtm32 + 1),=0A=
+ (caddr_t)(rtm + 1), (caddr_t)rtm32 + len32);=0A=
+ rtm->rtm_msglen =3D len;=0A=
+ } else=0A=
+#endif=0A=
+ m_copydata(m, 0, len, (caddr_t)rtm);=0A=
+=0A=
if (rtm->rtm_version !=3D RTM_VERSION) {=0A=
info.rti_info[RTAX_DST] =3D NULL;=0A=
senderr(EPROTONOSUPPORT);=0A=
@@ -543,6 +752,16 @@=0A=
info.rti_info[RTAX_DST] =3D NULL;=0A=
senderr(EINVAL);=0A=
}=0A=
+/*=0A=
+ int i; uint16_t *j;=0A=
+=0A=
+ for (i =3D 0; i * (int)sizeof(uint16_t) < len; i++) {=0A=
+ j =3D (uint16_t *)rtm;=0A=
+ if (i % 8 =3D=3D 0)=0A=
+ printf("\n0x%04jx ", =
(uintmax_t)i*sizeof(uint16_t));=0A=
+ printf("%04x ", bswap16(j[i]));=0A=
+ }; printf("\n");=0A=
+*/=0A=
info.rti_flags =3D rtm->rtm_flags;=0A=
if (info.rti_info[RTAX_DST] =3D=3D NULL ||=0A=
info.rti_info[RTAX_DST]->sa_family >=3D AF_MAX ||=0A=
@@ -877,13 +1096,41 @@=0A=
rp =3D sotorawcb(so);=0A=
}=0A=
if (rtm) {=0A=
- m_copyback(m, 0, rtm->rtm_msglen, (caddr_t)rtm);=0A=
- if (m->m_pkthdr.len < rtm->rtm_msglen) {=0A=
- m_freem(m);=0A=
- m =3D NULL;=0A=
- } else if (m->m_pkthdr.len > rtm->rtm_msglen)=0A=
- m_adj(m, rtm->rtm_msglen - m->m_pkthdr.len);=0A=
- Free(rtm);=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ if (wrap32) {=0A=
+ int rt_mspace_len32;=0A=
+=0A=
+ freebsd32_rt_mspace_len_out((caddr_t)(rtm + 1),=0A=
+ len + (caddr_t)rtm, &rt_mspace_len32);=0A=
+ len32 =3D rt_mspace_len32 + sizeof(*rtm32);=0A=
+ if (len32 > rtm32->rtm_msglen) {=0A=
+ struct rt_msghdr32 *new_rtm32;=0A=
+ R_Malloc(new_rtm32, struct rt_msghdr32 *, len32);=0A=
+ if (new_rtm32 =3D=3D NULL)=0A=
+ senderr(ENOBUFS);=0A=
+ bcopy(rtm32, new_rtm32, rtm32->rtm_msglen);=0A=
+ Free(rtm32); rtm32 =3D new_rtm32;=0A=
+ }=0A=
+ freebsd32_rt_msghdr_out(rtm, rtm32);=0A=
+ rtm32->rtm_msglen =3D len32;=0A=
+ freebsd32_rt_mspace_out((caddr_t)(rtm + 1),=0A=
+ (caddr_t)(rtm32 + 1), (caddr_t)rtm + len);=0A=
+=0A=
+ m_copyback(m, 0, rtm32->rtm_msglen, (caddr_t)rtm32);=0A=
+ if (m->m_pkthdr.len < rtm32->rtm_msglen) {=0A=
+ m_freem(m);=0A=
+ m =3D NULL;=0A=
+ } else if (m->m_pkthdr.len > rtm32->rtm_msglen)=0A=
+ m_adj(m, rtm32->rtm_msglen - m->m_pkthdr.len);=0A=
+ } else {=0A=
+#endif=0A=
+ m_copyback(m, 0, rtm->rtm_msglen, (caddr_t)rtm);=0A=
+ if (m->m_pkthdr.len < rtm->rtm_msglen) {=0A=
+ m_freem(m);=0A=
+ m =3D NULL;=0A=
+ } else if (m->m_pkthdr.len > rtm->rtm_msglen)=0A=
+ m_adj(m, rtm->rtm_msglen - m->m_pkthdr.len);=0A=
+ }=0A=
}=0A=
if (m) {=0A=
if (rp) {=0A=
@@ -898,6 +1145,13 @@=0A=
} else=0A=
rt_dispatch(m, info.rti_info[RTAX_DST]);=0A=
}=0A=
+ if (rtm) {=0A=
+ Free(rtm);=0A=
+#ifdef COMPAT_FREEBSD32=0A=
+ if (wrap32)=0A=
+ Free(rtm32);=0A=
+#endif=0A=
+ }=0A=
}=0A=
return (error);=0A=
#undef sa_equal=0A=
------=_NextPart_000_102E_01CC8CCB.63739270--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?102d01cc8d06$0fd26a70$2f773f50$>