Date: Mon, 12 Feb 2018 12:06:39 +0100 From: "O. Hartmann" <ohartmann@walstatt.org> To: freebsd-jail@freebsd.org, freebsd-current <freebsd-current@freebsd.org> Cc: "O. Hartmann" <ohartmann@walstatt.org>, Olivier =?ISO-8859-1?Q?Cochard?= =?ISO-8859-1?Q?-Labb=E9?= <olivier@freebsd.org> Subject: Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing Message-ID: <20180212120639.7931f609@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <20180212093747.20024e5f@freyja.zeit4.iv.bundesimmobilien.de> References: <20180208093052.7f5d7a98@freyja.zeit4.iv.bundesimmobilien.de> <20180209172259.1ec9b9f4@thor.intern.walstatt.dynvpn.de> <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> <CA%2Bq%2BTcqX_BxVpWRUorbnR_o_sBVxmryMND1jrw3UjYW2KQdg_w@mail.gmail.com> <20180212093747.20024e5f@freyja.zeit4.iv.bundesimmobilien.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 12 Feb 2018 09:37:47 +0100 "O. Hartmann" <ohartmann@walstatt.org> wrote: > On Sat, 10 Feb 2018 11:52:18 +0100 > Olivier Cochard-Labb=C3=A9 <olivier@freebsd.org> wrote: >=20 > > On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann <ohartmann@walstatt.org> w= rote: > > =20 > > > > > > The moment any of the bridges gets an additional member epair interfa= ce > > > (so the bridge > > > has at least three members including the on reaching into the virtual > > > router jail) the > > > vbridge seems to operate unpredictable (to me). Pinging jails memeber= of > > > that vbridge > > > are unreachable. > > > > > > =20 > > =E2=80=8BFirst idea: > > Did you try with a more simple setup, like with just 3 jails members of= one > > bridge ? =20 > > =3D> I've tried it on a -head, and all 4 members (3 jails and the host= ) =20 > > reach to communicate. > >=20 > > Second idea: > > Can you check that all epairs have different MAC address each ?=E2=80=8B > > I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D176= 671 =20 >=20 > Wow, that PR is from 2013(!) and it is still not solved? >=20 > >=20 > > Regards, > >=20 > > Olivier > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" =20 >=20 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" After rebooting recent CURRENT, the view with "ifconfig -a ether" looked go= od so far, each epair/bridge has its unique MAC. But then, login on the jails and checking the epair's counterpart owned by = the VIMAGE jail, I found almost EVERY jail has the same MAC, even those jails members of the same bridge: [...] jail 11 on bridge2 epair20129a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a jail 10 on bridge2 epair20128a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a jail 9 on bridge 1 epair10250a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a jail 8 on bridge1 epair10238a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 = mtu 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a jail 7 on bridge0 epair238a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mt= u 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a epair251a: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mt= u 1500 options=3D8<VLAN_MTU> ether 02:68:d0:00:07:0a The way I create epairs and put them into a jail's context/domain is straig= ht forward. In jail.conf, I have more generic setup with variables like: # DNS Master ns1 { $if =3D "2"; $ip4_addr =3D "10.10.0.${if}"; $ip4_cidr =3D "24"; $ip4_my_default_route =3D "10.10.0.1"; $vnet_if =3D "epair${if}"; $home_bridge =3D "${if_bridge_core}"; depend=3D "vrouter"; allow.raw_sockets=3D "1"; } and in the common portion of jail.conf definitions, I use this: [...] vnet =3D "new"; vnet.interface =3D "${vnet_if}a"; persist; exec.clean; exec.start=3D ""; exec.start+=3D "/sbin/ifconfig ${vnet_if}a inet ${ip4_addr}/${ip4_cidr} up"; exec.start+=3D "/bin/sh /etc/rc"; exec.start+=3D "/sbin/route add default ${ip4_my_default_route}"; exec.start+=3D "/sbin/sysctl net.link.bridge.pfil_member=3D0"; exec.start+=3D "/sbin/sysctl net.link.bridge.pfil_bridge=3D0"; exec.start+=3D "/sbin/sysctl net.link.bridge.pfil_onlyip=3D0"; exec.stop=3D "/bin/sh /etc/rc.shutdown"; exec.prestart=3D ""; exec.prestart+=3D "ifconfig ${vnet_if} create"; exec.prestart+=3D "ifconfig ${vnet_if}b up"; exec.prestart+=3D "ifconfig ${home_bridge} addm ${vnet_if}b up"; exec.poststop=3D "ifconfig ${home_bridge} deletem ${vnet_if}b"; exec.poststop+=3D "ifconfig ${vnet_if}b destroy"; exec.consolelog=3D "/var/log/jail_${name}_console.log"; The big question here is: when a jail with VIMAGE kernel "swallows" a epair-pseudo device, it leaves the ciontext or visibility of the host. How = can the FreeBSD VIMAGE kernel then know about what former epair's MAC was? Is t= his mechanism maybe the culprit? It is just a thought, so I do not want to be beheaded - I'm not much into system development. Kind regards, O. Hartmann
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180212120639.7931f609>