Date: Wed, 18 May 2016 15:08:07 +0000 From: Grzegorz Junka <list1@gjunka.com> To: freebsd-jail@freebsd.org Subject: Re: jails in different private subnets on the same host Message-ID: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> In-Reply-To: <AF80A4F2-3605-43A0-81CD-B68659B694C4@lists.zabbadoz.net> References: <faf9e698-baee-f988-df64-5bcda4b1c7c9@gjunka.com> <AF80A4F2-3605-43A0-81CD-B68659B694C4@lists.zabbadoz.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 18/05/2016 14:11, Bjoern A. Zeeb wrote:
> >> On 18 May 2016, at 14:00 , Grzegorz Junka <list1@gjunka.com> >>
wrote: >> >> Is it possible to have two jails on the same host each one
in a >> different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and
have >> routing between them working without issues? >> >> I know it's
possible to run jails with IPs in those two subnets >> but it seems
there is no routing and I am not sure if it's because >> I can't
configure my router properly or there is a more >> fundamental problem.
One issue I see is that the jail can't have a >> different default
gateway than the host, and that for now is >> 192.168.1.1, but I don't
see a reason why 10.33.1.0 wouldn't be >> able to use 192.168.1.1 as
it's default gateway provided there is >> routing between those two
subnets. > > Given they are both on the same base system host, both
addresses > are connected locally and thus the kernel knows where to
deliver > these packets. If that doesn’t work, there is a bug
somewhere. > > If you want different default gateways then you may want
to look > into using different FIBs for different jails. See route(8)
and > jail(8) for parameters to set and tune. > > /bz >
I can ping both jails from the main host, however when in the 10.33.1.0
jail I can't access any jail in the 192.168.1.0 network. This is what
netstat -r shows:
---------------------------------
root@dns1:/ # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
inet 192.168.1.60 netmask 0xffffffff broadcast 192.168.1.60
media: Ethernet autoselect
status: active
laggproto lacp lagghash l2,l3,l4
laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
root@dns1:/ # netstat -r
Routing tables
Internet:
Destination Gateway Flags Netif Expire
dns1 link#4 UHS lo0
---------------------------------
root@pjp1:/ # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:ae:e8:bc
inet 10.33.1.40 netmask 0xffffffff broadcast 10.33.1.40
media: Ethernet autoselect
status: active
laggproto lacp lagghash l2,l3,l4
laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
root@pjp1:/ # netstat -r
netstat: kvm not available: /dev/mem: No such file or directory
Routing tables
rt_tables: symbol not in namelist
---------------------------------
On the main host:
root@somehost:~ # netstat -r
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.1.1 UGS lagg0
pjp1.somehost.somedomain. link#4 UHS lo0
10.33.1.40/32 link#4 U lagg0
localhost link#3 UH lo0
192.168.1.0 link#4 U lagg0
somehost link#4 UHS lo0
web1.somehost.somedomain. link#4 UHS lo0
192.168.1.50/32 link#4 U lagg0
dns1.somehost.somedomain. link#4 UHS lo0
192.168.1.60/32 link#4 U lagg0
(... other jails)
Internet6:
Destination Gateway Flags Netif Expire
:: localhost UGRS lo0
localhost link#3 UH lo0
::ffff:0.0.0.0 localhost UGRS lo0
fe80:: localhost UGRS lo0
fe80::%lo0 link#3 U lo0
fe80::1%lo0 link#3 UHS lo0
ff01::%lo0 localhost U lo0
ff02:: localhost UGRS lo0
ff02::%lo0 localhost U lo0
---------------------------------
I would rather not set up different FIBs for different jails, unless
required. First of all I would like to establish what's wrong.
I just tried telnet 192.168.1.50 80 from the main host and from the
10.33.1.40 jail. From the main host it works without issues. From the
jail it eventually connected after 15 or so seconds of waiting.
Grzegorz
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?07d67bd5-206c-edd8-7f47-ef2b5c538e01>