Date: Tue, 22 Aug 2017 04:39:33 -0400 From: Boris <borisbsd@gmail.com> To: Eugene Grosbein <eugen@grosbein.net> Cc: freebsd-net@freebsd.org Subject: Re: bridge interface IP connectivity issue when using oce interface Message-ID: <CAJYdwgU13rBf%2BZ_UdL%2BQaBpgNQrLH8fVM3dEJy7pBP9Uv5xSgQ@mail.gmail.com> In-Reply-To: <599B8576.8030801@grosbein.net> References: <CAJYdwgUDWbp=2ONfSJdpNzNb8h7NomnJQecMqivg1j-tAjLhSg@mail.gmail.com> <599B8576.8030801@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Ok thanks Eugene.
net.link.bridge.inherit_mac=3D1 helped get the connectivity from the bridge
however, when I start a FreeBSD bhyve VM and attached that to a tap
interface in the bridge, I don't get connectivity from the VM.
SETUP:
Gateway - 192.168.0.222/29
Server - 192.168.0.218/29
VM - 192.168.0.219/29
On the VM, I see the ARP entries for the GW and the VM itself but cannot
ping the gateway nor the host.
--------- on the VM --------------
# uname -a
FreeBSD 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28
UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC
amd64
# ifconfig
vtnet0: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric=
0
mtu 1500
options=3D80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
ether 00:a0:98:52:c8:33
hwaddr 00:a0:98:52:c8:33
inet 192.168.0.219 netmask 0xfffffff8 broadcast 192.168.0.223
nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T <full-duplex>
status: active
# ping -c4 192.168.0.222
PING 192.168.0.222 (192.168.0.222): 56 data bytes
^C
--- 192.168.0.222 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
# arp -an
? (192.168.0.219) at 00:a0:98:52:c8:33 on vtnet0 permanent [ethernet]
? (192.168.0.218) at (incomplete) on vtnet0 expired [ethernet]
? (192.168.0.222) at 00:08:e3:ff:fd:90 on vtnet0 expires in 1126 seconds
[ethernet]
----------- end of VM ----------------
----------- on the host ---------------
root@bsdcan:~ # uname -a
FreeBSD bsdcan 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21
02:08:28 UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GEN=
ERIC
amd64
root@bsdcan:~ # ifconfig
[..]
oce3: flags=3D8143<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> metric 0 mtu 150=
0
options=3D500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_H=
WFILTER,VLAN_HWTSO>
ether 90:1b:0e:98:d3:93
hwaddr 90:1b:0e:98:d3:93
nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
[..]
tap0: flags=3D8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mt=
u
1500
options=3D80000<LINKSTATE>
ether 00:bd:0f:bb:27:00
hwaddr 00:bd:0f:bb:27:00
nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 81874
bridge0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500
ether 90:1b:0e:98:d3:93
inet 192.168.0.218 netmask 0xfffffff8 broadcast 192.168.0.223
nd6 options=3D9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=3D143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 7 priority 128 path cost 55
member: oce3 flags=3D143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000
root@bsdcan:~ # ifconfig bridge0 addr
00:08:e3:ff:fd:90 Vlan1 oce3 1200 flags=3D0<>
root@bsdcan:~ # ps aux | grep vmrun
root 47167 0.0 0.0 14828 2396 1 S+ 04:08 0:00.00 grep
vmrun
root 73264 0.0 0.0 13180 2740 2 I+ 03:39 0:00.00 sh
/usr/share/examples/bhyve/vmrun.sh -c 10 -m 8192M -t tap0 -d guest.img -i
-I FreeBSD-11.1-RELEASE-amd
root@bsdcan:~ # arp -an
? (192.168.0.218) at 90:1b:0e:98:d3:93 on bridge0 permanent [bridge]
? (192.168.0.222) at 00:08:e3:ff:fd:90 on bridge0 expires in 1191 seconds
[bridge]
root@bsdcan:~ # sysctl net.link.bridge
net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 1
net.link.bridge.inherit_mac: 1
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 1
net.link.bridge.pfil_member: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.pfil_onlyip: 1
--------------- end of host ----------
Shouldn't the VM mac address show up in the MAC address table of the
bridge0 ?
When I 'tcpdump -i tap0 -vv' I see literally only the ARP request from the
.222 towards the VM and nothing back from the VM at all which does not make
a lot of sense since I get the '-t tap0' when launching the VM, I would
expect some traffic on the tap0 intf from the VM.
Any thoughts on where filtering could happen?
I assume the VM should be able to ping the IP set on the bridge0. Is it
fair assumption?
Thanks.
On Mon, Aug 21, 2017 at 9:14 PM, Eugene Grosbein <eugen@grosbein.net> wrote=
:
> 22.08.2017 7:49, Boris =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
> > Hi all,
> >
> > I have two environments.
> >
> > Environment A:
> > Server running fresh install of 11.1-RELEASE with bge physical NIC.
> > If I just configure a bridge interface, add a physical NIC which has
> > working connectivity, say bge3, and add an IP address on the bridge
> > interface in the same subnet as bge3, I can ping that IP from any host =
on
> > the LAN.
> >
> > Environment B:
> > Server running fresh install of 11.1-RELEASE with oce physical NIC.
> > If I just configure a bridge interface, add a physical NIC which has
> > working connectivity, say oce3, and add an IP address on the bridge
> > interface in the same subnet as oce3, I CANNOT ping that IP from anywhe=
re
> > on the LAN.
>
> First, when you add member interfaces to a bridge, you should move all
> their
> IP addresses to the bridge. That is, bridge member interfaces should
> have no IP addresses, only bridge itself.
>
> Second, you should re-read bridge(4) manual page and use
> sysctl net.link.bridge.inherit_mac=3D1 and use physical NIC as first
> bridge member so that your uplink has no reasons to filter
> traffic of the bridge due to its fabricated MAC.
>
>
> >
> > I need the bridge as I would like to have bhyve VM's connected through
> that
> > bridge to the outside - plain bridged networking, no NAT or anything
> else.
> > Unfortunately, the VM does not have any connectivity to the outside. Wh=
at
> > is weird is that I see the ARP entries in the VM for its gateway, I see
> the
> > MAC addresses in the bridge for the VM and the gateway, but no IP
> > connectivity seems to work - ping fails.
> > I disabled TX checksum and other things using 'ifconfig oce3 -txcsum -l=
ro
> > -tso' to avoid messages around capabilities issues when adding a tap
> > interface which does not have the same features as the physical
> interface.
> > So far, I have not been able to get IP connectivity to the VM.
> >
> > In terms of documentation, I have used the handbook to create the VM:
> > https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html
> >
> > I have used the handbook to create the bridge:
> > https://www.freebsd.org/doc/handbook/network-bridging.html
> >
> > Under 30.6.1, it says, I should be able to configure the bridge with an
> IP
> > address which seems to fail when the 'oce' interface is used.
> >
> > Would anybody have any pointer at what to do next to help identify the
> > issue?
> >
> > Thanks !
> > _______________________________________________
> > freebsd-net@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
> >
> >
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJYdwgU13rBf%2BZ_UdL%2BQaBpgNQrLH8fVM3dEJy7pBP9Uv5xSgQ>