From owner-freebsd-questions@freebsd.org Wed Jul 12 19:43:55 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6D91DA1BC2; Wed, 12 Jul 2017 19:43:55 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 241F874BD7; Wed, 12 Jul 2017 19:43:54 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from thor.intern.walstatt.dynvpn.de ([77.180.71.3]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LbM2k-1dxWDw1jzu-00ktnw; Wed, 12 Jul 2017 21:43:46 +0200 Date: Wed, 12 Jul 2017 21:43:34 +0200 From: "O. Hartmann" To: FreeBSD CURRENT , FreeBSD Questions Subject: Inter-VLAN routing on CURRENT: any known issues? Message-ID: <20170712214334.4fc97335@thor.intern.walstatt.dynvpn.de> Organization: WALSTATT User-Agent: OutScare 3.1415926 X-Operating-System: ImNotAnOperatingSystem 3.141592527 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/hakecsp8FO_KOq4QSAYnpJl"; protocol="application/pgp-signature" X-Provags-ID: V03:K0:unxbJaZ8sAx0ZhWd4l1EvuV1X00rsh8BW6cvKtD/dd3F6NI0qD2 0I6R89BjuFQk1W1oyk3av6dPc2Q4D3a6FmoLcvaf+5zjkLZoHgvv+7tp9PA0WhF/vS/arfm sqjOGEwgte1WKqHJlpkAiDsjnJFU7HlrWEhVUbpUHXY2UEMQewYHSxGyt5ojNG+YJzvBjEe ud96Z3lbL8uZv4y2f+2EA== X-UI-Out-Filterresults: notjunk:1;V01:K0:pS61UUN7p2Q=:qa/4yIdC3DH9a1KAPOnHm4 R95LAW5wxE+4HbnP6QxEFwafpMgbzbLI0+rG1mdQEn5d9X3maOl7Zr9rEnih4JEl7a9462d0e e508jboTWpZ440ocLNMkIqoWnkr5SNT5Y5WTv+8oze4PjQ3bbqbWzQ6Z4aSUg1Nfp0w+ijaj8 5n2YAQpP3gC7VR2y7OVJwEPX6Lfg8cEUc9XlZ6J3+N5kdNjO+ntEYtWvLh7OevP0YyH94R1Hf uKLYjcRFmpWhrCUgU4a5y5TrPU1ehj691wrPdqxCxpfi3kRow+oUVEnn+Q2L2chorfLlCEP+v Dme6NzjeddkPDbazD8fgi7ikusBGniJOc71fez7xAejrfKiqd6PNOtPEH51BzW9qWKiaoiIEv 4PLnEOwfqfDjz5VCEn8bfrOTCKOo4nFrdbYuIT+0LNC+GFOXBwPZuNbuO2HKih0PsdaGf50qF n57qYO83lV16tecX0hD6D8iqtQMAWsZDAZ6MVD8dDXBXMD6MOoIv/aNgzB/NVCtJkTpKtaqd1 aXhEwI/hZqUoaOSpdRyWbCUOPpEuvVwCO/SjN1vCRObSDJ3cTJiilo3WPeMqqFpdqQQVaQycj ah34liJE6hVqnJWcRG50kccp7CkPsEVa2FR54HVE5HZajQdAlvF9T+ge3NS1irMyaDOwRAdFb E0T/L63pCrXy1EAA3r1LDRFB8VQ9YQ14m0GUsIsUtz2c14EnooBhYUAnEDbLMPCJ7DJTokygG BgkquYVBWOd6BD7CmG7oRVBhrqUaETGFI0kYhF2gIB10qq/xuPtCZBf1JVo= X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jul 2017 19:43:55 -0000 --Sig_/hakecsp8FO_KOq4QSAYnpJl Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Since a couple of days for now I fail to setup VLAN trunking on a FreeBSD 1= 2-CURRENT box (FreeBSD 12.0-CURRENT #9 r320913: Wed Jul 12 17:26:22 CEST 2017 amd64) whic= h is based on a PCEngines APU 2C4 board with three Intel i210 NICs. igb0 is connected to a Allnet VDSL modem via tun0/ppp. igb2 is unused. igb1 is considered "multihomed" and comprises several VLANs: [/etc/rc.conf] gateway_enable=3D"YES" ... ifconfig_igb1=3D"up" vlans_igb1=3D"1000 2 3 10 66 100" ifconfig_igb1_1000=3D"inet 192.168.0.1/24" create_args_igb1_1000=3D"vlanpcp 7" ifconfig_igb1_2=3D"inet 192.168.2.1/24" ifconfig_igb1_3=3D"inet 192.168.3.1/24" ifconfig_igb1_10=3D"inet 192.168.10.1/24" ifconfig_igb1_66=3D"inet 192.168.66.1/24" ifconfig_igb1_100=3D"inet 192.168.100.1/24" ... VLAN 1000 is considered my internal network, the others are for special pur= pose, e.g. VLAN 2 is for VoIP equiment. After booting (a customised) kernel the router shows the following settings: root@gate:~ # netstat -Warn Routing tables Internet: Destination Gateway Flags Use Mtu Netif Exp= ire default 111.111.111.111 US 570 1492 tun0 111.111.111.111 link#12 UHS 0 1492 tun0 22.33.44.55 link#12 UHS 0 16384 lo0 127.0.0.1 link#4 UH 115 16384 lo0 192.168.0.0/24 link#2 U 13930 1500 igb1.1000 192.168.0.1 link#2 UHS 0 16384 lo0 192.168.2.0/24 link#7 U 1 1500 igb1.2 192.168.2.1 link#7 UHS 0 16384 lo0 192.168.3.0/24 link#8 U 0 1500 igb1.3 192.168.3.1 link#8 UHS 0 16384 lo0 192.168.10.0/24 link#9 U 0 1500 igb1.10 192.168.10.1 link#9 UHS 0 16384 lo0 192.168.66.0/24 link#10 U 0 1500 igb1.66 192.168.66.1 link#10 UHS 0 16384 lo0 192.168.100.0/24 link#11 U 0 1500 igb1.100 192.168.100.1 link#11 UHS 0 16384 lo0 All interfaces (including vlan) show "UP" in their status.=20 sshd, named and services are bound on the router to 192.168.0.1, which is i= ts IP. The router's igb1-NIC is physically connected to a SoHo switch Netgear GS11= 0TP. Its config in short according to the manual (http://www.netgear.com/support/product/GS110TP.aspx#docs , chapter 3, pagu= s 84) is as follows. Port gs9 is considered the trunk/etherchannel port (via GBIC 1 Gig). Accord= ingly to my setup, the VLANs 1,2,3 (switch-native),10, 66, 100 and 1000 are defined. In= VLAN membership configuration for VLAN 1, only port g1 is marked "U", this is my maintenance port. For VLAN 1000 ports g1-g4 are "U" untagged, g9 is "T" tag= ged. For VLAN 2, port g7 is "U", g8 is "T" (the VoIP telephone has vlan tag 2) and the tr= unk is g9 "T". VLAN 100 occupies port g5 "U", port g9 is "T". The other VLANs are unused a= t the moment. According to handbook section "Port VLAN ID Configuration" (PVID), g1-g4 ar= e PVID 1000, Accept. Frame Type is "Admit All" and Ingress Filtering is "disabled". The = settings for the other so called "access ports" are accordingly.=20 g9, the trunk port, has PVID 1, Admit all, Ingress Filtering is disabled. O= ther configurations are mostly as the switch is set-up after factory reset. On ports g1 - g4 I have a dual-port NIC'ed server (one port vlan 1000, othe= r vlan 100) running and a notebook, which I can configure freely. Now the FUN PART: =46rom any host in any VLAN I'm able to ping hosts on the wild internet via t= heir IP, on VLAN 1000 there is a DNS running, so I'm also able to resolv names like goo= gle.com or FreeBSD.org. But I can NOT(!) access any host via http/www or ssh.=20 I also can not access a host's sshd in the neighbour VLAN routed via the ro= uter, say from a host/server on VLAN 1000 to a host/VoIP telephone on VLAN 2. I can p= ing the hosts from each VLAN to the other (so ICMP flows), but any IP service seems to ge= t sacked by a black hole. From hosts on VLAN 1000 I can access the router's sshd (192.168= .0.1). More disturbing: from the router itself, I'm able to access the sshd of eac= h host on each VLAN, i.e. VLAN 1000, VLAN 2 (VoIP), but when setting up a notebook (F= reeBSD 12-CURRENT of the same or similar revision) in VLAN 2 or VLAN 100 or VLAN 6= 6 with SSHD listening on all interfaces, I'm able to connect to that system. Also, from= the router itself, I can ping any host on any VLAN and the internet (routed via tun0/i= gb0/modem). =46rom any host on any VLAN, I can ping the router, I can ping the world, I c= an ping other hosts on other VLANs. Obviously, ICMP is routed. Any attempt to access a service from a host in any VLAN to a hosts's servic= e on another VLAN fails. IP is not routed and I do not see why. The kernel is compiled with in-kernel IPFW. No matter what I do, either ipf= w "OPEN" or using my ruleset which works in the special case I describe later, routing = through VLANs seems not to work for any IP packet! Using tcpdump on the router while trying to ssh into another host, I see th= e initial [S] marked attempt to connect, i.e. 192.168.0.128 > 192.168.2.50: [S]. Onece th= e packet has been sent from sender to the router, I never is passed to the recipient.=20 Before I start attempting making weird speculations, I must confess that us= ing tcpdump and other network tools is not my favourite and I'm quite new/novice on tha= t field.=20 I need advice. Also, I need to know whether the setup I showed is working o= r whether I make a serious and stupid mistake (maybe due to not having understood FreeB= SD's routing or routing at all).=20 If on the setup shown above the VLAN is dumped and when I use only igb1 as = the "vanilla" NIC, everything works smoothly - execpt the fact I do not have network sepa= rations. But it shows me that in principle the complete setup isn't complete bullshit. F= rom that persepctive, even just changing igb1 to igb1.1000 (a tagged VLAN), it shoul= d work. But it doesn't. I'm not sure whether IPFW is the culprit or not or anothe knob, for the rec= ord, these settings are for ipfw in the kernel: [...] options NETGRAPH # netgraph(4) system options NETGRAPH_IPFW options NETGRAPH_NETFLOW options NETGRAPH_ETHER options NETGRAPH_NAT options NETGRAPH_DEVICE options NETGRAPH_PPPOE options NETGRAPH_SOCKET options NETGRAPH_ASYNC options NETGRAPH_TEE # IPFW firewall options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=3D0 options IPFIREWALL_NAT #ipfw kernel nat support options LIBALIAS #ipfw kernel nat support options IPDIVERT # NAT options DUMMYNET # traffic shaper # #options IPFIREWALL_DEFAULT_TO_ACCEPT [...] and from sysctl: kern.features.ipfw_ctl3: 1 net.link.ether.ipfw: 0 net.link.bridge.ipfw: 0 net.link.bridge.ipfw_arp: 0 So, if someone is willing to give me some hints, I'd be glad to hear from y= ou. I'm starting getting insane over this problem :-( Kind regards and thanks for your patience, Oliver --=20 O. Hartmann Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Abs.= 4 BDSG). --Sig_/hakecsp8FO_KOq4QSAYnpJl Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWWZ75gAKCRDS528fyFhY lFDtAgCQfzw2Q1HpqjWeMDZDW52syCjmmcheUuOGUqKgikc+Dr1WYUMvhVM+FCkJ Thwef0zCavLdiTbUyf70hs8t3K24AfsHzAn3QMxr+XSgFsyATR1GEmPOjOF3tF/N sfMYty1efOBxW1FwjecyzvSoLu2yEyENt7ZnavjLTRYE8j5xn7tr =t2CV -----END PGP SIGNATURE----- --Sig_/hakecsp8FO_KOq4QSAYnpJl--