Date: Thu, 3 Mar 2016 23:00:25 +0300 From: Pakhom Golynga <pakhom706@gmail.com> To: freebsd-net@FreeBSD.org Subject: routing issue Message-ID: <56D897D9.2040502@gmail.com>
next in thread | raw e-mail | index | archive | help
Hello all!
Please help me to investigate this issue.
I have problem on 10.2-RELEASE-p12 with multiple network interfaces and
PF (rules, NAT)
# ifconfig
<--cut-->
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:64:14:50
inet 172.27.27.254 netmask 0xffffff00 broadcast 172.27.27.255
inet 172.27.27.240 netmask 0xffffffff broadcast 172.27.27.240
inet 172.27.27.252 netmask 0xffffffff broadcast 172.27.27.252
inet 172.27.27.251 netmask 0xffffffff broadcast 172.27.27.251
inet 172.27.27.250 netmask 0xffffffff broadcast 172.27.27.250
inet 172.27.27.249 netmask 0xffffffff broadcast 172.27.27.249
inet 172.27.27.248 netmask 0xffffffff broadcast 172.27.27.248
inet 172.27.27.247 netmask 0xffffffff broadcast 172.27.27.247
inet 172.27.27.246 netmask 0xffffffff broadcast 172.27.27.246
inet 172.27.27.245 netmask 0xffffffff broadcast 172.27.27.245
inet 172.27.27.244 netmask 0xffffffff broadcast 172.27.27.244
inet 172.27.27.243 netmask 0xffffffff broadcast 172.27.27.243
inet 172.27.27.242 netmask 0xffffffff broadcast 172.27.27.242
inet 172.27.27.241 netmask 0xffffffff broadcast 172.27.27.241
inet 172.27.27.239 netmask 0xffffffff broadcast 172.27.27.239
inet 172.27.27.238 netmask 0xffffffff broadcast 172.27.27.238
inet 172.27.27.237 netmask 0xffffffff broadcast 172.27.27.237
inet 172.27.27.236 netmask 0xffffffff broadcast 172.27.27.236
inet 172.27.27.235 netmask 0xffffffff broadcast 172.27.27.235
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
mtu 1500
options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:64:14:51
inet 10.24.44.50 netmask 0xfffffc00 broadcast 10.24.47.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
<--cut-->
em0 - local network. Aliases on em0 for Jail's
em1 - connected to ISP
This configuration successfully operate a long time but recently (for no
apparent reason) suddenly became manifest bug with routing.
Regardless from uptime (sometimes 12 hours, and maybe a couple of days)
after a certain time the traffic running between jails begins to route
via em1 (instead lo0 (net.link.ether.inet.useloopback=1)):
# tcpdump -n -e -ttt -i em1 host 172.27.27.252 or host 172.27.27.247 or
host 172.27.27.248
00:00:00.054989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: Flags [S],
seq 2075254199, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
161568673 ecr 0], length 0
00:00:00.318036 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.60176: Flags [.],
ack 772660872, win 1275, options [nop,nop,TS val 1737287913 ecr
161568991], length 0
00:00:00.036971 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.33080: Flags [.],
ack 1, win 1275, options [nop,nop,TS val 1701623130 ecr 161568238], length 0
00:00:00.050001 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.40200: Flags [.],
ack 1, win 1275, options [nop,nop,TS val 1477098721 ecr 161568288], length 0
00:00:02.794992 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: Flags [S],
seq 2075254199, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
161571873 ecr 0], length 0
00:00:03.312026 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25: Flags [S],
seq 1217253021, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
161575185 ecr 0], length 0
00:00:02.887982 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: Flags [S],
seq 2075254199, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
161578073 ecr 0], length 0
00:00:00.111989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25: Flags [S],
seq 1217253021, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
161578185 ecr 0], length 0
00:00:03.471007 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype IPv4
(0x0800), length 108: 172.27.27.247.80 > 172.27.27.248.26056: Flags
[FP.], seq 1448:1490, ack 1, win 1275, options [nop,nop,TS val
2142270801 ecr 161575507], length 42
In the same time:
# route -vn get -host
RTA_DST: inet 172.27.27.247; RTA_IFP: link ; RTM_GET: Report Metrics:
len 224, pid: 0, seq 1, errno 0, flags:<UP,GATEWAY,HOST,STATIC>
locks: inits:
sockaddrs: <DST,IFP>
172.27.27.247 link#0
route to: 172.27.27.247
destination: 172.27.27.247
fib: 0
interface: lo0
flags: <UP,HOST,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 16384 1 0
locks: inits:
sockaddrs: <DST,GATEWAY,IFP,IFA>
172.27.27.247 link#4 lo0 127.0.0.1
RTA_DST: inet 172.27.27.250; RTA_IFP: link ; RTM_GET: Report Metrics:
len 224, pid: 0, seq 1, errno 0, flags:<UP,GATEWAY,HOST,STATIC>
locks: inits:
sockaddrs: <DST,IFP>
172.27.27.250 link#0
route to: 172.27.27.250
destination: 172.27.27.250
fib: 0
interface: lo0
flags: <UP,HOST,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 16384 1 0
locks: inits:
sockaddrs: <DST,GATEWAY,IFP,IFA>
172.27.27.250 link#4 lo0 127.0.0.1
RTA_DST: inet 172.27.27.252; RTA_IFP: link ; RTM_GET: Report Metrics:
len 224, pid: 0, seq 1, errno 0, flags:<UP,GATEWAY,HOST,STATIC>
locks: inits:
sockaddrs: <DST,IFP>
172.27.27.252 link#0
route to: 172.27.27.252
destination: 172.27.27.252
fib: 0
interface: lo0
flags: <UP,HOST,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 16384 1 0
locks: inits:
sockaddrs: <DST,GATEWAY,IFP,IFA>
172.27.27.252 link#4 lo0 127.0.0.1
# netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 10.24.44.1 UGS em1
10.24.44.0/22 link#5 U em1
10.24.44.50 link#5 UHS lo0
127.0.0.1 link#7 UH lo0
172.27.27.0/24 link#4 U em0
172.27.27.235 link#4 UHS lo0
172.27.27.235/32 link#4 U em0
172.27.27.236 link#4 UHS lo0
172.27.27.236/32 link#4 U em0
172.27.27.237 link#4 UHS lo0
172.27.27.237/32 link#4 U em0
172.27.27.238 link#4 UHS lo0
172.27.27.238/32 link#4 U em0
172.27.27.239 link#4 UHS lo0
172.27.27.239/32 link#4 U em0
172.27.27.240 link#4 UHS lo0
172.27.27.240/32 link#4 U em0
172.27.27.241 link#4 UHS lo0
172.27.27.241/32 link#4 U em0
172.27.27.242 link#4 UHS lo0
172.27.27.242/32 link#4 U em0
172.27.27.243 link#4 UHS lo0
172.27.27.243/32 link#4 U em0
172.27.27.244 link#4 UHS lo0
172.27.27.244/32 link#4 U em0
172.27.27.245 link#4 UHS lo0
172.27.27.245/32 link#4 U em0
172.27.27.246 link#4 UHS lo0
172.27.27.246/32 link#4 U em0
172.27.27.247 link#4 UHS lo0
172.27.27.247/32 link#4 U em0
172.27.27.248 link#4 UHS lo0
172.27.27.248/32 link#4 U em0
172.27.27.249 link#4 UHS lo0
172.27.27.249/32 link#4 U em0
172.27.27.250 link#4 UHS lo0
172.27.27.250/32 link#4 U em0
172.27.27.251 link#4 UHS lo0
172.27.27.251/32 link#4 U em0
172.27.27.252 link#4 UHS lo0
172.27.27.252/32 link#4 U em0
172.27.27.254 link#4 UHS lo0
172.27.28.0/24 link#8 U bridge0
172.27.28.254 link#8 UHS lo0
172.27.30.0/24 172.27.30.2 UGS tun0
172.27.30.1 link#12 UHS lo0
172.27.30.2 link#12 UH tun0
After the reboot everything works well for some time.
Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56D897D9.2040502>