Date: Sun, 25 Mar 2018 22:13:41 +0700 From: Eugene Grosbein <eugen@grosbein.net> To: supportsobaka@mail.ru, freebsd-net@freebsd.org, freebsdnic@mailbox.intel.com Subject: Re: FreeBSD 11.1 with Intel(R) PRO/1000 unresponsive to all network interfaces from outside (seems, during idle), but immediately up when ping anything from inside the server Message-ID: <5AB7BCA5.2020201@grosbein.net> In-Reply-To: <1521974300.464728846@f345.i.mail.ru> References: <1521974300.464728846@f345.i.mail.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
25.03.2018 17:38, supportsobaka--- via freebsd-net wrote: > Hello guys, > Need help with issues I never met before for my long experience with FreeBSD. > > A new server in remote DC based of Intel S1200RP with FreeBSD 11.1 uses igb driver for Intel(R) PRO/1000. There is no any load or traffic yet, I'm just configuring it, so I believe that my Kitty (Putty) session is the only one that makes traffic. I lost connection to the server dozens times during last week. > > I never lost connection when I was doing something on the server via remote Kitty terminal, but it was always when I return back to Kitty after some idle. Then, I kicked out of terminal and server doesn't response to pings or 'telnet port' from anywhere. > > The server has IPMI (and so KVM) and I now can see that the server is live and network interface is up. No messages in dmesg when this happens. The network goes up (i.e. pings go trough from outside) immediately after I ping something from inside the server (via IPMI's KVM access) or immediately after I execute netstat -r. > > I now run GENERIC to exclude any issue with my own kernel. > > The problem is 100% repeatable right now while I'm writing this: > > 1) leave Kitty terminal for a period of time (about 10 minutes enough) > 2) come back to terminal, start typing, got kicked off, ping - no response > 3) login to server via KVM (I'm already logged in) and ping any URL from there > 4) server is responsive again > > I run continuous ping to this server last nigh and it never dropped. It looks to me like Intel card goes to some sleep mode during idle (when no traffic comes to the server at all, except Kitty's keep-alive perhaps). > > This is my first experience with FreeBSD 11.1 and ZFS (include root from ZFS). All my previous servers are on FreeBSD 9 and UFS, but not the first with Intel cards. Not sure if filesystem matter in this issue. > > I tried some things described here https://forums.freebsd.org/threads/workaround-freebsd-10-1-sudden-network-down.49264/ - it doesn't help. > > What else information do you need to debug this? It might be that network of your DC provider has famous bug: sometimes its MAC/ARP cache expires MAC address of your machine and does not re-ask it using ARP protocol nor delivers a packet to the server. When you run ping or netstat -r you make some outgoing traffic (ICMP for ping and DNS for netstat) so you forcibly re-fill MAC/ARP caches of DC provider and now things come to normal for some time. There is an easy way to check if this is the case. You can change sysctl net.link.ether.inet.max_age parameter to some low value like 60 (seconds), so your own ARP cache for gateway's MAC address would expire often producing outgoing ARP request that re-fills caches of DC provider too, before it expires. If this helps - use it as workaround and bug DC provider.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5AB7BCA5.2020201>