Date: Thu, 12 Jan 2017 15:13:45 +0100 From: Jakub Palider <jpa@semihalf.com> To: Gleb Smirnoff <glebius@freebsd.org> Cc: hiren panchasara <hiren@strugglingcoder.info>, freebsd-net@freebsd.org, karels@freebsd.org Subject: Re: FLOWTABLE aka TCP route caching panic Message-ID: <CAL7QUyORo=FiSJNE9TDQ7CGSfWBueG_4Hi_H=G83g3VYD1=y-A@mail.gmail.com> In-Reply-To: <CAL7QUyO0uQoF-01jJo7Nf5PHz2eubZf0WwjM6yT=qp2XS-Zhyg@mail.gmail.com> References: <CAL7QUyNntDmJuxfDxncxdXSM1rvR47kizA=WYwqeAtwrSCFLUg@mail.gmail.com> <20161128221033.GM55731@strugglingcoder.info> <20161129190509.GO27748@FreeBSD.org> <CAL7QUyO0uQoF-01jJo7Nf5PHz2eubZf0WwjM6yT=qp2XS-Zhyg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
An update: eventually, FLOWTABLE option also resulted in crash with __rw_lock_hard on FreeBSD release/11.0.0 (also on VM). This time, however, the backtrace was different, instead of tcp_output->ip_output->ether_output() chain, it happened in flowtable_clean_vnet() On Wed, Jan 11, 2017 at 10:42 AM, Jakub Palider <jpa@semihalf.com> wrote: > On Tue, 29 Nov 2016 11:05:09 -0800 > Gleb Smirnoff <glebius@FreeBSD.org> wrote: > > > On Mon, Nov 28, 2016 at 02:10:33PM -0800, hiren panchasara wrote: > > h> > Hi, > > h> > I have found that last month (19 Oct) the problem appeared on this > list, > > h> > and to my experience it persists, both on VM and bare metal > installation > > h> > (HEAD from yesterday). I looks that enabling FLOWTABLE option is > the only > > h> > source of this fault happening. It appears on our setup in 80% > cases within > > h> > one hour from boot up. > > h> > From our debugging, it is caused by lock on DESTROYED lock. Did you > find a > > h> > solution to this problem? > > > > Not yet. > > > > I'm pretty sure that reverting my r307234 will fix your crashes. > However, I still > > believe that r307234 is a proper way of fixing things, not r300854 which > just > > plugged the problem in the nearest place to the crash. But as we all see > r307234 > > is definitely missing some code path, which still allows for stale route > to be > > referenced. > > > > -- > > Totus tuus, Glebius. > > Thank you for your pointer, in helped at some point. The problem > however returned, especially when under heavy, continuous load. We are > running 4 iperf3 processes, each having 4 threads, and the machine > dies after 30-60 minutes of TCP traffic. I am running rS30911112 from > Nov 24. > On FreeBSD 11.0-RELEASE this problem (1 full day of testing) was not > noticed. Could you point to related commits that might also influence > this behaviour? > Thanks, > Jakub >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAL7QUyORo=FiSJNE9TDQ7CGSfWBueG_4Hi_H=G83g3VYD1=y-A>