From owner-freebsd-current@freebsd.org Tue Jun 25 20:21:26 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8034D15D67C2; Tue, 25 Jun 2019 20:21:26 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (ns-b.lerctr.org [IPv6:2001:470:1f0f:3ad::53:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C72827486D; Tue, 25 Jun 2019 20:21:25 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=EdEml7v5uiDm/dIo6greBIcb7Mc2RV+wK+WKmyn2JZA=; b=YzuqQJ6PpdCB8I0+IpRLJTwUKn HaUfpWlbhMP7biQU0gmgO2ZFstay3XDXSMwR3ZhKyCl/jd3vK2/YDBR1ptH1K2vzNly5/J3cnOZJA OVF8uGPQtqMyRBSYJqQ/ORgMeCPqhti3uZ9yow93t7iucqDJf7DAyNfb2NbrPnmv3JkpbMgJ7gAu3 cAxJNY8csO4uwEtU3TgtFdGIpNACBF/4elugfbrkh95VBMDHB+20LSlT4jouRksitV8QcgFTmPLFL X109IFeX6WSMD6G0OpltEjeYtikrW7pquvlunPuZiZt03MxV++cIxIHoXaIcwiEnLsjFzoRpYekvD Gmfxk0Gw==; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:20421 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hfrwQ-0003IF-QE; Tue, 25 Jun 2019 15:21:22 -0500 Received: from 2600:1700:210:b180:618b:e19b:4ff6:4b3e by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 25 Jun 2019 15:21:22 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 25 Jun 2019 15:21:22 -0500 From: Larry Rosenman To: "Andrey V. Elsukov" Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: ng_snd_item: Panic? In-Reply-To: <7b4c697c-e924-deb8-d227-2d188b8f8483@yandex.ru> References: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> <7b4c697c-e924-deb8-d227-2d188b8f8483@yandex.ru> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.3.9 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 20:21:26 -0000 On 06/25/2019 3:13 pm, Andrey V. Elsukov wrote: > On 25.06.2019 15:59, Larry Rosenman wrote: >> On 06/25/2019 4:18 am, Andrey V. Elsukov wrote: >>> On 24.06.2019 23:10, Larry Rosenman wrote: >>>>>> #5  0xffffffff828ee5b7 in ng_snd_item (item=0xfffff8021e3b4d80, >>>>>> flags=0) >>>>>>     at /usr/src/sys/netgraph/ng_base.c:2252 >>>>> >>>>> It looks like you use some netgraph based ethernet interface. >>>>> The system got received ARP request and is going to send the reply, >>>>> but somehow mbuf with this ARP request has initialized m_next >>>>> pointer, >>>>> thus it is considered as a chain of mbufs. >>>>> >>>>> in_arpinput() reuses received mbuf to construct the reply, but it >>>>> doesn't check that an mbut is a chain. It just sets m_len and sends >>>>> it. >>>>> Then since you have INVARIANTS in your kernel, the netgraph code >>>>> check >>>>> the actual length of the chain, and it doesn't match to m_len. It >>>>> panics. >>>> >>>> >>>> so, is this a bug?  Timing race? Other? >>> >>> I think we should determine that my assumption is correct :) >>> Can you show the output of the following commands from the kgdb for >>> this >>> core? >>> >>> (kgdb) f 7 >>> (kgdb) p *m >>> (kgdb) p *m->m_next >> >> >> (kgdb) fr 7 >> #7  0xffffffff805b1e43 in ether_output (ifp=, >> m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=) at >> /usr/src/sys/net/if_ethersubr.c:430 >> 430            if ((error = (*ng_ether_output_p)(ifp, &m)) != 0) { > > I failed to track the possible way to get this. > Please, show the output of the following commands: > (kgdb) f 7 > (kgdb) p/x (u_char[42])m->m_data > (kgdb) p/x (u_char[1372]m->m_next->m_data > > Did you used this configuration for the long time and these panics were > the first time? (kgdb) f 7 #7 0xffffffff805b1e43 in ether_output (ifp=, m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=) at /usr/src/sys/net/if_ethersubr.c:430 430 if ((error = (*ng_ether_output_p)(ifp, &m)) != 0) { (kgdb) p/x (u_char[42])m->m_data $1 = {0x72, 0xfb, 0xee, 0x59, 0x1f, 0xf8, 0xff, 0xff, 0x2a, 0x0, 0x0, 0x0, 0x1, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x60, 0xd, 0x53, 0x4c, 0x10, 0xf8, 0xff, 0xff, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0} (kgdb) p/x (u_char[1372]m->m_next->m_data A syntax error in expression, near `m->m_next->m_data'. (kgdb) p/x (u_char[1372])m->m_next->m_data $2 = {0x0, 0xb8, 0x44, 0x21, 0x0, 0xf8, 0xff, 0xff, 0x5c, 0x5, 0x0, 0x0, 0x1, 0x1, 0x0 , 0x1, 0x0, 0x0, 0x0, 0xde, 0xc0, 0xad, 0xde, 0x0, 0xb8, 0x44, 0x21, 0x0, 0xf8, 0xff, 0xff, 0x0, 0x8, 0x0, 0x0, 0x6, 0x1, 0x0 , 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0x0 , 0x4c, 0xe0, 0xd6, 0x79, 0x19, 0xf8, 0xff, 0xff, 0xa8, 0x3, 0x0, 0x0, 0x1, 0x3...} (kgdb) I've been using this configuration for quite a few months, and these 2 panics yesterday are the only ones I've seen. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106