Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 24 Jun 2019 15:10:40 -0500
From:      Larry Rosenman <ler@lerctr.org>
To:        "Andrey V. Elsukov" <bu7cher@yandex.ru>
Cc:        freebsd-current@freebsd.org, freebsd-net@freebsd.org
Subject:   Re: ng_snd_item: Panic?
Message-ID:  <d1d85d35671198f1cb41f4f781a91587@lerctr.org>
In-Reply-To: <c3de35e2-0954-9811-8600-85e059c61464@yandex.ru>
References:  <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> <c3de35e2-0954-9811-8600-85e059c61464@yandex.ru>

next in thread | previous in thread | raw e-mail | index | archive | help
On 06/24/2019 3:01 pm, Andrey V. Elsukov wrote:
> 24.06.2019 21:32, Larry Rosenman пишет:
>> Got 2 of these today, and I have cores....
>> Ideas?
>> r349200.
>> 
>> Unread portion of the kernel message buffer:
>> panic: ng_snd_item: 42 != 1414
>> cpuid = 10
>> time = 1561382494
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
>> 0xfffffe012628d400
>> vpanic() at vpanic+0x19d/frame 0xfffffe012628d450
>> panic() at panic+0x43/frame 0xfffffe012628d4b0
>> ng_snd_item() at ng_snd_item+0x477/frame 0xfffffe012628d4f0
>> ng_ether_output() at ng_ether_output+0x5e/frame 0xfffffe012628d520
>> ether_output() at ether_output+0x473/frame 0xfffffe012628d5c0
>> arpintr() at arpintr+0xfe3/frame 0xfffffe012628d780
>> netisr_dispatch_src() at netisr_dispatch_src+0x89/frame 
>> 0xfffffe012628d7f0
>> ether_demux() at ether_demux+0x137/frame 0xfffffe012628d820
>> ng_ether_rcv_upper() at ng_ether_rcv_upper+0x95/frame 
>> 0xfffffe012628d840
>> ng_apply_item() at ng_apply_item+0xf1/frame 0xfffffe012628d8c0
>> ng_snd_item() at ng_snd_item+0x2ab/frame 0xfffffe012628d900
>> ng_apply_item() at ng_apply_item+0xf1/frame 0xfffffe012628d980
>> ng_snd_item() at ng_snd_item+0x2ab/frame 0xfffffe012628d9c0
>> ng_ether_input() at ng_ether_input+0x4c/frame 0xfffffe012628d9f0
>> ether_nh_input() at ether_nh_input+0x2cd/frame 0xfffffe012628da40
>> netisr_dispatch_src() at netisr_dispatch_src+0x89/frame 
>> 0xfffffe012628dab0
>> ether_input() at ether_input+0x48/frame 0xfffffe012628dad0
>> bce_intr() at bce_intr+0x697/frame 0xfffffe012628db50
>> ithread_loop() at ithread_loop+0x187/frame 0xfffffe012628dbb0
>> fork_exit() at fork_exit+0x84/frame 0xfffffe012628dbf0
>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe012628dbf0
>> --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
>> Uptime: 4d18h45m34s
>> Dumping 24921 out of 131026 
>> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>> 
>> #5  0xffffffff828ee5b7 in ng_snd_item (item=0xfffff8021e3b4d80, 
>> flags=0)
>>     at /usr/src/sys/netgraph/ng_base.c:2252
> 
> It looks like you use some netgraph based ethernet interface.
> The system got received ARP request and is going to send the reply,
> but somehow mbuf with this ARP request has initialized m_next pointer,
> thus it is considered as a chain of mbufs.
> 
> in_arpinput() reuses received mbuf to construct the reply, but it
> doesn't check that an mbut is a chain. It just sets m_len and sends it.
> Then since you have INVARIANTS in your kernel, the netgraph code check
> the actual length of the chain, and it doesn't match to m_len. It 
> panics.


so, is this a bug?  Timing race? Other?

[I] ➜ sudo ngctl list
There are 7 total nodes:
   Name: bce0            Type: ether           ID: 00000001   Num hooks: 
2
   Name: bce1            Type: ether           ID: 00000002   Num hooks: 
0
   Name: bce2            Type: ether           ID: 00000003   Num hooks: 
0
   Name: bce3            Type: ether           ID: 00000004   Num hooks: 
0
   Name: netflow         Type: netflow         ID: 00000006   Num hooks: 
3
   Name: <unnamed>       Type: ksocket         ID: 00000007   Num hooks: 
1
   Name: ngctl3705       Type: socket          ID: 00000008   Num hooks: 
0

ler in ~ at borg
[I] ➜


> 
>> #6  0xffffffff82900c2e in ng_ether_output (ifp=<optimized out>,
>>     mp=0xfffffe012628d578) at /usr/src/sys/netgraph/ng_ether.c:294
>> #7  0xffffffff805b1e43 in ether_output (ifp=<optimized out>,
>>     m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=<optimized out>)
>>     at /usr/src/sys/net/if_ethersubr.c:430
>> #8  0xffffffff805cb3e3 in in_arpinput (m=<optimized out>)
>>     at /usr/src/sys/netinet/if_ether.c:1152
>> #9  arpintr (m=0xfffff81f59eefb00) at 
>> /usr/src/sys/netinet/if_ether.c:749
>> #10 0xffffffff805bcf89 in netisr_dispatch_src (proto=4,
>>     source=<optimized out>, m=<unavailable>) at 
>> /usr/src/sys/net/netisr.c:1123
>> #11 0xffffffff805b22d7 in ether_demux (ifp=0xfffff8012c902000,
>>     m=<unavailable>) at /usr/src/sys/net/if_ethersubr.c:913
>> #12 0xffffffff82901045 in ng_ether_rcv_upper (hook=<optimized out>,
>>     item=<optimized out>) at /usr/src/sys/netgraph/ng_ether.c:741
>> #13 0xffffffff828ee6e1 in ng_apply_item (node=0xfffff81054f43400,
>>     item=0xfffff8021e3b4d80, rw=0) at 
>> /usr/src/sys/netgraph/ng_base.c:2403
>> #14 0xffffffff828ee3eb in ng_snd_item (item=0xfffff8021e3b4d80, 
>> flags=0)
>>     at /usr/src/sys/netgraph/ng_base.c:2320
>> #15 0xffffffff828ee6e1 in ng_apply_item (node=0xfffff8012c2d3e00,
>>     item=0xfffff8021e3b4d80, rw=0) at 
>> /usr/src/sys/netgraph/ng_base.c:2403
>> #16 0xffffffff828ee3eb in ng_snd_item (item=0xfffff8021e3b4d80, 
>> flags=0)
>>     at /usr/src/sys/netgraph/ng_base.c:2320
>> #17 0xffffffff82900cbc in ng_ether_input (ifp=<optimized out>,
>>     mp=0xfffffe012628da18) at /usr/src/sys/netgraph/ng_ether.c:255
>> #18 0xffffffff805b34fd in ether_input_internal 
>> (ifp=0xfffff8012c902000,
>>     m=0xfffff81f59eefb00) at /usr/src/sys/net/if_ethersubr.c:654
>> #19 ether_nh_input (m=<optimized out>) at 
>> /usr/src/sys/net/if_ethersubr.c:735
>> #20 0xffffffff805bcf89 in netisr_dispatch_src (proto=5,
>>     source=<optimized out>, m=<unavailable>) at 
>> /usr/src/sys/net/netisr.c:1123
>> #21 0xffffffff805b26f8 in ether_input (ifp=0xfffff8012c902000, m=0x0)
>>     at /usr/src/sys/net/if_ethersubr.c:823
>> #22 0xffffffff8273c7f7 in bce_rx_intr (sc=<optimized out>)
>>     at /usr/src/sys/dev/bce/if_bce.c:6848
>> #23 bce_intr (xsc=0xfffffe01665c2000) at 
>> /usr/src/sys/dev/bce/if_bce.c:8017
>> #24 0xffffffff8047e0e7 in intr_event_execute_handlers (p=<optimized 
>> out>,
>>     ie=<optimized out>) at /usr/src/sys/kern/kern_intr.c:1148
>> #25 ithread_execute_handlers (p=<optimized out>, ie=<optimized out>)
>>     at /usr/src/sys/kern/kern_intr.c:1161
>> #26 ithread_loop (arg=<optimized out>) at 
>> /usr/src/sys/kern/kern_intr.c:1241
>> #27 0xffffffff8047ac74 in fork_exit (
>>     callout=0xffffffff8047df60 <ithread_loop>, arg=0xfffff8012c883100,
>>     frame=0xfffffe012628dc00) at /usr/src/sys/kern/kern_fork.c:1056
>> #28 <signal handler called>
>> (kgdb)
>> 

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d1d85d35671198f1cb41f4f781a91587>