Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 05 Jul 2011 21:11:45 +0400
From:      "Alexey V. Panfilov" <ports@subnets.ru>
To:        Arnaud Lacombe <lacombar@gmail.com>
Cc:        net@freebsd.org
Subject:   Re: Netgraph udp tunneling
Message-ID:  <4E1345D1.5080903@subnets.ru>
In-Reply-To: <CACqU3MUKebhJ3ZXMxDRMAKOXpXp2jiXNYf0Qo91qcbPE%2BKUV2A@mail.gmail.com>
References:  <4E11A8D3.40102@subnets.ru> <CACqU3MUKebhJ3ZXMxDRMAKOXpXp2jiXNYf0Qo91qcbPE%2BKUV2A@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
05.07.2011 20:57, Arnaud Lacombe wrote:
> Hi,
>
> 2011/7/4 Alexey V. Panfilov<ports@subnets.ru>:
>> Hi!
>>
>> We've three servers, connected as S1<-ng_tunnel->  S2<-ng_tunnel->  S3.
>>
>> Over tunnels runs BGP with full-view.
>>
>> Sometimes on S2 occures fatal trap with automatic reboot or without reboot
>> at all. It not depends of net loading (mbps or pps).
>>
>> If process of save a core was successfull, backtrace always shows that fatal
>> trap occures because of large packet (size around 60Kbyte) was received.
>>
>>
>> Any help are welcome. Thanks.
>>
>> ----------------------------------------------------------------------
>>
>> Info about S2:
>>
>> smbios.system.maker="IBM"
>> smbios.system.product="System x3250 M3 -[425232G]-"
>> hw.model: Intel(R) Core(TM) i3 CPU         530  @ 2.93GHz
>> hw.physmem: 4207792128
>> dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.1.9
>> dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x1014
>> subdevice=0x03bd class=0x020000
>>
>> NOTE: On S2 used only em0
>>
>> /etc/sysctl.conf:
>> net.inet.icmp.icmplim=50
>> net.inet.tcp.blackhole=2
>> net.inet.udp.blackhole=1
>> net.inet.icmp.drop_redirect=1
>> net.inet.icmp.log_redirect=0
>> net.inet.ip.redirect=0
>>
>> Netgraph's tunnels are configured such as it wrote at example
>> (/usr/share/examples/netgraph/udp.tunnel):
>>
>> ngctl mkpeer iface dummy inet
>> ngctl mkpeer ng0: ksocket inet inet/dgram/udp
>> ngctl name ng0:inet to_S1
>> ngctl msg ng0:inet bind inet/1.1.1.1:60001
>> ngctl msg ng0:inet connect inet/5.5.5.5:60001
>> ifconfig ng0 10.0.0.1 10.0.0.2 netmask 255.255.255.252
>>
>> ngctl mkpeer iface dummy inet
>> ngctl mkpeer ng1: ksocket inet inet/dgram/udp
>> ngctl name ng1:inet to_S3
>> ngctl msg ng1:inet bind inet/1.1.1.3:60002
>> ngctl msg ng1:inet connect inet/7.7.7.7:60002
>> ifconfig ng1 10.0.0.5 10.0.0.6 netmask 255.255.255.252
>>
>> FreeBSD S2.line 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #1: Wed Jun 22
>> 13:56:26 MSD 2011     root@S2.line:/usr/src/sys/amd64/compile/BGP  amd64
>>
>> Hardware and software configurations on S1 and S3 are identical to S2, but
>> they runs without problem.
>>
>> ------------------------------------------------------------------------
>>
>> backtrace:
>>
>> Unread portion of the kernel message buffer:
>>
>>
>> Fatal trap 12: page fault while in kernel mode
>> cpuid = 3; apic id = 05
>> fault virtual address   = 0x18
>> fault code              = supervisor read data, page not present
>> instruction pointer     = 0x20:0xffffffff803f4a87
>> stack pointer           = 0x28:0xffffff811c80a5a0
>> frame pointer           = 0x28:0xffffff811c80a600
>> code segment            = base 0x0, limit 0xfffff, type 0x1b
>>                         = DPL 0, pres 1, long 1, def32 0, gran 1
>> processor eflags        = interrupt enabled, resume, IOPL = 0
>> current process         = 1478 (ng_queue0)
>> trap number             = 12
>> panic: page fault
>> cpuid = 3
>> Uptime: 11d23h7m40s
>> Physical memory: 4012 MB
>> Dumping 674 MB: 659 643 627 611 595 579 563 547 531 515 499 483 467 451 435
>> 419 403 387 371 355 339 323 307 291 275 259 243 227 211 195 179 163 147 131
>> 115 99 83 67 51 35 19 3
>>
>> Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from
>> /boot/kernel/coretemp.ko.symbols...done.
>> done.
>> Loaded symbols for /boot/kernel/coretemp.ko
>> Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from
>> /boot/kernel/ng_socket.ko.symbols...done.
>> done.
>> Loaded symbols for /boot/kernel/ng_socket.ko
>> Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from
>> /boot/kernel/netgraph.ko.symbols...done.
>> done.
>> Loaded symbols for /boot/kernel/netgraph.ko
>> Reading symbols from /boot/kernel/ng_iface.ko...Reading symbols from
>> /boot/kernel/ng_iface.ko.symbols...done.
>> done.
>> Loaded symbols for /boot/kernel/ng_iface.ko
>> Reading symbols from /boot/kernel/ng_ksocket.ko...Reading symbols from
>> /boot/kernel/ng_ksocket.ko.symbols...done.
>> done.
>> Loaded symbols for /boot/kernel/ng_ksocket.ko
>> #0  doadump () at pcpu.h:224
>> 224     pcpu.h: No such file or directory.
>>         in pcpu.h
>> (kgdb) bt
>> #0  doadump () at pcpu.h:224
>> #1  0xffffffff80398d3e in boot (howto=260) at
>> ../../../kern/kern_shutdown.c:419
>> #2  0xffffffff80399153 in panic (fmt=0x0) at
>> ../../../kern/kern_shutdown.c:592
>> #3  0xffffffff8056f19d in trap_fatal (frame=0xffffff00070828c0, eva=Variable
>> "eva" is not available.
>> ) at ../../../amd64/amd64/trap.c:783
>> #4  0xffffffff8056f55f in trap_pfault (frame=0xffffff811c80a4f0, usermode=0)
>> at ../../../amd64/amd64/trap.c:699
>> #5  0xffffffff8056f95f in trap (frame=0xffffff811c80a4f0) at
>> ../../../amd64/amd64/trap.c:449
>> #6  0xffffffff80557ba4 in calltrap () at
>> ../../../amd64/amd64/exception.S:224
>> #7  0xffffffff803f4a87 in m_copym (m=0x0, off0=2980, len=1480, wait=1) at
>> ../../../kern/uipc_mbuf.c:542
> obviously enough, m_copym() does not like to be passed a NULL mbuf :)
>

Can you help detect why it happens?

>   - Arnaud
>
>> #8  0xffffffff8047fc07 in ip_fragment (ip=0xffffff010e8d0558,
>> m_frag=0xffffff811c80a718, mtu=Variable "mtu" is not available.
>> ) at ../../../netinet/ip_output.c:819
>> #9  0xffffffff80480d1f in ip_output (m=0xffffff010e8d0500, opt=Variable
>> "opt" is not available.
>> ) at ../../../netinet/ip_output.c:650
>> #10 0xffffffff8047ca00 in ip_forward (m=0xffffff010e317600, srcrt=Variable
>> "srcrt" is not available.
>> ) at ../../../netinet/ip_input.c:1521
>> #11 0xffffffff8047e1cd in ip_input (m=0xffffff010e317600) at
>> ../../../netinet/ip_input.c:729
>> #12 0xffffffff8044eebe in netisr_dispatch_src (proto=1, source=Variable
>> "source" is not available.
>> ) at ../../../net/netisr.c:917
>> #13 0xffffffff80a2dd76 in ng_linkinfo_type_fields () from
>> /boot/kernel/netgraph.ko
>> #14 0xffffffff80a26ae0 in ng_path2noderef (here=Variable "here" is not
>> available.
>> ) at netgraph.h:463
>> #15 0xffffffff80a25bee in ng_worklist_add (node=0xffffff016abda168) at
>> /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:3378
>> #16 0xffffffff80a30339 in ng_namehash_mtx () from /boot/kernel/netgraph.ko
>> #17 0xffffffff80a26be0 in ng_path2noderef (here=0xffffff002dd93c00,
>> address=Variable "address" is not available.
>> ) at /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:1773
>> #18 0xffffffff80a27ceb in ng_apply_item (node=0x246,
>> item=0xffffff002dd93c70, rw=0) at
>> /usr/src/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:2613
>> #19 0xffffffff8036ff68 in fork_exit (callout=0xffffffff80a27b80
>> <ng_apply_item+1920>, arg=0x0, frame=0xffffff811c80ac40) at
>> ../../../kern/kern_fork.c:845
>> #20 0xffffffff8055806e in fork_trampoline () at
>> ../../../amd64/amd64/exception.S:565
>> #21 0x0000000000000000 in ?? ()
>> #22 0x0000000000000000 in ?? ()
>> #23 0x0000000000000001 in ?? ()
>> #24 0x0000000000000000 in ?? ()
>> #25 0x0000000000000000 in ?? ()
>> #26 0x0000000000000000 in ?? ()
>> #27 0x0000000000000000 in ?? ()
>> #28 0x0000000000000000 in ?? ()
>> #29 0x0000000000000000 in ?? ()
>> #30 0x0000000000000000 in ?? ()
>> #31 0x0000000000000000 in ?? ()
>> #32 0x0000000000000000 in ?? ()
>> #33 0x0000000000000000 in ?? ()
>> #34 0x0000000000000000 in ?? ()
>> #35 0x0000000000000000 in ?? ()
>> #36 0x0000000000000000 in ?? ()
>> #37 0x0000000000000000 in ?? ()
>> #38 0x0000000000000000 in ?? ()
>> #39 0x0000000000000000 in ?? ()
>> #40 0x0000000000000000 in ?? ()
>> #41 0x0000000000000000 in ?? ()
>> #42 0x0000000000000000 in ?? ()
>> #43 0x0000000000000000 in ?? ()
>> #44 0x0000000000000000 in ?? ()
>> #45 0xffffffff807cce40 in affinity ()
>> #46 0x0000000000000000 in ?? ()
>> #47 0x0000000000000000 in ?? ()
>> #48 0xffffff0001efa000 in ?? ()
>> #49 0xffffff811c809c80 in ?? ()
>> #50 0xffffff811c809c28 in ?? ()
>> #51 0xffffff0001d20460 in ?? ()
>> #52 0xffffffff803be5f9 in sched_switch (td=0xffffffff80a27b80, newtd=0x0,
>> flags=Variable "flags" is not available.
>> ) at ../../../kern/sched_ule.c:1852
>> Previous frame inner to this frame (corrupt stack?)

--
Simple Lehisnoe ;-)



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4E1345D1.5080903>