Date: Sun, 22 Dec 2013 13:04:52 GMT From: Nikolay Denev <ndenev@gmail.com> To: freebsd-gnats-submit@FreeBSD.org Subject: misc/185092: panic: rtfree 2 (using RADIX_MPATH in a VNET jail) Message-ID: <201312221304.rBMD4q38060416@oldred.freebsd.org> Resent-Message-ID: <201312221310.rBMDA0Gf023033@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 185092
>Category: misc
>Synopsis: panic: rtfree 2 (using RADIX_MPATH in a VNET jail)
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sun Dec 22 13:10:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator: Nikolay Denev
>Release: 10.0-PRERELEASE
>Organization:
>Environment:
FreeBSD mars 10.0-PRERELEASE FreeBSD 10.0-PRERELEASE #5 r259547: Sat Dec 21 21:48:57 UTC 2013 ndenev@mars:/usr/obj/usr/src/sys/MARS i386
>Description:
I'm running with VIMAGE and RADIX_MPATH enabled on a Soekris Net5501 running 10.0-PRERELASE (r259547).
I have a VNET jail, liked with epair(4) interface to the main stack, and inside I have two OpenVPN instances running, that both install a default route (the way OpenVPN does this as 0.0.0.0/1) and the ECMP works fine.
However when I wanted to stop the OpenVPN processes and did a "killall openvpn" inside the VNET jail I got the following panic :
root@mars:/usr/obj/usr/src/sys/MARS # kgdb /boot/kernel/kernel.debug /var/crash/vmcore.2
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
panic: rtfree 2
KDB: stack backtrace:
#0 0xc0639ec2 at kdb_backtrace+0x52
#1 0xc0607ef5 at panic+0x85
#2 0xc06df879 at rtfree+0x139
#3 0xc06e2acd at route_output+0x48d
#4 0xc06df3cf at raw_usend+0x3f
#5 0xc066fef9 at sosend_generic+0x429
#6 0xc0670217 at sosend+0x57
#7 0xc0655d44 at soo_write+0x44
#8 0xc064dab6 at dofilewrite+0x86
#9 0xc064d7b6 at kern_writev+0x96
#10 0xc064d70c at sys_write+0x5c
#11 0xc0864233 at syscall+0x353
#12 0xc084fa01 at Xint0x80_syscall+0x21
Uptime: 14h35m12s
Physical memory: 503 MB
Dumping 90 MB: 75 59 43 27 11
No symbol "stopped_cpus" in current context.
#0 doadump (textdump=-1011979520) at pcpu.h:233
233 __asm("movl %%fs:%1,%0" : "=r" (td)
(kgdb) bt
#0 doadump (textdump=-1011979520) at pcpu.h:233
#1 0xc0607c02 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447
#2 0xc0607f5e in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:754
#3 0xc06df879 in rtfree (rt=<value optimized out>) at /usr/src/sys/net/route.c:464
#4 0xc06e2acd in route_output (m=<value optimized out>) at /usr/src/sys/net/rtsock.c:951
#5 0xc06df3cf in raw_usend (so=<value optimized out>, flags=0, m=<value optimized out>, nam=0x0, control=<value optimized out>,
td=0xc3db8310) at /usr/src/sys/net/raw_usrreq.c:238
#6 0xc066fef9 in sosend_generic (so=0xc3e35d40, uio=<value optimized out>, top=<value optimized out>, control=0x0,
flags=<value optimized out>, td=<value optimized out>) at /usr/src/sys/kern/uipc_socket.c:1271
#7 0xc0670217 in sosend (so=0xc3e35d40, addr=0x0, uio=0xd9bc1c10, top=0x0, control=0x0, flags=0, td=0xc3db8310)
at /usr/src/sys/kern/uipc_socket.c:1315
#8 0xc0655d44 in soo_write (fp=0xc3c0c428, uio=0xd9bc1c10, active_cred=0xc448ff00, flags=0, td=0xc3db8310)
at /usr/src/sys/kern/sys_socket.c:103
#9 0xc064dab6 in dofilewrite (td=0xc3db8310, fd=3, fp=0xc3c0c428, auio=0xd9bc1c10, offset=-1, flags=0) at file.h:303
#10 0xc064d7b6 in kern_writev (td=0xc3db8310, fd=3, auio=<value optimized out>) at /usr/src/sys/kern/sys_generic.c:467
#11 0xc064d70c in sys_write (td=<value optimized out>, uap=<value optimized out>) at /usr/src/sys/kern/sys_generic.c:382
#12 0xc0864233 in syscall (frame=<value optimized out>) at subr_syscall.c:134
#13 0xc084fa01 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:270
#14 0x00000033 in ?? ()
Code in sys/net/rtsock.c:951 looks responsible for this, and is marked with XXX.
<code>
if (rt) /* XXX can this be true? */
RTFREE(rt);
</code>
>How-To-Repeat:
Setup OpenVPN instances to install the same route using RADIX_MPATH inside a VNET jail. Kill the instances.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201312221304.rBMD4q38060416>