Date: Mon, 14 Apr 2008 21:58:25 GMT From: Ash Gokhale <ash@aeria.net> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/122772: em0 taskq panic, tcp reassembly bug causes radix tree corruption? Message-ID: <200804142158.m3ELwPJv028384@www.freebsd.org> Resent-Message-ID: <200804142200.m3EM08Tg039775@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 122772
>Category: kern
>Synopsis: em0 taskq panic, tcp reassembly bug causes radix tree corruption?
>Confidential: no
>Severity: critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Mon Apr 14 22:00:07 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Ash Gokhale
>Release: 7.0
>Organization:
aeria
>Environment:
FreeBSD dream 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
>Description:
lightly loaded box with pf and a few jails panics after several days uptime. After poking around kgdb, found that rn_match operating on a radix tree that appears to be corrupt.
__________________________________________________
#kgdb /boot/kernel/kernel /var/crash/vmcore.0
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x0
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc07f60df
stack pointer = 0x28:0xe750b964
frame pointer = 0x28:0xe750b990
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 24 (em0 taskq)
trap number = 12
panic: page fault
cpuid = 1
Uptime: 18d21h5m21s
Physical memory: 3570 MB
Dumping 439 MB: 424 408 392 376 360 344 328 312 296 280 264 248 232 216 200 184 168 152 136 120 104 88 72 56 40 24 8
(kgdb) bt
#0 doadump () at pcpu.h:195
#1 0xc0754457 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2 0xc0754719 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3 0xc0a4905c in trap_fatal (frame=0xe750b924, eva=0)
at /usr/src/sys/i386/i386/trap.c:899
#4 0xc0a492e0 in trap_pfault (frame=0xe750b924, usermode=0, eva=0)
at /usr/src/sys/i386/i386/trap.c:812
#5 0xc0a49c8c in trap (frame=0xe750b924) at /usr/src/sys/i386/i386/trap.c:490
#6 0xc0a2fc0b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7 0xc07f60df in rn_match (v_arg=0xd7058d0c, head=0xc9fa6600)
at /usr/src/sys/net/radix.c:294
#8 0xd7050020 in ?? ()
..
#95 0x00000000 in ?? ()
#96 0xc088b009 in tcp_input (m=0xcdbe79b0, off0=-810258404)
at /usr/src/sys/netinet/tcp_input.c:645
(kgdb)up 7
#7 0xc07f60df in rn_match (v_arg=0xd7058d0c, head=0xc9fa6600)
at /usr/src/sys/net/radix.c:294
294 if (*cp != *cp2)
Current language: auto; currently c
(kgdb) l
289 */
..
293 for (; cp < cplim; cp++, cp2++)
294 if (*cp != *cp2)
295 goto on1;
..
(kgdb) p cp2
$4 = 0x0 <<--------- local reason for the crash ???
(kgdb) up _a_lot_
#96 0xc088b009 in tcp_input (m=0xcdbe79b0, off0=-810258404)
at /usr/src/sys/netinet/tcp_input.c:645
645 tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen);
(kgdb) p th
$10 = (struct tcphdr *) 0x2 <<--------- that's not a good pointer
(kgdb) p m->M_dat
$14 = {MH = {MH_pkthdr = {rcvif = 0x0, header = 0x0, len = 0, csum_flags = 0,
csum_data = 0, tso_segsz = 0, ether_vtag = 0, tags = {slh_first = 0x0}},
MH_dat = {MH_ext = {ext_buf = 0x0, ext_free = 0, ext_args = 0x0, ext_size = 0,
ref_cnt = 0x0, ext_type = 0}, MH_databuf = '\0' <repeats 203 times>}},
M_databuf = '\0' <repeats 231 times>} <<----- that's not even a packet!
>How-To-Repeat:
unknown
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200804142158.m3ELwPJv028384>