Date: Fri, 7 Nov 2003 17:52:46 -0800 (PST) From: Doug White <dwhite@gumbysoft.com> To: Prafulla Deuskar <pdeuskar@freebsd.org> Cc: freebsd-stable@freebsd.org Subject: Re: Panic on -stable with 10 Gigabit driver Message-ID: <20031107174754.R27719@carver.gumbysoft.com> In-Reply-To: <20031107172535.A35573@hub.freebsd.org> References: <20031107172535.A35573@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 7 Nov 2003, Prafulla Deuskar wrote:
> We are having a reproducible panic with our 10 Gigabit driver on -stable
> under stress. (4.8-RELEASE, 4.9-RELEASE) We have not been able to
> reproduce the panic on 5.1-RELEASE with same load.
What revision of uipc_mbuf.c generated this crashdump? I'm going to guess
4.8 RELEASE since it points to an actual line of code.
I'm going to take a guess and shoot for cluster corruption of some sort.
The offending line is
MCLGET(mp, how);
Which would explode if the free list is wrecked perhaps. If you can, load
up the crashdump and poke around the freelist and see if something got
spammed perhaps.
>
> The test is nttcp with 1500 byte MTU.
>
> If anyone on the list has any ideas please let me know.
>
> Thanks,
> Prafulla
>
> Here is the crash dump output:
> ------------------------------
>
> #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
> 487 if (dumping++) {
> (kgdb) where
> #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
> #1 0xc021a823 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:316
> #2 0xc021ac61 in panic (fmt=0xc03b1944 "from debugger")
> at /usr/src/sys/kern/kern_shutdown.c:595
> #3 0xc014be59 in db_panic (addr=-1071421816, have_addr=0, count=-1,
> modif=0xc0417874 "") at /usr/src/sys/ddb/db_command.c:435
> #4 0xc014bdf9 in db_command (last_cmdp=0xc041daa4, cmd_table=0xc041d8e4,
> aux_cmd_tablep=0xc0460d98) at /usr/src/sys/ddb/db_command.c:333
> #5 0xc014bebe in db_command_loop () at /usr/src/sys/ddb/db_command.c:457
> #6 0xc014dffb in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_trap.c:71
> #7 0xc0379c8e in kdb_trap (type=12, code=0, regs=0xc04179c8)
> at /usr/src/sys/i386/i386/db_interface.c:158
> #8 0xc038a4a4 in trap_fatal (frame=0xc04179c8, eva=117471232)
> at /usr/src/sys/i386/i386/trap.c:969
> #9 0xc038a17d in trap_pfault (frame=0xc04179c8, usermode=0, eva=117471232)
> at /usr/src/sys/i386/i386/trap.c:867
> #10 0xc0389cf7 in trap (frame={tf_fs = -1069481968, tf_es = -1069481968, tf_ds = 16,
> tf_edi = 6756506, tf_esi = 1, tf_ebp = -1069450724, tf_isp = -1069450764,
> tf_ebx = -1039395840, tf_edx = -1039732736, tf_ecx = 117471232, tf_eax = 565021,
> tf_trapno = 12, tf_err = 0, tf_eip = -1071421816, tf_cs = 8, tf_eflags = 66050,
> tf_esp = 0, tf_ss = -955602944}) at /usr/src/sys/i386/i386/trap.c:466
> #11 0xc0236688 in m_getcl (how=1, type=1, flags=2) at /usr/src/sys/kern/uipc_mbuf.c:589
> #12 0xc03a4282 in ixgb_get_buf (i=254, adapter=0xc70aa800, nmp=0x0)
> at /usr/src/sys/dev/ixgb/if_ixgb.c:1768
> #13 0xc03a495d in ixgb_process_receive_interrupts (adapter=0xc70aa800, count=-35)
> at /usr/src/sys/dev/ixgb/if_ixgb.c:2095
> #14 0xc03a310e in ixgb_intr (arg=0xc70aa800) at /usr/src/sys/dev/ixgb/if_ixgb.c:798
> #15 0xc0394cf9 in intr_mux (arg=0xc70acfe0) at /usr/src/sys/i386/isa/intr_machdep.c:582
> #16 0xc0380e02 in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1022
> (kgdb) up 10
> #10 0xc0389cf7 in trap (frame={tf_fs = -1069481968, tf_es = -1069481968, tf_ds = 16,
> tf_edi = 6756506, tf_esi = 1, tf_ebp = -1069450724, tf_isp = -1069450764,
> tf_ebx = -1039395840, tf_edx = -1039732736, tf_ecx = 117471232, tf_eax = 565021,
> tf_trapno = 12, tf_err = 0, tf_eip = -1071421816, tf_cs = 8, tf_eflags = 66050,
> tf_esp = 0, tf_ss = -955602944}) at /usr/src/sys/i386/i386/trap.c:466
> 466 (void) trap_pfault(&frame, FALSE, eva);
> (kgdb) frame frame->tf_ebp frame->tf_eip
> #0 0xc0236688 in m_getcl (how=1, type=1, flags=2) at /usr/src/sys/kern/uipc_mbuf.c:589
> 589 MCLGET(mp, how);
> (kgdb) list
> 584 } else
> 585 MGETHDR(mp, how, type);
> 586 } else
> 587 MGET(mp, how, type);
> 588 if (mp) {
> 589 MCLGET(mp, how);
> 590 if ( (mp->m_flags & M_EXT) == 0) {
> 591 m_free(mp);
> 592 mp = NULL;
> 593 }
> (kgdb) print mp
> $1 = (struct mbuf *) 0xc20c1400
> (kgdb) print how
> $2 = 1
> (kgdb) up
> #1 0xc03a4282 in ixgb_get_buf (i=254, adapter=0xc70aa800, nmp=0x0)
> at /usr/src/sys/dev/ixgb/if_ixgb.c:1768
> 1768 mp->m_next = NULL;
> (kgdb) list
> 1763 }
> 1764 mp->m_len = mp->m_pkthdr.len = MCLBYTES;
> 1765 } else {
> 1766 mp->m_len = mp->m_pkthdr.len = MCLBYTES;
> 1767 mp->m_data = mp->m_ext.ext_buf;
> 1768 mp->m_next = NULL;
> 1769 }
> 1770
> 1771 if (ifp->if_mtu <= ETHERMTU) {
> 1772 m_adj(mp, ETHER_ALIGN);
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
>
--
Doug White | FreeBSD: The Power to Serve
dwhite@gumbysoft.com | www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20031107174754.R27719>