From owner-freebsd-current@FreeBSD.ORG Mon Jan 26 06:41:19 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A576A16A4CE for ; Mon, 26 Jan 2004 06:41:19 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1463943D31 for ; Mon, 26 Jan 2004 06:41:13 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i0QEcuUd011424; Mon, 26 Jan 2004 09:38:56 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i0QEctYL011421; Mon, 26 Jan 2004 09:38:56 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 26 Jan 2004 09:38:55 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Jun Su In-Reply-To: <003f01c3e3d4$af1e8250$90dea1d3@shasujunmv> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org Subject: Re: [PANIC] today's cvs with KTRACE and MUTEX_PROFILING enabled (Big Post) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 14:41:19 -0000 On Mon, 26 Jan 2004, Jun Su wrote: > I got the following panic when booting. Seems the if_data_mutex is > corrupt by something.If you need any information, please feel free to > let me know. Thanks. Are you running with kernel modules? If so, were all of them built as part of your kernel build? Turning on mutex profiling changes the size of the mutex structure, so if you have modules that aren't built to be aware of mutex profiling, you will get memory corruption. When using mutex profiling, I would generally suggest avoiding the use of kernel modules, since it avoids the whole issue... Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research > > panic: mutex "if_afdata" 0xc1655a04 already initialized > at line 778 in file /usr/src/sys/kern/kern_mutex.c > > (kgdb) where > #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240 > #1 0xc048c073 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:374 > #2 0xc048c4a9 in __panic () at /usr/src/sys/kern/kern_shutdown.c:552 > #3 0xc0423ce2 in db_panic () at /usr/src/sys/ddb/db_command.c:450 > #4 0xc0423c32 in db_command (last_cmdp=0xc0601a60, cmd_table=0x0, > aux_cmd_tablep=0xc05dae94, > aux_cmd_tablep_end=0xc05dae98) at /usr/src/sys/ddb/db_command.c:346 > #5 0xc0423d85 in db_command_loop () at /usr/src/sys/ddb/db_command.c:472 > #6 0xc0426d95 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_trap.c:73 > #7 0xc058b45c in kdb_trap (type=3, code=0, regs=0xc6fc8a08) > at /usr/src/sys/i386/i386/db_interface.c:171 > #8 0xc05a09a8 in trap (frame= > {tf_fs = 24, tf_es = 16, tf_ds = 16, tf_edi = -1067696406, tf_esi = 1, > tf_ebp = -956528044, tf\ > _isp = -956528076, tf_ebx = 0, tf_edx = 0, tf_ecx = -1067278624, tf_eax = > 18, tf_trapno = 3, tf_err \ > = 0, tf_eip = -1067927691, tf_cs = 8, tf_eflags = 646, tf_esp = -1067625032, > tf_ss = -1067693998}) > at /usr/src/sys/i386/i386/trap.c:580 > #9 0xc058ce88 in calltrap () at {standard input}:94 > #10 0xc048c442 in __panic (file=0xc05c3c1d "/usr/src/sys/kern/kern_mutex.c", > line=778, > fmt=0xc05c3eea "mutex \"%s\" %p already initialized") at > /usr/src/sys/kern/kern_shutdown.c:536 > #11 0xc0482bce in mtx_init (m=0xc1655a04, name=0xc05cb666 "if_afdata", > type=0x0, opts=0) > at /usr/src/sys/kern/kern_mutex.c:777 > #12 0xc04f9ad1 in if_attach (ifp=0xc165582c) at /usr/src/sys/net/if.c:370 > #13 0xc04fd462 in ether_ifattach (ifp=0xc165582c, llc=0xc1655a10 "") > at /usr/src/sys/net/if_ethersubr.c:830 > #14 0xc1893429 in lnc_attach_common (dev=0x0) at > /usr/src/sys/dev/lnc/if_lnc.c:895 > #15 0xc18949d5 in lnc_pci_attach (dev=0xc15c9080) at > /usr/src/sys/dev/lnc/if_lnc_pci.c:186 > #16 0xc04a6299 in device_probe_and_attach (dev=0xc15c9080) at device_if.h:39 > #17 0xc04a707e in bus_generic_driver_added (dev=0x0, driver=0xc1896860) > at /usr/src/sys/kern/subr_bus.c:1930 > #18 0xc04a4dd0 in devclass_add_driver (dc=0xc0db9880, driver=0xc1896860) at > bus_if.h:71 > #19 0xc04a82df in driver_module_handler (mod=0xc15cdbc0, what=-1047959456, > arg=0xc1896878) > at /usr/src/sys/kern/subr_bus.c:2369 > #20 0xc0481341 in module_register_init (arg=0xc189688c) at > /usr/src/sys/kern/kern_module.c:108 > #21 0xc047b7c0 in linker_file_sysinit (lf=0x0) at > /usr/src/sys/kern/kern_linker.c:192 > #22 0xc047bb4a in linker_load_file (filename=0xc16994a0 > "/boot/kernel/if_lnc.ko", result=0xc6fc8ca8) > at /usr/src/sys/kern/kern_linker.c:357 > #23 0xc047e407 in linker_load_module (kldname=0xc05e9aa0 > "\236\022\\\300@\232^??", > modname=0xc16994a0 "/boot/kernel/if_lnc.ko", parent=0x0, verinfo=0x0, > lfpp=0xc6fc8cd4) > at /usr/src/sys/kern/kern_linker.c:1670 > #24 0xc047c7d3 in kldload (td=0xc1626690, uap=0x0) at > /usr/src/sys/kern/kern_linker.c:773 > #25 0xc05a1390 in syscall (frame= > {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = -1077941000, > tf_ebp = -1077941044, t\ > f_isp = -956527244, tf_ebx = 0, tf_edx = -1077941004, tf_ecx = 2, tf_eax = > 304, tf_trapno = 12, tf_e\ > rr = 2, tf_eip = 671899791, tf_cs = 31, tf_eflags = 531, tf_esp > = -1077941092, tf_ss = 47}) > at /usr/src/sys/i386/i386/trap.c:1010 > #26 0xc058cedd in Xint0x80_syscall () at {standard input}:136 > ---Can't read userspace from dump, or kernel process--- > (kgdb) up 11 > #11 0xc0482bce in mtx_init (m=0xc1655a04, name=0xc05cb666 "if_afdata", > type=0x0, opts=0) > at /usr/src/sys/kern/kern_mutex.c:777 > (kgdb) print *m > $2 = {mtx_object = {lo_class = 0x0, lo_name = 0x0, lo_type = 0x0, lo_flags = > 1697188864, lo_list = { > tqe_next = 0xe4b, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 2, > mtx_recurse = 7, > mtx_acqtime = 1, mtx_filename = 0x0, mtx_lineno = 3, mtx_contest_holding = > 3341123584, > mtx_contest_locking = 0} > (kgdb) up > #12 0xc04f9ad1 in if_attach (ifp=0xc165582c) at /usr/src/sys/net/if.c:370 > (kgdb) print *ifp > $3 = {if_softc = 0xc1655800, if_link = {tqe_next = 0x0, tqe_prev = 0x0}, > if_xname = "lnc1", '\0' , if_dname = 0xc15cc420 "lnc", > if_dunit = 1, > if_addrhead = {tqh_first = 0x0, tqh_last = 0x0}, if_klist = {slh_first = > 0x0}, if_pcount = 0, > if_bpf = 0x0, if_index = 0, if_timer = 0, if_nvlans = 0, if_flags = 34818, > if_capabilities = 0, > if_capenable = 0, if_ipending = 0, if_linkmib = 0x0, if_linkmiblen = 0, > if_data = { > ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', > ifi_hdrlen = 14 '\016', > ifi_recvquota = 0 '\0', ifi_xmitquota = 0 '\0', ifi_mtu = 0, ifi_metric > = 0, ifi_baudrate = 0, > ifi_ipackets = 0, ifi_ierrors = 0, ifi_opackets = 0, ifi_oerrors = 0, > ifi_collisions = 0, > ifi_ibytes = 0, ifi_obytes = 0, ifi_imcasts = 0, ifi_omcasts = 0, > ifi_iqdrops = 0, > ifi_noproto = 0, ifi_hwassist = 0, ifi_unused = 0, ifi_lastchange = > {tv_sec = 0, tv_usec = 0}}, > if_multiaddrs = {tqh_first = 0x0, tqh_last = 0x0}, if_amcount = 0, > if_output = 0xc04fc5f0 , if_input = 0, if_start = 0xc1893e20 > , > if_done = 0, if_ioctl = 0xc1894370 , if_watchdog = 0xc18944b0 > , > if_poll_recv = 0, if_poll_xmit = 0, if_poll_intren = 0, if_poll_slowinput > = 0, > if_init = 0xc18934c0 , if_resolvemulti = 0, if_snd = {ifq_head = > 0x0, ifq_tail = 0x0, > ifq_len = 0, ifq_maxlen = 50, ifq_drops = 0, ifq_mtx = {mtx_object = > {lo_class = 0x0, > lo_name = 0x0, lo_type = 0x0, lo_flags = 0, lo_list = {tqe_next = > 0x0, tqe_prev = 0x0}, > lo_witness = 0x0}, mtx_lock = 0, mtx_recurse = 0, mtx_acqtime = 0, > mtx_filename = 0x0, > mtx_lineno = 0, mtx_contest_holding = 0, mtx_contest_locking = 0}}, > if_poll_slowq = 0x0, > if_prefixhead = {tqh_first = 0x0, tqh_last = 0x0}, if_broadcastaddr = 0x0, > if_label = 0x0, > if_afdata = {0x0 }, if_afdata_initialized = 0, > if_afdata_mtx = {mtx_object = { > lo_class = 0x0, lo_name = 0x0, lo_type = 0x0, lo_flags = 1697188864, > lo_list = { > tqe_next = 0xe4b, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 2, > mtx_recurse = 7, > mtx_acqtime = 1, mtx_filename = 0x0, mtx_lineno = 3, mtx_contest_holding > = 3341123584, > mtx_contest_locking = 0}} > (kgdb) > >