From owner-freebsd-current@FreeBSD.ORG Sat Jun 22 00:17:18 2013 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CD813217; Sat, 22 Jun 2013 00:17:18 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from onyx.glenbarber.us (onyx.glenbarber.us [IPv6:2607:fc50:1000:c200::face]) by mx1.freebsd.org (Postfix) with ESMTP id 71CED10E3; Sat, 22 Jun 2013 00:17:18 +0000 (UTC) Received: from glenbarber.us (nucleus.glenbarber.us [IPv6:2001:470:8:1205:2:2:ff:29]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by onyx.glenbarber.us (Postfix) with ESMTPSA id ACA6C23F840; Fri, 21 Jun 2013 20:17:15 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.8.3 onyx.glenbarber.us ACA6C23F840 Authentication-Results: onyx.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Fri, 21 Jun 2013 20:17:12 -0400 From: Glen Barber To: freebsd-current@FreeBSD.org Subject: [panic] swi4 page fault (ip_slowtimo()) Message-ID: <20130622001712.GA1888@glenbarber.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="bp/iNruPH9dso1Pn" Content-Disposition: inline X-Operating-System: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jun 2013 00:17:18 -0000 --bp/iNruPH9dso1Pn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I have the following kgdb session from a page fault seemingly triggered in pf(4). I realize the -CURRENT is about a month old, but I cannot find any commits that seem relevant to this area of the code. I am happy to dig further and provide any information that is requested. Glen Script started on Fri Jun 21 19:57:21 2013 root@orion:/usr/obj/usr/src/sys/ORION # uname -a FreeBSD orion 10.0-CURRENT FreeBSD 10.0-CURRENT #10 r250476: Fri May 10 16:= 29:54 EDT 2013 root@orion:/usr/obj/usr/src/sys/ORION amd64 root@orion:/usr/obj/usr/src/sys/ORION # kgdb ./kernel.debug /var/crash/vmco= re.8 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x11 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80772688 stack pointer =3D 0x28:0xffffff800026da20 frame pointer =3D 0x28:0xffffff800026da40 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 12 (swi4: clock) trap number =3D 12 panic: page fault cpuid =3D 0 KDB: stack backtrace: #0 0xffffffff80676a46 at kdb_backtrace+0x66 #1 0xffffffff8063ae6b at panic+0x13b #2 0xffffffff80918ba0 at trap_fatal+0x290 #3 0xffffffff80918f11 at trap_pfault+0x221 #4 0xffffffff809194c4 at trap+0x344 #5 0xffffffff80902c53 at calltrap+0x8 #6 0xffffffff806a29ce at pfslowtimo+0x2e #7 0xffffffff80651476 at softclock_call_cc+0x106 #8 0xffffffff80651b09 at softclock+0xa9 #9 0xffffffff8060c06d at intr_event_execute_handlers+0xfd #10 0xffffffff8060d81b at ithread_loop+0x9b #11 0xffffffff80608c1f at fork_exit+0x11f #12 0xffffffff8090317e at fork_trampoline+0xe Uptime: 42d1h53m40s (ada0:ahcich0:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress (ada0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): Synchronize cache failed (ada1:ahcich1:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 (ada1:ahcich1:0:0:0): CAM status: CCB request is in progress (ada1:ahcich1:0:0:0): Error 5, Retries exhausted (ada1:ahcich1:0:0:0): Synchronize cache failed (ada2:ahcich4:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 (ada2:ahcich4:0:0:0): CAM status: CCB request is in progress (ada2:ahcich4:0:0:0): Error 5, Retries exhausted (ada2:ahcich4:0:0:0): Synchronize cache failed (ada3:ahcich5:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 (ada3:ahcich5:0:0:0): CAM status: CCB request is in progress (ada3:ahcich5:0:0:0): Error 5, Retries exhausted (ada3:ahcich5:0:0:0): Synchronize cache failed Dumping 2263 out of 6048 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..9= 1% Reading symbols from /boot/kernel/zfs.ko.symbols...done. Loaded symbols for /boot/kernel/zfs.ko.symbols Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. Loaded symbols for /boot/kernel/opensolaris.ko.symbols #0 doadump (textdump=3D) at pcpu.h:231 231 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) bt #0 doadump (textdump=3D) at pcpu.h:231 #1 0xffffffff8063a9d6 in kern_reboot (howto=3D260) at /usr/src/sys/kern/ke= rn_shutdown.c:447 #2 0xffffffff8063ae55 in panic (fmt=3D) at /usr/src/s= ys/kern/kern_shutdown.c:754 #3 0xffffffff80918ba0 in trap_fatal (frame=3D0xc, eva=3D) at /usr/src/sys/amd64/amd64/trap.c:872 #4 0xffffffff80918f11 in trap_pfault (frame=3D0xffffff800026d970, usermode= =3D0) at /usr/src/sys/amd64/amd64/trap.c:789 #5 0xffffffff809194c4 in trap (frame=3D0xffffff800026d970) at /usr/src/sys= /amd64/amd64/trap.c:463 #6 0xffffffff80902c53 in calltrap () at /usr/src/sys/amd64/amd64/exception= =2ES:228 #7 0xffffffff80772688 in ip_slowtimo () at /usr/src/sys/netinet/ip_input.c= :1237 #8 0xffffffff806a29ce in pfslowtimo (arg=3D0x0) at /usr/src/sys/kern/uipc_= domain.c:508 #9 0xffffffff80651476 in softclock_call_cc (c=3D0xffffffff80e1ac60, cc=3D0= xffffffff80dc6800, direct=3D0) at /usr/src/sys/kern/kern_timeout.c:674 #10 0xffffffff80651b09 in softclock (arg=3D) at /usr/s= rc/sys/kern/kern_timeout.c:802 #11 0xffffffff8060c06d in intr_event_execute_handlers (p=3D, ie=3D0xfffffe0010811900) at /usr/src/sys/kern/kern_intr.c:1263 #12 0xffffffff8060d81b in ithread_loop (arg=3D0xfffffe0010819000) at /usr/s= rc/sys/kern/kern_intr.c:1276 #13 0xffffffff80608c1f in fork_exit (callout=3D0xffffffff8060d780 , arg=3D0xfffffe0010819000, frame=3D0xffffff800026dc00) at /usr/src/sys/kern/kern_fork.c:991 #14 0xffffffff8090317e in fork_trampoline () at /usr/src/sys/amd64/amd64/ex= ception.S:602 #15 0x0000000000000000 in ?? () (kgdb) frame 6 #6 0xffffffff80902c53 in calltrap () at /usr/src/sys/amd64/amd64/exception= =2ES:228 228 call trap Current language: auto; currently asm (kgdb) list *0xffffffff80902c53 0xffffffff80902c53 is at /usr/src/sys/amd64/amd64/exception.S:230. 225 .type calltrap,@function 226 calltrap: 227 movq %rsp,%rdi 228 call trap 229 MEXITCOUNT 230 jmp doreti /* Handle any pending ASTs */ 231=09 232 /* 233 * alltraps_noen entry point. Unlike alltraps above, we want to 234 * leave the interrupts disabled. This corresponds to (kgdb) up #7 0xffffffff80772688 in ip_slowtimo () at /usr/src/sys/netinet/ip_input.c= :1237 1237 for(fp =3D TAILQ_FIRST(&V_ipq[i]); fp;) { Current language: auto; currently c (kgdb) list *0xffffffff80772688 0xffffffff80772688 is in ip_slowtimo (/usr/src/sys/netinet/ip_input.c:1242). 1237 for(fp =3D TAILQ_FIRST(&V_ipq[i]); fp;) { 1238 struct ipq *fpp; 1239=09 1240 fpp =3D fp; 1241 fp =3D TAILQ_NEXT(fp, ipq_list); 1242 if(--fpp->ipq_ttl =3D=3D 0) { 1243 IPSTAT_ADD(ips_fragtimeout, 1244 fpp->ipq_nfrags); 1245 ip_freef(&V_ipq[i], fpp); 1246 } (kgdb) p *ipq $1 =3D {tqh_first =3D 0x0, tqh_last =3D 0xffffffff80e20e80} (kgdb) up #8 0xffffffff806a29ce in pfslowtimo (arg=3D0x0) at /usr/src/sys/kern/uipc_= domain.c:508 508 (*pr->pr_slowtimo)(); (kgdb) list *0xffffffff806a29ce 0xffffffff806a29ce is in pfslowtimo (/usr/src/sys/kern/uipc_domain.c:506). 501 { 502 struct domain *dp; 503 struct protosw *pr; 504=09 505 for (dp =3D domains; dp; dp =3D dp->dom_next) 506 for (pr =3D dp->dom_protosw; pr < dp->dom_protoswNPROTOSW; pr++) 507 if (pr->pr_slowtimo) 508 (*pr->pr_slowtimo)(); 509 callout_reset(&pfslow_callout, hz/2, pfslowtimo, NULL); 510 } (kgdb) p *dp $2 =3D {dom_family =3D 2, dom_name =3D 0xffffffff80a56512 "internet", dom_i= nit =3D 0, dom_destroy =3D 0, dom_externalize =3D 0, dom_dispose =3D 0,=20 dom_protosw =3D 0xffffffff80d16320, dom_protoswNPROTOSW =3D 0xffffffff80d= 16ce0, dom_next =3D 0x0,=20 dom_rtattach =3D 0xffffffff8076d070 , dom_rtdetach =3D 0, do= m_rtoffset =3D 32, dom_maxrtkey =3D 16,=20 dom_ifattach =3D 0xffffffff807626c0 , dom_ifdetach =3D 0x= ffffffff80762690 } (kgdb) p *dp $3 =3D {dom_family =3D 2, dom_name =3D 0xffffffff80a56512 "internet", dom_i= nit =3D 0, dom_destroy =3D 0, dom_externalize =3D 0, dom_dispose =3D 0,=20 dom_protosw =3D 0xffffffff80d16320, dom_protoswNPROTOSW =3D 0xffffffff80d= 16ce0, dom_next =3D 0x0,=20 dom_rtattach =3D 0xffffffff8076d070 , dom_rtdetach =3D 0, do= m_rtoffset =3D 32, dom_maxrtkey =3D 16,=20 dom_ifattach =3D 0xffffffff807626c0 , dom_ifdetach =3D 0x= ffffffff80762690 } (kgdb) p *domains $4 =3D {dom_family =3D 17, dom_name =3D 0xffffffff809acd08 "route", dom_ini= t =3D 0, dom_destroy =3D 0, dom_externalize =3D 0, dom_dispose =3D 0,=20 dom_protosw =3D 0xffffffff80d11300, dom_protoswNPROTOSW =3D 0xffffffff80d= 11368, dom_next =3D 0xffffffff80d21de0, dom_rtattach =3D 0,=20 dom_rtdetach =3D 0, dom_rtoffset =3D 0, dom_maxrtkey =3D 0, dom_ifattach = =3D 0, dom_ifdetach =3D 0} (kgdb) p *dp->dom_protoswNPROTOSW $5 =3D {pr_type =3D 2, pr_domain =3D 0xffffffff80a56512, pr_protocol =3D 0,= pr_flags =3D 0, pr_input =3D 0, pr_output =3D 0, pr_ctlinput =3D 0,=20 pr_ctloutput =3D 0xffffffff80d16320 , pr_init =3D 0xffffffff80d16= ce0 , pr_destroy =3D 0,=20 pr_fasttimo =3D 0xffffffff8076d070 , pr_slowtimo =3D 0, pr_d= rain =3D 0x1000000020, pr_usrreqs =3D 0xffffffff807626c0} (kgdb) p pfslow_callout $6 =3D {c_links =3D {le =3D {le_next =3D 0x0, le_prev =3D 0xffffffff80dc691= 0}, sle =3D {sle_next =3D 0x0}, tqe =3D {tqe_next =3D 0x0,=20 tqe_prev =3D 0xffffffff80dc6910}}, c_time =3D 15614872462233060, c_pr= ecision =3D 134217718, c_arg =3D 0x0,=20 c_func =3D 0xffffffff806a29a0 , c_lock =3D 0x0, c_flags =3D 1= 46, c_cpu =3D 0} (kgdb) p *pfslowtimo $7 =3D {void (void *)} 0xffffffff806a29a0 (kgdb) up #9 0xffffffff80651476 in softclock_call_cc (c=3D0xffffffff80e1ac60, cc=3D0= xffffffff80dc6800, direct=3D0) at /usr/src/sys/kern/kern_timeout.c:674 674 c_func(c_arg); (kgdb) list *0xffffffff80651476 0xffffffff80651476 is in softclock_call_cc (/usr/src/sys/kern/kern_timeout.= c:675). 670 sbt1 =3D sbinuptime(); 671 #endif 672 THREAD_NO_SLEEPING(); 673 SDT_PROBE(callout_execute, kernel, , callout_start, c, 0, 0, 0, 0); 674 c_func(c_arg); 675 SDT_PROBE(callout_execute, kernel, , callout_end, c, 0, 0, 0, 0); 676 THREAD_SLEEPING_OK(); 677 #if defined(DIAGNOSTIC) || defined(CALLOUT_PROFILING) 678 sbt2 =3D sbinuptime(); 679 sbt2 -=3D sbt1; (kgdb) quit root@orion:/usr/obj/usr/src/sys/ORION # ^D Script done on Fri Jun 21 19:57:39 2013 --bp/iNruPH9dso1Pn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQEcBAEBCAAGBQJRxO0IAAoJEFJPDDeguUaj4hYH/2bjj3VtJraUQk6/gyxR80Y5 h9JKEoWQuHJD6FgjiPHy4NYSS9lwCVMWv8s/VNi6TkGuvff+qHPKoVq6ATFYkrBL e/DWT2fi/Dc9eUeI14WapSbjC/p2/zfOWmx9qqE1ZxcCh7jsLqeUncpberUr363n CVU12xAQiPstdrzPS3UReYF+E8OH9C4V8uUI6HIMkFvZiV2QASfCxxTc27MR2j91 TRc25Xf7e9RwoCxQ2MjgnAzGrwMiHOtrZ5ffEMUQWUQyuP2zpNQk9BryZ7FwJ2O4 17fXaER9NNqGq3Iwm+8IBbY7SOMVBsfvOWuBUTH36xuy/L9gFUOlF209k7gKSQw= =Ls4x -----END PGP SIGNATURE----- --bp/iNruPH9dso1Pn--