Date: Mon, 6 Mar 2006 12:35:57 -0500 From: "Carroll Kong" <me@carrollkong.com> To: <hackers@freebsd.org> Subject: RE: FreeBSD 4.11 P13 Crash Message-ID: <20060306173558.A19AC43D60@mx1.FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Well bad news. It happened again even with a new CPU and new =
PowerSupply.
However, the good news is that it seems to be saving the core dumps a =
bit
more consistently now. I swapped the motherboard back to the old one.
Honestly, I've had similar core dumps in either case, I'm starting to =
think
it isn't the hardware but more of a software or software configuration =
issue
(one that works but is apparently not stable). I can swap the =
motherboard
back to the new one, but I got the same error in either case.
It sure looks a lot like the other backtraces, but I suppose if =
something
corrupted the data in memory, it could still be anything.
> So far I am thinking
> - IPFilter intermittent bug with some packets, but I run a=20
> box with 112 days of uptime with the same version of=20
> IPFilter, albeit not with 4 NICs.
It keeps failing around the net or ppp process. It might not mean =
anything
though since the box will always log 'junk'. Although, I find it odd =
that
it has never crashed when using the other, cable link, ONLY on the PPP
process. Only an experienced hacker can tell me if my hypothesis is =
correct
in pointing the finger closer to PPP. Then again, there is no 'process' =
to
refer to if the issue was on the cable link! Maybe a bug in handling 4 =
FXP
Intel cards? I already swapped one of them (the other 2 are built =
onboard,
and the last one I have not swapped out yet that connects to the cable
link).
> - 3Ware driver is flakey, but I have a 4.10 box with 3Ware=20
> that is somewhat stable
> - CPU (I would tend to think this would result in HARD lock=20
> ups vs Fatal Trap 12s though)
New CPU didn't fix it, so let's scratch that out. In fact, it's been =
pulled
from a working system.
> - PowerSupply (I suppose anything is possible, please note it=20
> is on an APC UPS, but the power supply might be delivering bad juice?)
New Power supply. Antec 380 or so. I don't have a method of testing it =
so
it 'could' be a bad new power supply but honestly, I would expect it to =
have
crashed with a different error.
> - Harddisks and 3Ware driver have incompatible firmware=20
> issue, I doubt this is it though since I purchased new=20
> Seagates in 9/2004 for the RAID1, then I added another=20
> Seagate as a JBOD, and that disk is not being written to=20
> during the crash.
This is still a possibility, although it seems to fail in a memory
operation. Unless the 3Ware somehow corrupted memory ahead of time, =
which
seems kind of odd but possible.
Someone suggested I go back to 4.9. I don't mind doing so although I =
wonder
if I would be vulnerable to certain security issues. Furthermore, I'm =
not
sure how to do this right. I would guess cvsup with 4.9_RELEASE tag?
Anyway, I am going to try to go back to 4.9, but wanted to throw some =
more
information to the list to see if anyone had any other ideas.
The only hardware I have not changed so far is
- cdrom
- floppy
- case
:)
Here is another backtrace. It seems to be doing the standard log to the
ipf.log file thing, then die during an mbuf operation.
Fatal trap 12: page fault while in kernel mode
fault virtual address =3D 0x41f59
fault code =3D supervisor read, page not present
instruction pointer =3D 0x8:0xc0192696
stack pointer =3D 0x10:0xd71cbbd0
frame pointer =3D 0x10:0xd71cbbd8
code segment =3D base 0x0, limit 0xfffff, type 0x1b
=3D DPL 0, pres 1, def32 1, gran 1
processor eflags =3D interrupt enabled, resume, IOPL =3D 0
current process =3D 105 (ppp)
interrupt mask =3D net tty=20
trap number =3D 12
panic: page fault
---------------
#0 dumpsys () at ../../kern/kern_shutdown.c:487
487 if (dumping++) {
(kgdb) bt
#0 dumpsys () at ../../kern/kern_shutdown.c:487
#1 0xc0173f3f in boot (howto=3D256) at ../../kern/kern_shutdown.c:316
#2 0xc0174364 in poweroff_wait (junk=3D0xc02f2aac, howto=3D-1070651985) =
at
../../kern/kern_shutdown.c:595
#3 0xc02a77ba in trap_fatal (frame=3D0xd71cbb90, eva=3D270169) at
../../i386/i386/trap.c:974
#4 0xc02a748d in trap_pfault (frame=3D0xd71cbb90, usermode=3D0, =
eva=3D270169) at
../../i386/i386/trap.c:867
#5 0xc02a704b in trap (frame=3D{tf_fs =3D -1072562160, tf_es =3D 16, =
tf_ds =3D
-686030832, tf_edi =3D 6757530, tf_esi =3D -1055310592,=20
tf_ebp =3D -685982760, tf_isp =3D -685982788, tf_ebx =3D 270169, =
tf_edx =3D
6757530, tf_ecx =3D -1055840256, tf_eax =3D -28864,=20
tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1072093546, tf_cs =3D =
8, tf_eflags
=3D 66054, tf_esp =3D -1055310592, tf_ss =3D -1055310592})
at ../../i386/i386/trap.c:466
#6 0xc0192696 in m_tag_delete_chain (m=3D0xc1193d00, t=3D0x0) at
../../kern/uipc_mbuf2.c:358
#7 0xc01904d3 in m_free (m=3D0xc1193d00) at ../../kern/uipc_mbuf.c:734
#8 0xc0190606 in m_freem (m=3D0xc117ef00) at ../../kern/uipc_mbuf.c:763
#9 0xc0127df8 in fr_check (ip=3D0xc117ef30, hlen=3D20, =
ifp=3D0xc21a9508, out=3D0,
mp=3D0xd71cbce8)
at ../../contrib/ipfilter/netinet/fil.c:1387
#10 0xc01d7d06 in ip_input (m=3D0xc117ef00) at =
../../netinet/ip_input.c:478
#11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
#12 0xc0299ee9 in swi_net_next ()
#13 0xc016e7c8 in lockmgr (lkp=3D0xc21a9600, flags=3D16973826,
interlkp=3D0xd71c162c, p=3D0xd48bd5a0) at ../../kern/kern_lock.c:355
#14 0xc019fda8 in vop_stdlock (ap=3D0xd71cbdd0) at
../../kern/vfs_default.c:256
#15 0xc025a9c9 in ufs_vnoperatespec (ap=3D0xd71cbdd0) at
../../ufs/ufs/ufs_vnops.c:2394
#16 0xc01a9ffd in vn_lock (vp=3D0xd71c15c0, flags=3D131074, =
p=3D0xd48bd5a0) at
vnode_if.h:861
#17 0xc01ad842 in spec_write (ap=3D0xd71cbe64) at
../../miscfs/specfs/spec_vnops.c:284
#18 0xc025a3ac in ufsspec_write (ap=3D0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:1827
#19 0xc025a9c9 in ufs_vnoperatespec (ap=3D0xd71cbe64) at
../../ufs/ufs/ufs_vnops.c:2394
#20 0xc01a9b9a in vn_write (fp=3D0xc21a8c40, uio=3D0xd71cbed4, =
cred=3D0xc219b780,
flags=3D0, p=3D0xd48bd5a0) at vnode_if.h:363
#21 0xc018330d in dofilewrite (p=3D0xd48bd5a0, fp=3D0xc21a8c40, fd=3D9,
buf=3D0xbfbfe89c, nbyte=3D213, offset=3D-1, flags=3D0)
at ../../sys/file.h:163
#22 0xc01831c6 in write (p=3D0xd48bd5a0, uap=3D0xd71cbf80) at
../../kern/sys_generic.c:329
#23 0xc02a7a69 in syscall2 (frame=3D{tf_fs =3D 134938671, tf_es =3D 47, =
tf_ds =3D
-1078001617, tf_edi =3D 134996736, tf_esi =3D 213,=20
tf_ebp =3D -1077940064, tf_isp =3D -685981740, tf_ebx =3D =
-1077942112,
tf_edx =3D 0, tf_ecx =3D 13, tf_eax =3D 4, tf_trapno =3D 7,=20
tf_err =3D 2, tf_eip =3D 673683504, tf_cs =3D 31, tf_eflags =3D =
663, tf_esp =3D
-1077942172, tf_ss =3D 47})
at ../../i386/i386/trap.c:1175
#24 0xc0298a85 in Xint0x80_syscall ()
#25 0x80655de in ?? ()
#26 0x806c2fb in ?? ()
#27 0x806c21d in ?? ()
#28 0x807470a in ?? ()
#29 0x8083a78 in ?? ()
#30 0x805b84b in ?? ()
#31 0x804d484 in ?? ()
#32 0x806ed77 in ?? ()
#33 0x806e967 in ?? ()
#34 0x804b62a in ?? ()
(kgdb) quit
daemon# nm /kernel | grep c0192696
daemon# nm /kernel | grep c019269=20
daemon# nm /kernel | grep c01926=20
c01926e8 T m_tag_copy
c0192658 T m_tag_delete
c0192674 T m_tag_delete_chain
c0192600 T m_tag_free
c01926ac T m_tag_locate
c0192614 T m_tag_prepend
c0192628 T m_tag_unlink
> -----Original Message-----
> From: Carroll Kong=20
> Sent: Monday, February 27, 2006 8:53 PM
> To: 'hackers@freebsd.org'
> Subject: FreeBSD 4.11 P13 Crash
>=20
> Okay this time my kernel was recompiled so there are no=20
> modules to make it easier to see all of the symbols.
>=20
> Sometimes the box cycles through the fatal traps 12. Other=20
> times it does not. Based on my other Fatal trap errors, it=20
> seems to interrupt more often with the m_tag_delete function.=20
> I don't think this necessarily means the problem is with=20
> IPFilter or PPP mostly because this box acts as a firewall=20
> and logs constantly. Therefore, it is not surprising it=20
> always fails after logging with IPFilter, but I am always=20
> open to the possibility.
>=20
> This box was stable before I upgraded from 4.9->4.11. Among=20
> one of the software changes was probably the change of=20
> IPFilter. I used to use the IPFilter 3.4.33pre modules, but=20
> after I moved to 4.11 I just used the distribution packaged=20
> 3.4.35. This might be the source of the problem, but I could=20
> not google for relevant entries.
>=20
> I have since swapped the RAM, motherboard, RAM again (I=20
> bought another stick thinking maybe my new RAM was=20
> coincidentally bugged), one of the Intel NICs, and my 3Ware=20
> controller. The problem still occurred and actually more=20
> frequently. The usual frequency was about 14 days or so. It=20
> just crashed in less than 23 hours and then again within 25 minutes.
>=20
> The final pieces of hardware that still can be swapped is the=20
> other Intel NIC (but this NIC is NOT connected to the PPPoE),=20
> CPU, Power Supply, CDROM (not used), Harddisks, or Case. :)
>=20
> I tried disabling physical swap completely, and the system=20
> still crashed, so I doubt it is the 3Ware, but who knows.
>=20
> So far I am thinking
> - IPFilter intermittent bug with some packets, but I run a=20
> box with 112 days of uptime with the same version of=20
> IPFilter, albeit not with 4 NICs.
> - 3Ware driver is flakey, but I have a 4.10 box with 3Ware=20
> that is somewhat stable
> - CPU (I would tend to think this would result in HARD lock=20
> ups vs Fatal Trap 12s though)
> - PowerSupply (I suppose anything is possible, please note it=20
> is on an APC UPS, but the power supply might be delivering bad juice?)
> - Harddisks and 3Ware driver have incompatible firmware=20
> issue, I doubt this is it though since I purchased new=20
> Seagates in 9/2004 for the RAID1, then I added another=20
> Seagate as a JBOD, and that disk is not being written to=20
> during the crash.
>=20
> I am tempted to consider upgrading to 5.X, but I am a=20
> conservative person and somehow doubt 4.X is the source of=20
> the problem as the system worked fine for over a year.
>=20
> The box does a lot of things however I omitted this=20
> information to avoid flooding the list with too much=20
> information since it has worked fine for a year in the past. =20
> As a note, the problem is NOT load related. In fact, one=20
> time the fatal panic said the running process was "idle". :)=20
> Furthermore, I haven't really updated the software=20
> unnecessarily except for security issues and the system has=20
> been stable in the past with the same hardware and same=20
> software. I am very conservative when it comes to servers,=20
> so this seems like a hardware issue but I already swapped so=20
> much of it, I am beginning to wonder.
>=20
> I am going to buy a new CPU and power supply as I have=20
> replaced nearly every other part by now.
>=20
> I have included my dmesg, nm greps for the functions, a=20
> backtrace, uname output. I have the kernel dump so if there=20
> are any commands someone needs me to punch through, I will=20
> gladly do so. I included some of my own feeble debugging. I=20
> didn't like the line that said "address is out of bounds" in=20
> one of the mbuf structures. I am guessing that means the=20
> mbuf was already corrupted way before we got there. Any=20
> suggestions and advice are welcome. Thanks in advance!
>=20
>=20
>=20
> Fatal trap 12: page fault while in kernel mode
> fault virtual address =3D 0xc11e4402
> fault code =3D supervisor write, page not present
> instruction pointer =3D 0x8:0xc018ffcf
> stack pointer =3D 0x10:0xc02fa6f0
> frame pointer =3D 0x10:0xc02fa704
> code segment =3D base 0x0, limit 0xfffff, type 0x1b
> =3D DPL 0, pres 1, def32 1, gran 1
> processor eflags =3D interrupt enabled, resume, IOPL =3D 0
> current process =3D Idle
> interrupt mask =3D net tty bio cam=20
> trap number =3D 12
> panic: page fault
> Uptime: 6h5m10s
> twe0: Cannot delete unit. error =3D 16
>=20
> Fatal trap 12: page fault while in kernel mode
> fault virtual address =3D 0xc11e4402
> fault code =3D supervisor write, page not present
> instruction pointer =3D 0x8:0xc018ffcf
> stack pointer =3D 0x10:0xc02fa444
> frame pointer =3D 0x10:0xc02fa458
> code segment =3D base 0x0, limit 0xfffff, type 0x1b
> =3D DPL 0, pres 1, def32 1, gran 1
> processor eflags =3D interrupt enabled, resume, IOPL =3D 0
> current process =3D Idle
> interrupt mask =3D net tty bio cam=20
> tx0, limit 0xfffff, type 0x1b
>=20
> nm -n /kernel | grep c018f
> c018f058 T accept_filt_del
> c018f07c T accept_filt_get
> c018f0b4 T accept_filt_generic_mod_event
> c018f134 t net_init_domain
> c018f1bc T net_add_domain
> c018f1ec t domaininit
> c018f244 T pffindtype
> c018f290 T pffindproto
> c018f304 T pfctlinput
> c018f34c T pfctlinput2
> c018f3a4 t pfslowtimo
> c018f3fc t pffasttimo
> c018f45c t tunable_mbinit
> c018f4ac t mbinit
> c018f53c T m_mballoc
> c018f5f8 T m_mballoc_wait
> c018f7e8 T m_clalloc
> c018f8b4 T m_clalloc_wait
> c018f9a0 T m_retry
> c018fa74 T m_retryhdr
> c018fb60 t m_reclaim
> c018fbb0 T m_get
> c018fc54 T m_gethdr
> c018fd0c T m_getclr
> c018fdd0 T m_getcl
>=20
> -------------------------------------------------------------------
>=20
> Fatal trap 12: page fault while in kernel mode
> fault virtual address =3D 0x28067100
> fault code =3D supervisor read, page not present
> instruction pointer =3D 0x8:0xc0192696
> stack pointer =3D 0x10:0xd71cbbd0
> frame pointer =3D 0x10:0xd71cbbd8
> code segment =3D base 0x0, limit 0xfffff, type 0x1b
> =3D DPL 0, pres 1, def32 1, gran 1
> processor eflags =3D interrupt enabled, resume, IOPL =3D 0
> current process =3D 110 (ppp)
> interrupt mask =3D net tty=20
> trap number =3D 12
> panic: page fault
>=20
> syncing disks... 7
> done
> Uptime: 25m51s
> twe0: Cannot delete unit. error =3D 16
>=20
> dumping to dev #twed/0x20001, offset 3146240 dump 511 510 509=20
> 508 507 506 505 504 503 502 501 500 499 498 497 496 495 494=20
> 493 492 491 490 489 488 487 486 485 484 483 482 481 480 479=20
> 478 477 476 475 474 473 472 471 470 469 468 467 466 465 464=20
> 463 462 461 460 459 458 457 456 455 454 453 452 451 450 449=20
> 448 447 446 445 444 443 442 441 440 439 438 437 436 435 434=20
> 433 432 431 430 429 428 427 426 425 424 423 422 421 420 419=20
> 418 417 416 415 414 413 412 411 410 409 408 407 406 405 404=20
> 403 402 401 400 399 398 397 396 395 394 393 392 391 390 389=20
> 388 387 386 385 384 383 382 381 380 379 378 377 376 375 374=20
> 373 372 371 370 369 368 367 366 365 364 363 362 361 360 359=20
> 358 357 356 355 354 353 352 351 350 349 348 347 346 345 344=20
> 343 342 341 340 339 338 337 336 335 334 333 332 331 330 329=20
> 328 327 326 325 324 323 322 321 320 319 318 317 316 315 314=20
> 313 312 311 310 309 308 307 306 305 304 303 302 301 300 299=20
> 298 297 296 295 294 293 292 291 290 289 288 287 286 285 284=20
> 283 282 281 280 279 278 277 276 275 274 273 272 271 270 269=20
> 268 267 266 265 264 263 262 261 260 259 258 257 256 255 254=20
> 253 252 251 250 249 248 247 246 245 244 243 242 241 240 239=20
> 238 237 236 235 234 233 232 231 230 229 228 227 226 225 224=20
> 223 222 221 220 219 218 217 216 215 214 213 212 211 210 209=20
> 208 207 206 205 204 203 202 201 200 199 198 197 196 195 194=20
> 193 192 191 190 189 188 187 186 185 184 183 182 181 180 179=20
> 178 177 176 175 174 173 172 171 170 169 168 167 166 165 164=20
> 163 162 161 160 159 158 157 156 155 154 153 152 151 150 149=20
> 148 147 146 145 144 143 142 141 140 139 138 137 136 135 134=20
> 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119=20
> 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104=20
> 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85=20
> 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65=20
> 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45=20
> 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25=20
> 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2=20
> 1 0 succeeded Automatic reboot in 15 seconds - press a key on=20
> the console to abort Rebooting...
>=20
> nm -n /kernel | grep c01926
> c0192600 T m_tag_free
> c0192614 T m_tag_prepend
> c0192628 T m_tag_unlink
> c0192658 T m_tag_delete
> c0192674 T m_tag_delete_chain
> c01926ac T m_tag_locate
> c01926e8 T m_tag_copy
>=20
> ---------------
>=20
> (kgdb) bt
> #0 dumpsys () at ../../kern/kern_shutdown.c:487
> #1 0xc0173f3f in boot (howto=3D256) at ../../kern/kern_shutdown.c:316
> #2 0xc0174364 in poweroff_wait (junk=3D0xc02f2aac,=20
> howto=3D-1070651985) at ../../kern/kern_shutdown.c:595
> #3 0xc02a77ba in trap_fatal (frame=3D0xd71cbb90,=20
> eva=3D671510784) at ../../i386/i386/trap.c:974
> #4 0xc02a748d in trap_pfault (frame=3D0xd71cbb90, usermode=3D0,=20
> eva=3D671510784) at ../../i386/i386/trap.c:867
> #5 0xc02a704b in trap (frame=3D{tf_fs =3D -1072562160, tf_es =3D=20
> 16, tf_ds =3D -686030832, tf_edi =3D 6757530, tf_esi =3D -1055667456,=20
> tf_ebp =3D -685982760, tf_isp =3D -685982788, tf_ebx =3D=20
> 671510784, tf_edx =3D 6757530, tf_ecx =3D -1056411648, tf_eax =3D =
28672,=20
> tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1072093546, tf_cs=20
> =3D 8, tf_eflags =3D 66054, tf_esp =3D -1055667456, tf_ss =3D =
-1055667456})
> at ../../i386/i386/trap.c:466
> #6 0xc0192696 in m_tag_delete_chain (m=3D0xc113cb00, t=3D0x0) at=20
> ../../kern/uipc_mbuf2.c:358
> #7 0xc01904d3 in m_free (m=3D0xc113cb00) at =
../../kern/uipc_mbuf.c:734
> #8 0xc0190606 in m_freem (m=3D0xc1094000) at =
../../kern/uipc_mbuf.c:763
> #9 0xc0127df8 in fr_check (ip=3D0xc1094030, hlen=3D20,=20
> ifp=3D0xc21a7008, out=3D0, mp=3D0xd71cbce8)
> at ../../contrib/ipfilter/netinet/fil.c:1387
> #10 0xc01d7d06 in ip_input (m=3D0xc1094000) at=20
> ../../netinet/ip_input.c:478
> #11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
> #12 0xc0299ee9 in swi_net_next ()
> #13 0xc016e7c8 in lockmgr (lkp=3D0xc21a7100, flags=3D16973826,=20
> interlkp=3D0xd71c0bac, p=3D0xd48bd5a0) at ../../kern/kern_lock.c:355
> #14 0xc019fda8 in vop_stdlock (ap=3D0xd71cbdd0) at=20
> ../../kern/vfs_default.c:256
> #15 0xc025a9c9 in ufs_vnoperatespec (ap=3D0xd71cbdd0) at=20
> ../../ufs/ufs/ufs_vnops.c:2394
> #16 0xc01a9ffd in vn_lock (vp=3D0xd71c0b40, flags=3D131074,=20
> p=3D0xd48bd5a0) at vnode_if.h:861
> #17 0xc01ad842 in spec_write (ap=3D0xd71cbe64) at=20
> ../../miscfs/specfs/spec_vnops.c:284
> #18 0xc025a3ac in ufsspec_write (ap=3D0xd71cbe64) at=20
> ../../ufs/ufs/ufs_vnops.c:1827
> #19 0xc025a9c9 in ufs_vnoperatespec (ap=3D0xd71cbe64) at=20
> ../../ufs/ufs/ufs_vnops.c:2394 #20 0xc01a9b9a in vn_write=20
> (fp=3D0xc219b100, uio=3D0xd71cbed4, cred=3D0xc2197080, flags=3D0,=20
> p=3D0xd48bd5a0) at vnode_if.h:363
> #21 0xc018330d in dofilewrite (p=3D0xd48bd5a0, fp=3D0xc219b100,=20
> fd=3D9, buf=3D0xbfbfe89c, nbyte=3D580, offset=3D-1, flags=3D0)
> at ../../sys/file.h:163
> #22 0xc01831c6 in write (p=3D0xd48bd5a0, uap=3D0xd71cbf80) at=20
> ../../kern/sys_generic.c:329
> #23 0xc02a7a69 in syscall2 (frame=3D{tf_fs =3D -1078001617, tf_es=20
> =3D 134938671, tf_ds =3D -1078001617, tf_edi =3D 135090176, tf_esi =3D =
580,=20
> tf_ebp =3D -1077940064, tf_isp =3D -685981740, tf_ebx =3D=20
> -1077942112, tf_edx =3D 0, tf_ecx =3D 13, tf_eax =3D 4, tf_trapno =3D =
7,=20
> tf_err =3D 2, tf_eip =3D 673683504, tf_cs =3D 31, tf_eflags =3D=20
> 663, tf_esp =3D -1077942172, tf_ss =3D 47})
> at ../../i386/i386/trap.c:1175
> #24 0xc0298a85 in Xint0x80_syscall ()
> #25 0x80655de in ?? ()
> #26 0x806c2fb in ?? ()
> #27 0x806c21d in ?? ()
> #28 0x807470a in ?? ()
> #29 0x8083a78 in ?? ()
> #30 0x805b84b in ?? ()
> #31 0x804d484 in ?? ()
> #32 0x806ed77 in ?? ()
> #33 0x806e967 in ?? ()
> #34 0x804b62a in ?? ()
>=20
> (kgdb) f 11
> #11 0xc01d838b in ipintr () at ../../netinet/ip_input.c:971
> 971 ip_input(m);
> (kgdb) print *m
> $13 =3D {m_hdr =3D {mh_next =3D 0xc1128100, mh_nextpkt =3D 0x0,=20
> mh_data =3D 0xc1094030 "E", mh_len =3D 208, mh_type =3D 0, mh_flags =
=3D 2},
> M_dat =3D {MH =3D {MH_pkthdr =3D {rcvif =3D 0xc21a7008, len =3D 576, =
> header =3D 0x0, csum_flags =3D 0, csum_data =3D 0, tags =3D {
> slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {ext_buf =3D=20
> 0x2000000 <Address 0x2000000 out of bounds>, ext_free =3D 0x2400045,=20
> ext_size =3D 28454, ext_ref =3D 0x6d3d012e},=20
> MH_databuf =3D=20
> =
"\000\000\000\002E\000@\002&o\000\000.\001=3Dm;=BD=F30=D8=B6\037=3D\003\0=
0
> =
1=BA\006\000\000\000\000E\000\005=C8=E9=BA@\000/\006=A2t=D8=B6\037=3D=C0=A8=
\001e'B
> \rmy=BB6\216^\222@\002P\020=E1\000=B1=A1\000\000\000\000@\t\a\000\000\
> =
002r\000\003=C0\000\227=AB\212!\225@\204]\001\214\027=C1=F9\177\232=F9=E3=
=F2
> =
\222\016\000=F9au1=3D\216=DD\204=A8\207O\002+=F80=E9H=F0\eD\2056n\001=F7U=
\025=E0
> =
\222=FE\f:S=F5PI\037)T=D8(=FD=A8\r=D3@\210=EA\217(S=F5c=E2=E9=B8\n?\217%x=
&=B5\177=F4UqX\
> =
222\020\225\\=D1=D9~\fh=EA=A9\036\t\"Az\206=E1p=FE+}=C7=A3=A3=A2"...}},=20
> M_databuf =3D "\bp\032=C2@\002", '\000' <repeats 21 times>,=20
> =
"\002E\000@\002&o\000\000.\001=3Dm;=BD=F30=D8=B6\037=3D\003\001=BA\006\00=
0\0
> =
00\000\000E\000\005=C8=E9=BA@\000/\006=A2t=D8=B6\037=3D=C0=A8\001e'B\rmy=BB=
6\216^\
> 222@\002P\020=E1\000=B1=A1\000\000\000\000@\t\a\000\000\002r\000\003
> =
=C0\000\227=AB\212!\225@\204]\001\214\027=C1=F9\177\232=F9=E3=F2\222\016\=
000
> =
=F9au1=3D\216=DD\204=A8\207O\002+=F80=E9H=F0\eD\2056n\001=F7U\025=E0\222=FE=
\f:S=F5PI
> =
\037)T=D8(=FD=A8\r=D3@\210=EA\217(S=F5c=E2=E9=B8\n?\217%x&=B5\177=F4UqX\2=
22\020\225\
> \=D1=D9~\fh=EA=A9\036\t"...}}
> (kgdb) f 6
> #6 0xc0192696 in m_tag_delete_chain (m=3D0xc113cb00, t=3D0x0) at=20
> ../../kern/uipc_mbuf2.c:358
> 358 m_tag_delete(m, q);
> (kgdb) print *m
> $14 =3D {m_hdr =3D {mh_next =3D 0xc113ca00, mh_nextpkt =3D=20
> 0x280ef4cd, mh_data =3D 0x14 <Address 0x14 out of bounds>,=20
> mh_len =3D 663,=20
> mh_type =3D 28672, mh_flags =3D 10246}, M_dat =3D {MH =3D=20
> {MH_pkthdr =3D {rcvif =3D 0x280ef492, len =3D 672120748, header =3D =
0x2,=20
> csum_flags =3D 16384, csum_data =3D 1, tags =3D {slh_first=20
> =3D 0x28067100}}, MH_dat =3D {MH_ext =3D {
> ext_buf =3D 0x28067200 <Address 0x28067200 out of=20
> bounds>, ext_free =3D 0x280541fd, ext_size =3D 134516476,=20
> ext_ref =3D 0x280819da},=20
> MH_databuf =3D=20
> =
"\000r\006(=FDA\005(=FC\216\004\b=DA\031\b(\000\000\000\000=A2A\005(=A8:
> \006(@\200\006(\000\000\000\000\000\000\000\000`=FB=BF=BF@\200\006\0
> =
01\234=FB=BF=BFOA\005(=FC\216\004\b\004=CFe\000\000r\006(\001\000\000\00
> 0=A8:\006(\000p\006(=FC\216\004\b=FDA\005(=FC\216\004\b\t\013\005(\200
> =
=E6\020(=A2A\005(=A8:\006(\000=E9\a(\200=3D\006(\227?\005(5(\005(=A8:\006=
(
> \f=FC=BF=BF=CF@\005(=20
> \006\005(\004=CFe\000\200=3D\006(\001\000\000\000\000p\006(\000q\0
> =
06(\000r\006(=DA>\005(=A8:\006(\000p\006(=FC\216\004\b=D0=FC=BF=BFS=D4\00=
4\b\2
> 00=E6\020(@ \005\b\000r\006("...}},=20
> M_databuf =3D=20
> "\222=F4\016(=AC=BF\017(\002\000\000\000\000@\000\000\001\000\000\00
> 0\000q\006(\000r\006(=FDA\005(=FC\216\004\b=DA\031\b(\000\000\000\00
> =
0=A2A\005(=A8:\006(@\200\006(\000\000\000\000\000\000\000\000`=FB=BF=BF@
> =
\200\006\001\234=FB=BF=BFOA\005(=FC\216\004\b\004=CFe\000\000r\006(\001\
> 000\000\000=A8:\006(\000p\006(=FC\216\004\b=FDA\005(=FC\216\004\b\t\01
> =
3\005(\200=E6\020(=A2A\005(=A8:\006(\000=E9\a(\200=3D\006(\227?\005(5(\0
> 05(=A8:\006(\f=FC=BF=BF=CF@\005(=20
> \006\005(\004=CFe\000\200=3D\006(\001\000\000\000\000p\006(\000q\0
> 06(\000r\006(=DA>\005(=A8:\006(\000p\006("...}}
> (kgdb) print *q
> Cannot access memory at address 0x0.
> (kgdb)
> ----------------------------
> Copyright (c) 1992-2005 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992,=20
> 1993, 1994
> The Regents of the University of California. All=20
> rights reserved.
> FreeBSD 4.11-RELEASE-p13 #3: Thu Feb 23 13:09:31 EST 2006
> damascus@daemon.faerunhome.com:/usr/src/sys/compile/DAEMON
> Timecounter "i8254" frequency 1193182 Hz
> CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (1993.54-MHz 686-class CPU)
> Origin =3D "GenuineIntel" Id =3D 0xf24 Stepping =3D 4
> =20
> Features=3D0x3febfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SE
> P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SS
> E2,SS,HTT,TM>
> real memory =3D 536608768 (524032K bytes) avail memory =3D=20
> 518377472 (506228K bytes) Preloaded elf kernel "kernel" at 0xc03af000.
> Warning: Pentium 4 CPU: PSE disabled
> Pentium Pro MTRR support enabled
> md0: Malloc disk
> Using $PIR table, 11 entries at 0xc00f28c0
> npx0: <math processor> on motherboard
> npx0: INT 16 interface
> pcib0: <Intel 82845 Host to PCI bridge> on motherboard
> pci0: <PCI bus> on pcib0
> pcib1: <Intel 82845 PCI-PCI (AGP) bridge> at device 1.0 on pci0
> pci1: <PCI bus> on pcib1
> pcib2: <Intel 82801BA/BAM (ICH2) Hub to PCI bridge> at device=20
> 30.0 on pci0
> pci2: <PCI bus> on pcib2
> twe0: <3ware Storage Controller driver ver. 1.40.01.002> port=20
> 0xdfa0-0xdfaf mem 0xfe000000-0xfe7fffff,0xfeafec00-0xfeafec0f=20
> irq 9 at device 9.0 on pci2
> twe0: 4 ports, Firmware FE7X 1.05.00.068, BIOS BE7X 1.08.00.048
> fxp0: <Intel 82550 Pro/100 Ethernet> port 0xdf00-0xdf3f mem=20
> 0xfeaa0000-0xfeabffff,0xfeafd000-0xfeafdfff irq 11 at device=20
> 10.0 on pci2
> fxp0: Ethernet address 00:02:b3:d0:e3:73
> inphy0: <i82555 10/100 media interface> on miibus0
> inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> fxp1: <Intel 82550 Pro/100 Ethernet> port 0xde80-0xdebf mem=20
> 0xfea80000-0xfea9ffff,0xfeafc000-0xfeafcfff irq 10 at device=20
> 11.0 on pci2
> fxp1: Ethernet address 00:02:b3:ee:65:88
> inphy1: <i82555 10/100 media interface> on miibus1
> inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> fxp2: <Intel 82550 Pro/100 Ethernet> port 0xdd80-0xddbf mem=20
> 0xfea40000-0xfea5ffff,0xfeafb000-0xfeafbfff irq 11 at device=20
> 12.0 on pci2
> fxp2: Ethernet address 00:11:11:c1:a2:e5
> inphy2: <i82555 10/100 media interface> on miibus2
> inphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> fxp3: <Intel 82550 Pro/100 Ethernet> port 0xdd00-0xdd3f mem=20
> 0xfea20000-0xfea3ffff,0xfeafa000-0xfeafafff irq 11 at device=20
> 13.0 on pci2
> fxp3: Ethernet address 00:11:11:c1:a2:e7
> inphy3: <i82555 10/100 media interface> on miibus3
> inphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> atapci0: <Promise ATA100 controller> port=20
> 0xdcc0-0xdcff,0xdfe0-0xdfe3,0xdf98-0xdf9f,0xdfe4-0xdfe7,0xdff0
> -0xdff7 mem 0xfe9e0000-0xfe9fffff irq 11 at device 14.0 on pci2
> ata2: at 0xdff0 on atapci0
> ata3: at 0xdf98 on atapci0
> pci2: <ATI Mach64-GR graphics accelerator> at 15.0 irq 11
> isab0: <Intel 82801BA/BAM (ICH2) PCI to LPC bridge> at device=20
> 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci1: <Intel ICH2 ATA100 controller> port 0xffa0-0xffaf at=20
> device 31.1 on pci0
> ata0: at 0x1f0 irq 14 on atapci1
> ata1: at 0x170 irq 15 on atapci1
> uhci0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> port=20
> 0xef40-0xef5f irq 11 at device 31.2 on pci0
> usb0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> on uhci0
> usb0: USB revision 1.0
> uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub0: 2 ports with 2 removable, self powered
> pci0: <unknown card> (vendor=3D0x8086, dev=3D0x2443) at 31.3 irq 11
> uhci1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> port=20
> 0xef80-0xef9f irq 10 at device 31.4 on pci0
> usb1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> on uhci1
> usb1: USB revision 1.0
> uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub1: 2 ports with 2 removable, self powered
> orm0: <Option ROMs> at iomem=20
> 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9800-0xcafff,0xcb000-0xcbff
> f,0xcc000-0xccfff on isa0 pmtimer0 on isa0
> fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6=20
> drq 2 on isa0
> fdc0: FIFO enabled, 8 bytes threshold
> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: model VersaPad, device ID 0
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem=20
> 0xa0000-0xbffff on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=3D0x100> sio0 at port=20
> 0x3f8-0x3ff irq 4 flags 0x10 on isa0
> sio0: type 16550A, console
> sio1 at port 0x2f8-0x2ff irq 3 on isa0
> sio1: type 16550A
> ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
> ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
> ppbus0: IEEE1284 device found /NIBBLE/PS2/ECP Probing for PnP=20
> devices on ppbus0:
> ppbus0: <Lexmark International Lexmark E323> PRINTER PCL 6=20
> Emulation, PostScript Level 3 Emulation, NPAP, PJL
> plip0: <PLIP network interface> on ppbus0
> lpt0: <Printer> on ppbus0
> lpt0: Interrupt-driven port
> ppi0: <Parallel I/O> on ppbus0
> DUMMYNET initialized (011031)
> IP packet filtering initialized, divert enabled, rule-based=20
> forwarding enabled, default to deny, unlimited logging IP=20
> Filter: v3.4.35 initialized. Default =3D pass all, Logging =3D =
enabled
> acd0: CDROM <GCR-8521B> at ata0-master PIO4
> twed0: <Unit 0, TwinStor, Normal> on twe0
> twed0: 152626MB (312579760 sectors)
> twed1: <Unit 2, JBOD, Normal> on twe0
> twed1: 78167MB (160086528 sectors)
> twed2: <Unit 3, JBOD, Normal> on twe0
> twed2: 152627MB (312581808 sectors)
> Mounting root from ufs:/dev/twed0s1a
> WARNING: / was not properly dismounted
> -------------------------------------------------
> daemon# cat /usr/src/sys/i386/conf/DAEMON # # GENERIC --=20
> Generic kernel configuration file for FreeBSD/i386 # # For=20
> more information on this file, please read the handbook=20
> section on # Kernel Configuration Files:
> #
> # =20
> http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kern
> elconfig-config.html
> #
> # The handbook is also available locally in=20
> /usr/share/doc/handbook # if you've installed the doc=20
> distribution, otherwise always see the # FreeBSD World Wide=20
> Web server (http://www.FreeBSD.org/) for the # latest information.
> #
> # An exhaustive list of options and more detailed=20
> explanations of the # device lines is also present in the=20
> ./LINT configuration file. If you are # in doubt as to the=20
> purpose or necessity of a line, check first in LINT.
> #
> # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.246.2.48 2002/08/31=20
> 20:28:26 obrien Exp $
>=20
> machine i386
> cpu I386_CPU
> cpu I486_CPU
> cpu I586_CPU
> cpu I686_CPU
> ident "DAEMON"
> maxusers 0
>=20
> makeoptions DEBUG=3D-g #Build kernel with=20
> gdb(1) debug symbols
>=20
> #options DDB
> options MATH_EMULATE #Support for x87 emulation
> options INET #InterNETworking
> options INET6 #IPv6 communications protocols
> options FFS #Berkeley Fast Filesystem
> options FFS_ROOT #FFS usable as root=20
> device [keep this!]
> options SOFTUPDATES #Enable FFS soft=20
> updates support
> options UFS_DIRHASH #Improve performance=20
> on big directories
> options MFS #Memory Filesystem
> options MD_ROOT #MD is a potential root device
> options NFS #Network Filesystem
> options NFS_ROOT #NFS usable as root=20
> device, NFS required
> options MSDOSFS #MSDOS Filesystem
> options CD9660 #ISO 9660 Filesystem
> options CD9660_ROOT #CD-ROM usable as=20
> root, CD9660 required
> options PROCFS #Process filesystem
> options COMPAT_43 #Compatible with BSD=20
> 4.3 [KEEP THIS!]
> options SCSI_DELAY=3D15000 #Delay (in ms) before=20
> probing SCSI
> options UCONSOLE #Allow users to grab=20
> the console
> options USERCONFIG #boot -c editor
> options VISUAL_USERCONFIG #visual boot -c editor
> options KTRACE #ktrace(1) support
> options SYSVSHM #SYSV-style shared memory
> options SYSVMSG #SYSV-style message queues
> options SYSVSEM #SYSV-style semaphores
> options P1003_1B #Posix P1003_1B=20
> real-time extensions
> options _KPOSIX_PRIORITY_SCHEDULING
> options ICMP_BANDLIM #Rate limit bad replies
> options KBD_INSTALL_CDEV # install a CDEV entry in /dev
> options AHC_REG_PRETTY_PRINT # Print register=20
> bitfields in debug
> # output. Adds ~128k=20
> to driver.
> options AHD_REG_PRETTY_PRINT # Print register=20
> bitfields in debug=20
> # output. Adds ~215k=20
> to driver.
>=20
> # To make an SMP kernel, the next two are needed
> #options SMP # Symmetric=20
> MultiProcessor Kernel
> #options APIC_IO # Symmetric (APIC) I/O
>=20
> device isa
> device pci
>=20
> # Floppy drives
> device fdc0 at isa? port IO_FD1 irq 6 drq 2
> device fd0 at fdc0 drive 0
> device fd1 at fdc0 drive 1
> #
> # If you have a Toshiba Libretto with its Y-E Data PCMCIA=20
> floppy, # don't use the above line for fdc0 but the following one:
> #device fdc0
>=20
> # ATA and ATAPI devices
> device ata0 at isa? port IO_WD1 irq 14
> device ata1 at isa? port IO_WD2 irq 15
> device ata
> device atadisk # ATA disk drives
> device atapicd # ATAPI CDROM drives
> device atapifd # ATAPI floppy drives
> device atapist # ATAPI tape drives
> options ATA_STATIC_ID #Static device numbering
>=20
> # RAID controllers
> device twe # 3ware Escalade
>=20
> # atkbdc0 controls both the keyboard and the PS/2 mouse
> device atkbdc0 at isa? port IO_KBD
> device atkbd0 at atkbdc? irq 1=20
> device psm0 at atkbdc? irq 12
>=20
> device vga0 at isa?
>=20
> # splash screen/screen saver
> pseudo-device splash
>=20
> # syscons is the default console driver, resembling an SCO console
> device sc0 at isa? flags 0x100
>=20
> # Floating point support - do not disable.
> device npx0 at nexus? port IO_NPX irq 13
>=20
> # Power management support (see LINT for more options)
> device apm0 at nexus? disable flags 0x20 #=20
> Advanced Power Management
>=20
> # Serial (COM) ports
> device sio0 at isa? port IO_COM1 flags 0x10 irq 4
> device sio1 at isa? port IO_COM2 irq 3
>=20
> # Parallel port
> device ppc0 at isa? irq 7
> device ppbus # Parallel port bus (required)
> device lpt # Printer
> device plip # TCP/IP over parallel
> device ppi # Parallel port interface device
> #device vpo # Requires scbus and da
>=20
>=20
> # PCI Ethernet NICs.
> device de # DEC/Intel DC21x4x (``Tulip'')
>=20
> # PCI Ethernet NICs that use the common MII bus controller code.
> # NOTE: Be sure to keep the 'device miibus' line in order to=20
> use these NICs!
> device miibus # MII bus support
> device fxp # Intel EtherExpress PRO/100B=20
> (82557, 82558)
>=20
> # Pseudo devices - the number indicates how many units to allocate.
> pseudo-device loop # Network loopback
> pseudo-device ether # Ethernet support
> pseudo-device sl 1 # Kernel SLIP
> pseudo-device ppp 1 # Kernel PPP
> pseudo-device tun # Packet tunnel.
> pseudo-device pty # Pseudo-ttys (telnet etc)
> pseudo-device md # Memory "disks"
> pseudo-device gif # IPv6 and IPv4 tunneling
> pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation)
>=20
> # The `bpf' pseudo-device enables the Berkeley Packet Filter.
> # Be aware of the administrative consequences of enabling this!
> pseudo-device bpf #Berkeley packet filter
>=20
> # USB support
> device uhci # UHCI PCI->USB interface
> device ohci # OHCI PCI->USB interface
> device usb # USB Bus (required)
> device ugen # Generic
> device uhid # "Human Interface Devices"
> device ukbd # Keyboard
> device ulpt # Printer
> device ums # Mouse
> device uscanner # Scanners
> device urio # Diamond Rio MP3 Player
> # USB Ethernet, requires mii
> device aue # ADMtek USB ethernet
> device cue # CATC USB ethernet
> device kue # Kawasaki LSI USB ethernet
>=20
> #NATD
> options IPFIREWALL
> options IPDIVERT
> options IPFIREWALL_FORWARD
>=20
> #DUMMYNET
> options IPFIREWALL_VERBOSE
> options DUMMYNET
>=20
> #NMBCLUSTER
> options NMBCLUSTERS=3D5120
>=20
> #IPFILTER
> options IPFILTER
> options IPFILTER_LOG
>=20
> #SysV Memory
>=20
> options SEMMAP=3D31
> options SEMMNI=3D70
> options SEMMNS=3D200
> options SEMMNU=3D31
> options SEMMSL=3D61
> options SEMOPM=3D101
> options SEMUME=3D11
> options SHMMAXPGS=3D16384
> options SHMSEG=3D10
>=20
> #pseudo
> pseudo-device vn
> pseudo-device snp 3
>=20
> options CONSPEED=3D115200 #default speed for=20
> serial console (default 9600)
> #options IPSEC #IP security
> #options IPSEC_ESP #IP security=20
> (crypto; define w/ IPSEC)
> #options IPSEC_DEBUG #debug for IP security
>=20
> options NETGRAPH #netgraph(4) system
> options NETGRAPH_PPPOE=20
> options NETGRAPH_SOCKET
> options NETGRAPH_ETHER
> -----------------
> FreeBSD daemon.faerunhome.com 4.11-RELEASE-p13 FreeBSD=20
> 4.11-RELEASE-p13 #3: Thu Feb 23 13:09:31 EST 2006 =20
> damascus@daemon.faerunhome.com:/usr/src/sys/compile/DAEMON i386
>=20
>=20
>=20
>=20
>=20
> - Carroll Kong
>=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060306173558.A19AC43D60>