Date: Fri, 20 Jul 2012 17:43:51 -0700 From: Sean Bruno <seanbru@yahoo-inc.com> To: Andriy Gapon <avg@FreeBSD.org> Cc: "sbruno@FreeBSD.org" <sbruno@FreeBSD.org>, "freebsd-stable@freebsd.org" <freebsd-stable@FreeBSD.org>, John Baldwin <jhb@FreeBSD.org> Subject: Re: stable/9 panic Bad tailq NEXT(0xffffffff80e52660->tqh_last) != NULL Message-ID: <1342831431.2645.1.camel@powernoodle.corp.yahoo.com> In-Reply-To: <5003E14E.5020302@FreeBSD.org> References: <1342197082.2664.4.camel@powernoodle.corp.yahoo.com> <5003E14E.5020302@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 2012-07-16 at 02:39 -0700, Andriy Gapon wrote: > on 13/07/2012 19:31 Sean Bruno said the following: > > Well this is new. I haven't a clue what Dell has done on this R620, but > > this popped up today after I did a boat load of BIOS updates and tried > > to install stable/9 from our yahoo tree. If anyone sees the obvious > > solution here, I'd love to figure it out. > > > > found-> vendor=0x14e4, dev=0x165f, revid=0x00 > > domain=0, bus=2, slot=0, func=1 > > class=02-00-00, hdrtype=0x00, mfdev=1 > > cmdreg=0x0006, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=b, irq=6 > > powerspec 3 supports D0 D3 current D0 > > MSI supports 8 messages, 64 bit > > MSI-X supports 17 messages in map 0x20 > > map[10]: type Prefetchable Memory, range 64, base 0xd50d0000, > > size 16, enabled > > pcib1: allocated prefetch range (0xd50d0000-0xd50dffff) for rid 10 of > > pci0:2:0:1 > > map[18]: type Prefetchable Memory, range 64, base 0xd50e0000, > > size 16, enabled > > pcib1: allocated prefetch range (0xd50e0000-0xd50effff) for rid 18 of > > pci0:2:0:1 > > map[20]: type Prefetchable Memory, range 64, base 0xd50f0000, > > size 16, enabled > > pcib1: allocated prefetch range (0xd50f0000-0xd50fffff) for rid 20 of > > pci0:2:0:1 > > pcib1: matched entry for 2.0.INTB > > pcib1: slot 0 INTB hardwired to IRQ 36 > > bge0: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem > > 0xd50a0000-0xd50affff,0xd50b0000-0xd50bffff,0xd50c0000-0xd50cffff irq 34 > > at device 0.0 on pci2 > > bge0: APE FW version: NCSI v1.0.80.0 > > bge0: attempting to allocate 1 MSI vectors (8 supported) > > msi: routing MSI IRQ 264 to local APIC 0 vector 59 > > bge0: using IRQ 264 for MSI > > bge0: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E > > bge0: Disabling fastboot > > bge0: Disabling fastboot > > miibus0: <MII bus> on bge0 > > brgphy0: <BCM5720C 1000BASE-T media interface> PHY 1 on miibus0 > > brgphy0: OUI 0x001be9, model 0x0036, rev. 0 > > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow > > bge0: bpf attached > > bge0: Ethernet address: 18:03:73:fd:9e:36 > > bge1: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem > > 0xd50d0000-0xd50dffff,0xd50e0000-0xd50effff,0xd50f0000-0xd50fffff irq 36 > > at device 0.1 on pci2 > > bge1: APE FW version: NCSI v1.0.80.0 > > bge1: attempting to allocate 1 MSI vectors (8 supported) > > msi: routing MSI IRQ 265 to local APIC 0 vector 60 > > bge1: using IRQ 265 for MSI > > bge1: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E > > bge1: Disabling fastboot > > bge1: Disabling fastboot > > miibus1: <MII bus> on bge1 > > brgphy1: <BCM5720C 1000BASE-T media interface> PHY 2 on miibus1 > > brgphy1: OUI 0x001be9, model 0x0036, rev. 0 > > brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow > > bge1: bpf attached > > bge1: Ethernet address: 18:03:73:fd:9e:37 > > pcib2: <ACPI PCI-PCI bridge> irq 53 at device 1.1 on pci0 > > pcib0: allocated type 3 (0xd8800000-0xd8ffffff) for rid 20 of pcib2 > > pcib0: allocated type 3 (0xd5100000-0xd51fffff) for rid 24 of pcib2 > > pcib2: domain 0 > > pcib2: secondary bus 1 > > pcib2: subordinate bus 1 > > pcib2: memory decode 0xd8800000-0xd8ffffff > > pcib2: prefetched decode 0xd5100000-0xd51fffff > > pci1: <ACPI PCI bus> on pcib2 > > pci1: domain=0, physical bus=1 > > found-> vendor=0x14e4, dev=0x165f, revid=0x00 > > domain=0, bus=1, slot=0, func=0 > > class=02-00-00, hdrtype=0x00, mfdev=1 > > cmdreg=0x0006, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=a, irq=15 > > powerspec 3 supports D0 D3 current D0 > > MSI supports 8 messages, 64 bit > > MSI-X supports 17 messages in map 0x20 > > map[10]: type Prefetchable Memory, range 64, base 0xd51a0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51a0000-0xd51affff) for rid 10 of > > pci0:1:0:0 > > map[18]: type Prefetchable Memory, range 64, base 0xd51b0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51b0000-0xd51bffff) for rid 18 of > > pci0:1:0:0 > > map[20]: type Prefetchable Memory, range 64, base 0xd51c0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51c0000-0xd51cffff) for rid 20 of > > pci0:1:0:0 > > pcib2: matched entry for 1.0.INTA > > pcib2: slot 0 INTA hardwired to IRQ 35 > > found-> vendor=0x14e4, dev=0x165f, revid=0x00 > > domain=0, bus=1, slot=0, func=1 > > class=02-00-00, hdrtype=0x00, mfdev=1 > > cmdreg=0x0006, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=b, irq=6 > > powerspec 3 supports D0 D3 current D0 > > MSI supports 8 messages, 64 bit > > MSI-X supports 17 messages in map 0x20 > > map[10]: type Prefetchable Memory, range 64, base 0xd51d0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51d0000-0xd51dffff) for rid 10 of > > pci0:1:0:1 > > map[18]: type Prefetchable Memory, range 64, base 0xd51e0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51e0000-0xd51effff) for rid 18 of > > pci0:1:0:1 > > map[20]: type Prefetchable Memory, range 64, base 0xd51f0000, > > size 16, enabled > > pcib2: allocated prefetch range (0xd51f0000-0xd51fffff) for rid 20 of > > pci0:1:0:1 > > pcib2: matched entry for 1.0.INTB > > pcib2: slot 0 INTB hardwired to IRQ 38 > > bge2: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem > > 0xd51a0000-0xd51affff,0xd51b0000-0xd51bffff,0xd51c0000-0xd51cffff irq 35 > > at device 0.0 on pci1 > > bge2: APE FW version: NCSI v1.0.80.0 > > bge2: attempting to allocate 1 MSI vectors (8 supported) > > msi: routing MSI IRQ 266 to local APIC 0 vector 61 > > bge2: using IRQ 266 for MSI > > bge2: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E > > bge2: Disabling fastboot > > bge2: Disabling fastboot > > miibus2: <MII bus> on bge2 > > brgphy2: <BCM5720C 1000BASE-T media interface> PHY 1 on miibus2 > > brgphy2: OUI 0x001be9, model 0x0036, rev. 0 > > brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow > > bge2: bpf attached > > bge2: Ethernet address: 18:03:73:fd:9e:34 > > bge3: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem > > 0xd51d0000-0xd51dffff,0xd51e0000-0xd51effff,0xd51f0000-0xd51fffff irq 38 > > at device 0.1 on pci1 > > bge3: APE FW version: NCSI v1.0.80.0 > > bge3: attempting to allocate 1 MSI vectors (8 supported) > > msi: routing MSI IRQ 267 to local APIC 0 vector 62 > > bge3: using IRQ 267 for MSI > > bge3: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E > > bge3: Disabling fastboot > > bge3: Disabling fastboot > > miibus3: <MII bus> on bge3 > > brgphy3: <BCM5720C 1000BASE-T media interface> PHY 2 on miibus3 > > brgphy3: OUI 0x001be9, model 0x0036, rev. 0 > > brgphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow > > bge3: bpf attached > > bge3: Ethernet address: 18:03:73:fd:9e:35 > > pcib3: <ACPI PCI-PCI bridge> irq 53 at device 2.0 on pci0 > > pcib3: domain 0 > > pcib3: secondary bus 4 > > pcib3: subordinate bus 4 > > pcib3: no prefetched decode > > pci4: <ACPI PCI bus> on pcib3 > > pci4: domain=0, physical bus=4 > > pcib4: <ACPI PCI-PCI bridge> irq 53 at device 2.2 on pci0 > > pcib0: allocated type 4 (0xf000-0xffff) for rid 1c of pcib4 > > pcib0: allocated type 3 (0xd9000000-0xd9ffffff) for rid 20 of pcib4 > > pcib4: domain 0 > > pcib4: secondary bus 3 > > pcib4: subordinate bus 3 > > pcib4: I/O decode 0xf000-0xffff > > pcib4: memory decode 0xd9000000-0xd9ffffff > > pcib4: no prefetched decode > > pci3: <ACPI PCI bus> on pcib4 > > pci3: domain=0, physical bus=3 > > found-> vendor=0x1000, dev=0x005b, revid=0x01 > > domain=0, bus=3, slot=0, func=0 > > class=01-04-00, hdrtype=0x00, mfdev=0 > > cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=a, irq=15 > > powerspec 3 supports D0 D1 D2 D3 current D0 > > MSI supports 1 message, 64 bit > > MSI-X supports 16 messages in map 0x14 > > map[10]: type I/O Port, range 32, base 0xfc00, size 8, enabled > > pcib4: allocated I/O port range (0xfc00-0xfcff) for rid 10 of pci0:3:0:0 > > map[14]: type Memory, range 64, base 0xd9ffc000, size 14, > > enabled > > pcib4: allocated memory range (0xd9ffc000-0xd9ffffff) for rid 14 of > > pci0:3:0:0 > > map[1c]: type Memory, range 64, base 0xd9f80000, size 18, > > enabled > > pcib4: allocated memory range (0xd9f80000-0xd9fbffff) for rid 1c of > > pci0:3:0:0 > > pcib4: matched entry for 3.0.INTA > > pcib4: slot 0 INTA hardwired to IRQ 42 > > mfi0: <Dell PERC H710P Mini (monolithics)> port 0xfc00-0xfcff mem > > 0xd9ffc000-0xd9ffffff,0xd9f80000-0xd9fbffff irq 42 at device 0.0 on pci3 > > mfi0: attempting to allocate 1 MSI vectors (1 supported) > > msi: routing MSI IRQ 268 to local APIC 0 vector 63 > > mfi0: using IRQ 268 for MSI > > mfi0: Using MSI > > mfi0: Megaraid SAS driver Ver 4.23 > > mfi0: MaxCmd = 3f0 MaxSgl = 46 state = b73c03f0 > > mfi0: Max fw cmds= 1008, sizing driver pool to 128 > > mfip0: <SCSI Passthrough Bus> on mfi0 > > pcib5: <ACPI PCI-PCI bridge> irq 53 at device 3.0 on pci0 > > pcib0: allocated type 4 (0xe000-0xefff) for rid 1c of pcib5 > > pcib0: allocated type 3 (0xda000000-0xdaffffff) for rid 20 of pcib5 > > pcib5: domain 0 > > pcib5: secondary bus 5 > > pcib5: subordinate bus 5 > > pcib5: I/O decode 0xe000-0xefff > > pcib5: memory decode 0xda000000-0xdaffffff > > pcib5: no prefetched decode > > pci5: <ACPI PCI bus> on pcib5 > > pci5: domain=0, physical bus=5 > > found-> vendor=0x8086, dev=0x105e, revid=0x06 > > domain=0, bus=5, slot=0, func=0 > > class=02-00-00, hdrtype=0x00, mfdev=1 > > cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=a, irq=15 > > powerspec 2 supports D0 D3 current D0 > > MSI supports 1 message, 64 bit > > map[10]: type Memory, range 32, base 0xdaf80000, size 17, > > enabled > > pcib5: allocated memory range (0xdaf80000-0xdaf9ffff) for rid 10 of > > pci0:5:0:0 > > map[14]: type Memory, range 32, base 0xdafa0000, size 17, > > enabled > > pcib5: allocated memory range (0xdafa0000-0xdafbffff) for rid 14 of > > pci0:5:0:0 > > map[18]: type I/O Port, range 32, base 0xecc0, size 5, enabled > > pcib5: allocated I/O port range (0xecc0-0xecdf) for rid 18 of pci0:5:0:0 > > pcib5: matched entry for 5.0.INTA > > pcib5: slot 0 INTA hardwired to IRQ 48 > > found-> vendor=0x8086, dev=0x105e, revid=0x06 > > domain=0, bus=5, slot=0, func=1 > > class=02-00-00, hdrtype=0x00, mfdev=1 > > cmdreg=0x0007, statreg=0x0010, cachelnsz=16 (dwords) > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > > intpin=b, irq=6 > > powerspec 2 supports D0 D3 current D0 > > MSI supports 1 message, 64 bit > > map[10]: type Memory, range 32, base 0xdafc0000, size 17, > > enabled > > pcib5: allocated memory range (0xdafc0000-0xdafdffff) for rid 10 of > > pci0:5:0:1 > > map[14]: type Memory, range 32, base 0xdafe0000, size 17, > > enabled > > pcib5: allocated memory range (0xdafe0000-0xdaffffff) for rid 14 of > > pci0:5:0:1 > > map[18]: type I/O Port, range 32, base 0xece0, size 5, enabled > > pcib5: allocated I/O port range (0xece0-0xecff) for rid 18 of pci0:5:0:1 > > pcib5: matched entry for 5.0.INTB > > pcib5: slot 0 INTB hardwired to IRQ 52 > > em0: <Intel(R) PRO/1000 Network Connection 7.3.2> port 0xecc0-0xecdf mem > > 0xdaf80000-0xdaf9ffff,0xdafa0000-0xdafbffff irq 48 at device 0.0 on pci5 > > em0: attempting to allocate 1 MSI vectors (1 supported) > > msi: routing MSI IRQ 269 to local APIC 0 vector 64 > > em0: using IRQ 269 for MSI > > em0: Using an MSI interrupt > > em0: bpf attached > > em0: Ethernet address: 00:15:17:78:89:dc > > em1: <Intel(R) PRO/1000 Network Connection 7.3.2> port 0xece0-0xecff mem > > 0xdafc0000-0xdafdffff,0xdafe0000-0xdaffffff irq 52 at device 0.1 on pci5 > > em1: attempting to allocate 1 MSI vectors (1 supported) > > msi: routing MSI IRQ 270 to local APIC 0 vector 65 > > em1: using IRQ 270 for MSI > > em1: Using an MSI interrupt > > em1: bpf attached > > em1: Ethernet address: 00:15:17:78:89:dd > > pci0: <base peripheral> at device 4.0 (no driver attached) > > pci0: <base peripheral> at device 4.1 (no driver attached) > > pci0: <base peripheral> at device 4.2 (no driver attached) > > pci0: <base peripheral> at device 4.3 (no driver attached) > > pci0: <base peripheral> at device 4.4 (no driver attached) > > pci0: <base peripheral> at device 4.5 (no driver attached) > > pci0: <base peripheral> at device 4.6 (no driver attached) > > pci0: <base peripheral> at device 4.7 (no driver attached) > > pci0: <base peripheral> at device 5.0 (no driver attached) > > pci0: <base peripheral> at device 5.2 (no driver attached) > > pcib6: <PCI-PCI bridge> irq 16 at device 17.0 on pci0 > > pcib6: domain 0 > > pcib6: secondary bus 6 > > pcib6: subordinate bus 6 > > pcib6: no prefetched decode > > pci6: <PCI bus> on pcib6 > > pci6: domain=0, physical bus=6 > > pci0: <simple comms> at device 22.0 (no driver attached) > > pci0: <simple comms> at device 22.1 (no driver attached) > > ehci0: <EHCI (generic) USB 2.0 controller> mem 0xdc8fd000-0xdc8fd3ff irq > > 23 at device 26.0 on pci0 > > ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 66 > > usbus0: EHCI version 1.0 > > usbus0 on ehci0 > > pcib7: <ACPI PCI-PCI bridge> at device 28.0 on pci0 > > pcib7: domain 0 > > pcib7: secondary bus 7 > > pcib7: subordinate bus 7 > > pcib7: no prefetched decode > > device_attach: pcib7 attach returned 6 > > It seems that the following panic is caused by the above attach failure. > Namely, it seems that there are rman members in pcib_softc via pcib_window fields. > Those members are passed to rman_init during attach attempt, but they are not > passed to rman_fini if the attach fails (no attach failure cleanup). > After newbus code deallocates pcib_softc the global rman list (rman_head) becomes > corrupted. > > > pcib7: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0 > > panic: Bad tailq NEXT(0xffffffff80e52660->tqh_last) != NULL > > cpuid = 0 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > > kdb_backtrace() at kdb_backtrace+0x37 > > panic() at panic+0x1d8 > > rman_init() at rman_init+0x17c > > pcib_alloc_window() at pcib_alloc_window+0x9f > > pcib_attach_common() at pcib_attach_common+0x457 > > acpi_pcib_pci_attach() at acpi_pcib_pci_attach+0x1c > > device_attach() at device_attach+0x72 > > bus_generic_attach() at bus_generic_attach+0x1a > > acpi_pci_attach() at acpi_pci_attach+0x164 > > device_attach() at device_attach+0x72 > > bus_generic_attach() at bus_generic_attach+0x1a > > acpi_pcib_attach() at acpi_pcib_attach+0x1a7 > > acpi_pcib_acpi_attach() at acpi_pcib_acpi_attach+0x1f6 > > device_attach() at device_attach+0x72 > > bus_generic_attach() at bus_generic_attach+0x1a > > acpi_attach() at acpi_attach+0xbc1 > > device_attach() at device_attach+0x72 > > bus_generic_attach() at bus_generic_attach+0x1a > > nexus_acpi_attach() at nexus_acpi_attach+0x69 > > device_attach() at device_attach+0x72 > > bus_generic_new_pass() at bus_generic_new_pass+0xd6 > > bus_set_pass() at bus_set_pass+0x7a > > configure() at configure+0xa > > mi_startup() at mi_startup+0x77 > > btext() at btext+0x2c > > Uptime: 1s > > Automatic reboot in 15 seconds - press a key on the console to abort > > --> Press a key on the console to reboot, > > --> or switch off the system now. > > > > -- > Andriy Gapon > good lord, this took me all day to unwind what had happened. Dell has "updated" their BIOS between when this box was working and when I last looked at it. http://people.freebsd.org/~sbruno/R620-010206BIOS.txt Between 1.1.3 and 1.2.6 there was *a lot* of changes. I'll try and do some more investigations in the near future. 1.2.6 does *not* work and yeilds this kpanic. 1.1.3 boots up just fine. Sean
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1342831431.2645.1.camel>