Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 25 Nov 2011 00:08:49 -0500
From:      Glen Barber <gjb@FreeBSD.org>
To:        current@FreeBSD.org
Subject:   [ndis] "panic: IRQL_NOT_GREATER_THAN" - triggered by heavy wlan0 activity
Message-ID:  <20111125050849.GC37731@glenbarber.us>

next in thread | raw e-mail | index | archive | help

--PHCdUe6m4AxPMzOu
Content-Type: multipart/mixed; boundary="vEao7xgI/oilGqZ+"
Content-Disposition: inline


--vEao7xgI/oilGqZ+
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi,

I'm running 9-STABLE on a MacBook Pro (late 2009, I think):

schism % uname -a
FreeBSD schism 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #7 r227746:
  Thu Nov 24 13:05:31 EST 2011 gjb@kaos:/usr/obj/usr/src/sys/MACBOOK
  amd64

The integrated wireless chipset is a Broadcom bcm4322, which is
currently not supported by bwi(4) or bwm(4), however ndisgen(8) with
Windows XP 64-bit drivers gets the card in an operational state.

There are just a few oddities:

1.)  If I load the driver (bcmwl564_sys) with loader.conf, the machine
crashes partway through booting, sometime before invoking rc(8).  I
can't tell where specifically this happens, because the machine
completely resets whent his happens.

2.)  If the machine boots without the driver automatically loaded
between sometime rc(8) invocation and a login prompt, the ndis(4) wlan0
interface never gets assigned an IP (automatically, or by dhclient(8)
manually).  Calling kldload(8) from rc.conf.local seems to mostly solve
this, but I have seen occasional panics if the wlan0 device does not
obtain a DHCP lease within a, as of now, indeterminite amount of time.

3.)  This is the one I can actually give a whole bunch of information.
Under heavy network load, the machine panics.  vmcore, KERNCONF, and
dmesg are all attached.  I've observed the heavy load panics on a few
occasions, but finally stumbled on a way to trigger it (somewhat
reproducibly, but overall irrelevant, I guess) today.

The panics in this case are from sys/compat/ndis/subr_hal.c,
specifically:

% 410 void
% 411 KfLowerIrql(uint8_t oldirql)
% 412 {
% 413         if (oldirql =3D=3D DISPATCH_LEVEL)
% 414                 return;
% 415=20
% 416         if (KeGetCurrentIrql() !=3D DISPATCH_LEVEL)
% 417                 panic("IRQL_NOT_GREATER_THAN");
% 418=20
% 419         mtx_unlock(&disp_lock[curthread->td_oncpu]);
% 420         sched_unpin();
% 421 }

An (admittedly quick) Google search didn't turn up anything fruitful
with that particular panic reason.  'svn annotate' on subr_hal.c shows
the following from 2004:

% kaos % svn log -r128295
% ------------------------------------------------------------------------
% r128295 | wpaul | 2004-04-15 20:04:28 -0400 (Thu, 15 Apr 2004) | 13 lines
%=20
% - Use memory barrier with atomic operations in ntoskrnl_lock_dpc() and
%   ntoskrnl_unlocl_dpc().
% - hal_raise_irql(), hal_lower_irql() and hal_irql() didn't work right
%   on SMP (priority inheritance makes things... interesting). For now,
%   use only two states: DISPATCH_LEVEL (PI_REALTIME) and PASSIVE_LEVEL
%   (everything else). Tested on a dual PIII box.
%   [...]

So, at this point, I'm a bit stuck with where to go from here.  If I am
interpreting reason the panic() call was added in r128295 correctly,
this could potentially be an issue with SMP, as oldirql is not available
in the backtrace.  That is, of course, assuming it's not a problem with
the XP drivers for the wireless NIC.

Any thoughts on where to go from here would be appreciated.

Thanks.

Glen

--=20
Glen Barber | gjb@FreeBSD.org
FreeBSD Documentation Project


--vEao7xgI/oilGqZ+
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="dmesg.txt"
Content-Transfer-Encoding: quoted-printable

Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-PRERELEASE #7 r227746: Thu Nov 24 13:05:31 EST 2011
    gjb@kaos:/usr/obj/usr/src/sys/MACBOOK amd64
CPU: Intel(R) Core(TM)2 Duo CPU     P8700  @ 2.53GHz (2520.72-MHz K8-class =
CPU)
  Origin =3D "GenuineIntel"  Id =3D 0x1067a  Family =3D 6  Model =3D 17  St=
epping =3D 10
  Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PG=
E,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=3D0x408e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,x=
TPR,PDCM,SSE4.1,XSAVE>
  AMD Features=3D0x20100800<SYSCALL,NX,LM>
  AMD Features2=3D0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  =3D 8589934592 (8192 MB)
avail memory =3D 7920873472 (7553 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <APPLE  Apple00>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0: Changing APIC ID to 1
ioapic0 <Version 1.1> irqs 0-23 on motherboard
Cuse4BSD v0.1.21 @ /dev/cuse
kbd0 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <APPLE Apple00> on motherboard
acpi_ec0: <Embedded Controller: GPE 0x3f, ECDT> port 0x62,0x66 on acpi0
acpi0: Power Button (fixed)
unknown: I/O range not supported
unknown: I/O range not supported
Timecounter "HPET" frequency 25000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_acad0: <AC Adapter> on acpi0
acpi_lid0: <Control Method Lid Switch> on acpi0
acpi_button0: <Power Button> on acpi0
acpi_button1: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.1 (no driver attached)
isab0: <PCI-ISA bridge> port 0x2000-0x20ff at device 3.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory, RAM> at device 3.1 (no driver attached)
pci0: <serial bus, SMBus> at device 3.2 (no driver attached)
pci0: <memory, RAM> at device 3.3 (no driver attached)
pci0: <memory, RAM> at device 3.4 (no driver attached)
pci0: <processor> at device 3.5 (no driver attached)
ohci0: <nVidia nForce MCP79 USB Controller> mem 0xd3588000-0xd3588fff irq 1=
8 at device 4.0 on pci0
usbus0: <nVidia nForce MCP79 USB Controller> on ohci0
ehci0: <NVIDIA nForce MCP79 USB 2.0 controller> mem 0xd3589200-0xd35892ff i=
rq 19 at device 4.1 on pci0
usbus1: EHCI version 1.0
usbus1: <NVIDIA nForce MCP79 USB 2.0 controller> on ehci0
ohci1: <nVidia nForce MCP79 USB Controller> mem 0xd3587000-0xd3587fff irq 2=
0 at device 6.0 on pci0
usbus2: <nVidia nForce MCP79 USB Controller> on ohci1
ehci1: <NVIDIA nForce MCP79 USB 2.0 controller> mem 0xd3589100-0xd35891ff i=
rq 21 at device 6.1 on pci0
usbus3: EHCI version 1.0
usbus3: <NVIDIA nForce MCP79 USB 2.0 controller> on ehci1
hdac0: <NVidia MCP79 High Definition Audio Controller> mem 0xd3580000-0xd35=
83fff irq 22 at device 8.0 on pci0
pcib1: <PCI-PCI bridge> at device 9.0 on pci0
pci1: <PCI bus> on pcib1
nfe0: <NVIDIA nForce MCP79 Networking Adapter> port 0x21e0-0x21e7 mem 0xd35=
86000-0xd3586fff,0xd3589000-0xd35890ff,0xd3589300-0xd358930f irq 23 at devi=
ce 10.0 on pci0
miibus0: <MII bus> on nfe0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseT=
X-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000=
baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto=
-flow
nfe0: Ethernet address: 34:15:9e:1f:41:04
ahci0: <NVIDIA MCP79 AHCI SATA controller> port 0x21d8-0x21df,0x21ec-0x21ef=
,0x21d0-0x21d7,0x21e8-0x21eb,0x21c0-0x21cf mem 0xd3584000-0xd3585fff irq 16=
 at device 11.0 on pci0
ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
pcib2: <ACPI PCI-PCI bridge> at device 12.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 16.0 on pci0
pci3: <ACPI PCI bus> on pcib3
vgapci0: <VGA-compatible display> port 0x1000-0x107f mem 0xd2000000-0xd2fff=
fff,0xc0000000-0xcfffffff,0xd0000000-0xd1ffffff irq 17 at device 0.0 on pci3
nvidia0: <GeForce 9400M> on vgapci0
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
pcib4: <ACPI PCI-PCI bridge> at device 21.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pci4: <network> at device 0.0 (no driver attached)
pcib5: <ACPI PCI-PCI bridge> at device 22.0 on pci0
pci5: <ACPI PCI bus> on pcib5
fwohci0: <1394 Open Host Controller Interface> mem 0xd3100000-0xd3100fff ir=
q 19 at device 0.0 on pci5
fwohci0: OHCI version 1.10 (ROM=3D0)
fwohci0: No. of Isochronous channels is 8.
fwohci0: EUI64 34:15:9e:ff:fe:1f:41:04
fwohci0: invalid speed 7 (fixed to 3).
fwohci0: Phy 1394a available S800, 3 ports.
fwohci0: Link S800, max_rec 4096 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 36:15:9e:1f:41:04
fwe0: Ethernet address: 36:15:9e:1f:41:04
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 34:15:9e:ff:fe:1f:41:04 @ 0xfffe00000000, S800, ma=
xrec 4096
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0xae718000
fwohci0: Initiate bus reset
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=3D0x00000000, SelfID Count=3D2, CYCLEMAS=
TER mode
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
battery0: <ACPI Control Method Battery> on acpi0
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
EcGpeHandler: queuing GPE query handler failed
atrtc0: <AT realtime clock> port 0x70-0x77 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=3D0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
AcpiOsExecute: failed to enqueue task, consider increasing the debug.acpi.m=
ax_tasks tunable
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
firewire0: 1 nodes, maxhop <=3D 0 cable IRM irm(0)  (me)=20
firewire0: bus manager 0=20
hdac0: HDA Codec #0: Cirrus Logic CS4206
pcm0: <HDA Cirrus Logic CS4206 PCM #0 Analog> at cad 0 nid 1 on hdac0
pcm1: <HDA Cirrus Logic CS4206 PCM #1 Analog> at cad 0 nid 1 on hdac0
pcm2: <HDA Cirrus Logic CS4206 PCM #2 Digital> at cad 0 nid 1 on hdac0
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen0.1: <nVidia> at usbus0
uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <nVidia> at usbus1
uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen2.1: <nVidia> at usbus2
uhub2: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <nVidia> at usbus3
uhub3: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST9500420AS 0001BSM2> ATA-8 SATA 1.x device
ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
SMP: AP CPU #1 Launched!
cd0 at ahcich1 bus 0 scbus1 target 0 lun 0
cd0: <HL-DT-ST DVDRW  GS23N SB03> Removable CD-ROM SCSI-0 device=20
cd0: 150.000MB/s transfers (SATA 1.x, UDMA6, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
GEOM: ada0: enabling Boot Camp
uhub2: 5 ports with 5 removable, self powered
uhub0: 7 ports with 7 removable, self powered
Root mount waiting for: usbus3 usbus1
Root mount waiting for: usbus3 usbus1
uhub3: 5 ports with 5 removable, self powered
Root mount waiting for: usbus3 usbus1
uhub1: 7 ports with 7 removable, self powered
Root mount waiting for: usbus3 usbus1
ugen2.2: <Apple Inc.> at usbus2
uhub4: <Apple Inc. BRCM2046 Hub, class 9/0, rev 2.00/1.00, addr 2> on usbus2
ugen1.2: <Apple Inc.> at usbus1
uhub4: 3 ports with 0 removable, self powered
Root mount waiting for: usbus3 usbus1
ugen2.3: <Apple Inc.> at usbus2
ugen3.2: <Apple> at usbus3
umass0: <Apple Card Reader, class 0/0, rev 2.00/98.33, addr 2> on usbus3
umass0:  SCSI over Bulk-Only; quirks =3D 0x4100
umass0:2:0:-1: Attached to scbus2
Trying to mount root from ufs:/dev/ada0p4 [rw]...
WARNING: / was not properly dismounted
ugen0.2: <Apple Computer, Inc.> at usbus0
uhid0: <Apple Computer, Inc. IR Receiver, class 0/0, rev 2.00/0.16, addr 2>=
 on usbus0
(probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0=20
(probe0:umass-sim0:0:0:0): CAM status: SCSI Status Error
(probe0:umass-sim0:0:0:0): SCSI status: Check Condition
(probe0:umass-sim0:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not prese=
nt)
da0 at umass-sim0 bus 0 scbus2 target 0 lun 0
da0: <APPLE SD Card Reader 1.00> Removable Direct Access SCSI-0 device=20
da0: 40.000MB/s transfers
da0: Attempt to query device size failed: NOT READY, Medium not present
ugen2.4: <vendor 0x04d9> at usbus2
ums0: <vendor 0x04d9 product 0x048e, class 0/0, rev 1.10/7.00, addr 4> on u=
sbus2
ums0: 3 buttons and [XYZ] coordinates ID=3D0
ugen0.3: <Apple Inc.> at usbus0
ukbd0: <Apple Internal Keyboard> on usbus0
kbd1 at ukbd0
uhid1: <Touchpad> on usbus0
ums1: <Touchpad> on usbus0
ums1: 3 buttons and [XY] coordinates ID=3D2
ndis0: <Broadcom 802.11n Network Adapter> mem 0xd3200000-0xd3203fff irq 18 =
at device 0.0 on pci4
ndis0: NDIS API version: 5.1
wlan0: Ethernet address: f8:1e:df:e5:cc:5f
ubt0: <Apple Inc. Bluetooth USB Host Controller, class 224/1, rev 2.00/1.80=
, addr 3> on usbus2
WARNING: attempt to domain_add(bluetooth) after domainfinalize()
WARNING: attempt to domain_add(netgraph) after domainfinalize()
ndis0: link state changed to UP

--vEao7xgI/oilGqZ+
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="kernconf.txt"

Script started on Thu Nov 24 22:35:22 2011

schism % egrep "^[A-Za-z]" /usr/src/sys/amd64/conf/MACBOOK
cpu             HAMMER
ident           MACBOOK
makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols
options         SCHED_ULE               # ULE scheduler
options         PREEMPTION              # Enable kernel thread preemption
options         INET                    # InterNETworking
options         INET6                   # IPv6 communications protocols
options         SCTP                    # Stream Control Transmission Protocol
options         FFS                     # Berkeley Fast Filesystem
options         SOFTUPDATES             # Enable FFS soft updates support
options         UFS_ACL                 # Support for access control lists
options         UFS_DIRHASH             # Improve performance on big directories
options         UFS_GJOURNAL            # Enable gjournal-based UFS journaling
options         MD_ROOT                 # MD is a potential root device
options         NFSCL                   # New Network Filesystem Client
options         NFSD                    # New Network Filesystem Server
options         NFSLOCKD                # Network Lock Manager
options         NFS_ROOT                # NFS usable as /, requires NFSCL
options         MSDOSFS                 # MSDOS Filesystem
options         CD9660                  # ISO 9660 Filesystem
options         PROCFS                  # Process filesystem (requires PSEUDOFS)
options         PSEUDOFS                # Pseudo-filesystem framework
options         GEOM_PART_GPT           # GUID Partition Tables.
options         GEOM_LABEL              # Provides labelization
options         COMPAT_FREEBSD32        # Compatible with i386 binaries
options         COMPAT_FREEBSD4         # Compatible with FreeBSD4
options         COMPAT_FREEBSD5         # Compatible with FreeBSD5
options         COMPAT_FREEBSD6         # Compatible with FreeBSD6
options         COMPAT_FREEBSD7         # Compatible with FreeBSD7
options         SCSI_DELAY=5000         # Delay (in ms) before probing SCSI
options         KTRACE                  # ktrace(1) support
options         STACK                   # stack(9) support
options         SYSVSHM                 # SYSV-style shared memory
options         SYSVMSG                 # SYSV-style message queues
options         SYSVSEM                 # SYSV-style semaphores
options         _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options         PRINTF_BUFR_SIZE=128    # Prevent printf output being interspersed.
options         KBD_INSTALL_CDEV        # install a CDEV entry in /dev
options         HWPMC_HOOKS             # Necessary kernel hooks for hwpmc(4)
options         AUDIT                   # Security event auditing
options         MAC                     # TrustedBSD MAC Framework
options         INCLUDE_CONFIG_FILE     # Include this file in kernel
options         KDB                     # Kernel debugger related code
options         KDB_TRACE               # Print a stack trace for a panic
options         IPSEC                   # ipsec(4)
device          crypto                  # crypto(4) device for ipsec(4)
options         SMP                     # Symmetric MultiProcessor Kernel
device          cpufreq
device          acpi
device          pci
device          ahci            # AHCI-compatible SATA controllers
device          ata             # Legacy ATA/SATA controllers
options         ATA_CAM         # Handle legacy controllers with CAM
options         ATA_STATIC_ID   # Static device numbering
device          mvs             # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA
device          siis            # SiliconImage SiI3124/SiI3132/SiI3531 SATA
device          ahc             # AHA2940 and onboard AIC7xxx devices
options         AHC_REG_PRETTY_PRINT    # Print register bitfields in debug
device          ahd             # AHA39320/29320 and onboard AIC79xx devices
options         AHD_REG_PRETTY_PRINT    # Print register bitfields in debug
device          esp             # AMD Am53C974 (Tekram DC-390(T))
device          hptiop          # Highpoint RocketRaid 3xxx series
device          isp             # Qlogic family
device          mpt             # LSI-Logic MPT-Fusion
device          mps             # LSI-Logic MPT-Fusion 2
device          sym             # NCR/Symbios Logic (newer chipsets + those of `ncr')
device          trm             # Tekram DC395U/UW/F DC315U adapters
device          adv             # Advansys SCSI adapters
device          adw             # Advansys wide SCSI adapters
device          aic             # Adaptec 15[012]x SCSI adapters, AIC-6[23]60.
device          bt              # Buslogic/Mylex MultiMaster SCSI adapters
device          scbus           # SCSI bus (required for ATA/SCSI)
device          ch              # SCSI media changers
device          da              # Direct Access (disks)
device          sa              # Sequential Access (tape etc)
device          cd              # CD
device          pass            # Passthrough device (direct ATA/SCSI access)
device          ses             # SCSI Environmental Services (and SAF-TE)
device          amr             # AMI MegaRAID
device          arcmsr          # Areca SATA II RAID
device          ciss            # Compaq Smart RAID 5*
device          dpt             # DPT Smartcache III, IV - See NOTES for options
device          hptmv           # Highpoint RocketRAID 182x
device          hptrr           # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx
device          iir             # Intel Integrated RAID
device          ips             # IBM (Adaptec) ServeRAID
device          mly             # Mylex AcceleRAID/eXtremeRAID
device          twa             # 3ware 9000 series PATA/SATA RAID
device          aac             # Adaptec FSA RAID
device          aacp            # SCSI passthrough for aac (requires CAM)
device          ida             # Compaq Smart RAID
device          mfi             # LSI MegaRAID SAS
device          mlx             # Mylex DAC960 family
device          twe             # 3ware ATA RAID
device          tws             # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller
device          atkbdc          # AT keyboard controller
device          atkbd           # AT keyboard
device          psm             # PS/2 mouse
device          kbdmux          # keyboard multiplexer
device          vga             # VGA video card driver
device          splash          # Splash screen and screen saver support
device          sc
options         SC_PIXEL_MODE   # add support for the raster text mode
device          agp             # support several AGP chipsets
device          cbb             # cardbus (yenta) bridge
device          pccard          # PC Card (16-bit) bus
device          cardbus         # CardBus (32-bit) bus
device          uart            # Generic UART driver
device          ppc
device          ppbus           # Parallel port bus (required)
device          lpt             # Printer
device          plip            # TCP/IP over parallel
device          ppi             # Parallel port interface device
device          puc             # Multi I/O cards and multi-channel UARTs
device          bxe             # Broadcom BCM57710/BCM57711/BCM57711E 10Gb Ethernet
device          de              # DEC/Intel DC21x4x (``Tulip'')
device          em              # Intel PRO/1000 Gigabit Ethernet Family
device          igb             # Intel PRO/1000 PCIE Server Gigabit Family
device          ixgbe           # Intel PRO/10GbE PCIE Ethernet Family
device          le              # AMD Am7900 LANCE and Am79C9xx PCnet
device          ti              # Alteon Networks Tigon I/II gigabit Ethernet
device          txp             # 3Com 3cR990 (``Typhoon'')
device          vx              # 3Com 3c590, 3c595 (``Vortex'')
device          miibus          # MII bus support
device          ae              # Attansic/Atheros L2 FastEthernet
device          age             # Attansic/Atheros L1 Gigabit Ethernet
device          alc             # Atheros AR8131/AR8132 Ethernet
device          ale             # Atheros AR8121/AR8113/AR8114 Ethernet
device          bce             # Broadcom BCM5706/BCM5708 Gigabit Ethernet
device          bfe             # Broadcom BCM440x 10/100 Ethernet
device          bge             # Broadcom BCM570xx Gigabit Ethernet
device          dc              # DEC/Intel 21143 and various workalikes
device          et              # Agere ET1310 10/100/Gigabit Ethernet
device          fxp             # Intel EtherExpress PRO/100B (82557, 82558)
device          jme             # JMicron JMC250 Gigabit/JMC260 Fast Ethernet
device          lge             # Level 1 LXT1001 gigabit Ethernet
device          msk             # Marvell/SysKonnect Yukon II Gigabit Ethernet
device          nfe             # nVidia nForce MCP on-board Ethernet
device          nge             # NatSemi DP83820 gigabit Ethernet
device          pcn             # AMD Am79C97x PCI 10/100 (precedence over 'le')
device          re              # RealTek 8139C+/8169/8169S/8110S
device          rl              # RealTek 8129/8139
device          sf              # Adaptec AIC-6915 (``Starfire'')
device          sge             # Silicon Integrated Systems SiS190/191
device          sis             # Silicon Integrated Systems SiS 900/SiS 7016
device          sk              # SysKonnect SK-984x & SK-982x gigabit Ethernet
device          ste             # Sundance ST201 (D-Link DFE-550TX)
device          stge            # Sundance/Tamarack TC9021 gigabit Ethernet
device          tl              # Texas Instruments ThunderLAN
device          tx              # SMC EtherPower II (83c170 ``EPIC'')
device          vge             # VIA VT612x gigabit Ethernet
device          vr              # VIA Rhine, Rhine II
device          wb              # Winbond W89C840F
device          xl              # 3Com 3c90x (``Boomerang'', ``Cyclone'')
device          cs              # Crystal Semiconductor CS89x0 NIC
device          ed              # NE[12]000, SMC Ultra, 3c503, DS8390 cards
device          ex              # Intel EtherExpress Pro/10 and Pro/10+
device          ep              # Etherlink III based cards
device          fe              # Fujitsu MB8696x based cards
device          sn              # SMC's 9000 series of Ethernet chips
device          xe              # Xircom pccard Ethernet
device          wlan            # 802.11 support
options         IEEE80211_DEBUG # enable debug msgs
options         IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's
options         IEEE80211_SUPPORT_MESH  # enable 802.11s draft support
device          wlan_wep        # 802.11 WEP support
device          wlan_ccmp       # 802.11 CCMP support
device          wlan_tkip       # 802.11 TKIP support
device          wlan_amrr       # AMRR transmit rate control algorithm
device          an              # Aironet 4500/4800 802.11 wireless NICs.
device          ath             # Atheros NIC's
device          ath_pci         # Atheros pci/cardbus glue
device          ath_hal         # pci/cardbus chip support
options         AH_SUPPORT_AR5416       # enable AR5416 tx/rx descriptors
device          ath_rate_sample # SampleRate tx rate control for ath
device          ipw             # Intel 2100 wireless NICs.
device          iwi             # Intel 2200BG/2225BG/2915ABG wireless NICs.
device          iwn             # Intel 4965/1000/5000/6000 wireless NICs.
device          malo            # Marvell Libertas wireless NICs.
device          mwl             # Marvell 88W8363 802.11n wireless NICs.
device          ral             # Ralink Technology RT2500 wireless NICs.
device          wi              # WaveLAN/Intersil/Symbol 802.11 wireless NICs.
device          wpi             # Intel 3945ABG wireless NICs.
device          loop            # Network loopback
device          random          # Entropy device
device          ether           # Ethernet support
device          vlan            # 802.1Q VLAN support
device          tun             # Packet tunnel.
device          pty             # BSD-style compatibility pseudo ttys
device          md              # Memory "disks"
device          gif             # IPv6 and IPv4 tunneling
device          faith           # IPv6-to-IPv4 relaying (translation)
device          firmware        # firmware assist module
device          bpf             # Berkeley packet filter
options         USB_DEBUG       # enable debug msgs
device          uhci            # UHCI PCI->USB interface
device          ohci            # OHCI PCI->USB interface
device          ehci            # EHCI PCI->USB interface (USB 2.0)
device          xhci            # XHCI PCI->USB interface (USB 3.0)
device          usb             # USB Bus (required)
device          uhid            # "Human Interface Devices"
device          ukbd            # Keyboard
device          ulpt            # Printer
device          umass           # Disks/Mass storage - Requires scbus and da
device          ums             # Mouse
device          urio            # Diamond Rio 500 MP3 player
device          u3g             # USB-based 3G modems (Option, Huawei, Sierra)
device          uark            # Technologies ARK3116 based serial adapters
device          ubsa            # Belkin F5U103 and compatible serial adapters
device          uftdi           # For FTDI usb serial adapters
device          uipaq           # Some WinCE based devices
device          uplcom          # Prolific PL-2303 serial adapters
device          uslcom          # SI Labs CP2101/CP2102 serial adapters
device          uvisor          # Visor and Palm devices
device          uvscom          # USB serial support for DDI pocket's PHS
device          aue             # ADMtek USB Ethernet
device          axe             # ASIX Electronics USB Ethernet
device          cdce            # Generic USB over Ethernet
device          cue             # CATC USB Ethernet
device          kue             # Kawasaki LSI USB Ethernet
device          rue             # RealTek RTL8150 USB Ethernet
device          udav            # Davicom DM9601E USB
device          rum             # Ralink Technology RT2501USB wireless NICs
device          run             # Ralink Technology RT2700/RT2800/RT3000 NICs.
device          uath            # Atheros AR5523 wireless NICs
device          upgt            # Conexant/Intersil PrismGT wireless NICs.
device          ural            # Ralink Technology RT2500USB wireless NICs
device          urtw            # Realtek RTL8187B/L wireless NICs
device          zyd             # ZyDAS zd1211/zd1211b wireless NICs
device          firewire        # FireWire bus code
device          fwe             # Ethernet over FireWire (non-standard!)
device          fwip            # IP over FireWire (RFC 2734,3146)
device          dcons           # Dumb console driver
device          dcons_crom      # Configuration ROM for dcons
device          sound           # Generic sound driver (required)
device          snd_es137x      # Ensoniq AudioPCI ES137x
device          snd_hda         # Intel High Definition Audio
device          snd_ich         # Intel, NVidia and other ICH AC'97 Audio
device          snd_uaudio      # USB Audio
device          snd_via8233     # VIA VT8233x Audio
schism % ^D

Script done on Thu Nov 24 22:35:27 2011

--vEao7xgI/oilGqZ+
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="vmcore.1.txt"
Content-Transfer-Encoding: quoted-printable

Script started on Thu Nov 24 18:40:59 2011

schism# kgdb -c /home/gjb/panics/vmcore.1 MACBOOK/kernel.debug
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain condition=
s.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
panic: IRQL_NOT_GREATER_THAN
cpuid =3D 1
KDB: stack backtrace:
#0 0xffffffff8087090e at kdb_backtrace+0x5e
#1 0xffffffff8083b577 at panic+0x187
#2 0xffffffff82a82fe2 at KfLowerIrql+0x62
#3 0xfffffe0008dc90ce at dmapbase+0x8dc90ce
#4 0xffffffff82638f45 at __stop_set_sysinit_set+0x25c45
#5 0xffffffff82a8c649 at x86_64_call2+0x9
#6 0xffffffff8080f0ef at fork_exit+0x11f
#7 0xffffffff80b3291e at fork_trampoline+0xe
Uptime: 2h34m38s
Dumping 990 out of 7875 MB:..2%..12%..21%..31%..41%..51%..62%..72%..81%..91%

Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/ker=
nel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /boot/=
kernel/coretemp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/coretemp.ko
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /boot/modules/cuse4bsd.ko...done.
Loaded symbols for /boot/modules/cuse4bsd.ko
Reading symbols from /boot/modules/bcmwl564_sys.ko...done.
Loaded symbols for /boot/modules/bcmwl564_sys.ko
Reading symbols from /boot/kernel/if_ndis.ko...Reading symbols from /boot/k=
ernel/if_ndis.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_ndis.ko
Reading symbols from /boot/kernel/ndis.ko...Reading symbols from /boot/kern=
el/ndis.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ndis.ko
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /boot/k=
ernel/fdescfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/ng_ubt.ko...Reading symbols from /boot/ke=
rnel/ng_ubt.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ubt.ko
Reading symbols from /boot/kernel/ng_hci.ko...Reading symbols from /boot/ke=
rnel/ng_hci.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_hci.ko
Reading symbols from /boot/kernel/ng_bluetooth.ko...Reading symbols from /b=
oot/kernel/ng_bluetooth.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_bluetooth.ko
Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/=
kernel/netgraph.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/kernel/ng_l2cap.ko...Reading symbols from /boot/=
kernel/ng_l2cap.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_l2cap.ko
Reading symbols from /boot/kernel/ng_btsocket.ko...Reading symbols from /bo=
ot/kernel/ng_btsocket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_btsocket.ko
Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot=
/kernel/ng_socket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_socket.ko
#0  doadump (textdump=3DVariable "textdump" is not available.
) at pcpu.h:224
224	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) bt
#0  doadump (textdump=3DVariable "textdump" is not available.
) at pcpu.h:224
#1  0xffffffff8083b0b5 in kern_reboot (howto=3D260)
    at /usr/src/sys/kern/kern_shutdown.c:442
#2  0xffffffff8083b561 in panic (fmt=3DVariable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:607
#3  0xffffffff82a82fe2 in KfLowerIrql (oldirql=3DVariable "oldirql" is not =
available.
)
    at /usr/src/sys/modules/ndis/../../compat/ndis/subr_hal.c:417
#4  0xfffffe0008dc90ce in ?? ()
#5  0xfffffe0008def280 in ?? ()
#6  0xfffffe000d239d20 in ?? ()
#7  0xfffffe0008def280 in ?? ()
#8  0xfffffe0008def270 in ?? ()
#9  0xffffff82204bfb80 in ?? ()
#10 0xfffffe0008dc804e in ?? ()
#11 0x0000000000000000 in ?? ()
#12 0x0000000000000000 in ?? ()
#13 0xfffffe00088f3460 in ?? ()
#14 0xfffffe0008d96910 in ?? ()
#15 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#16 0xfffffe000d239d00 in ?? ()
#17 0xffffff82204bfb80 in ?? ()
#18 0xffffffff82638f45 in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#19 0x0000000082a87970 in ?? ()
---Type <return> to continue, or q <return> to quit---
#20 0xfffffe000d239d20 in ?? ()
#21 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#22 0xfffffe0008d94520 in ?? ()
#23 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#24 0xffffffff82a8c649 in x86_64_call2 ()
    at /usr/src/sys/modules/ndis/../../compat/ndis/winx64_wrap.S:138
#25 0xfffffe0008def280 in ?? ()
#26 0xfffffe000d239d20 in ?? ()
#27 0xfffffe0008def280 in ?? ()
#28 0xfffffe0008def270 in ?? ()
#29 0xffffff82204bfb80 in ?? ()
#30 0xfffffe0008dc804e in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0xfffffe00088f3460 in ?? ()
#34 0xfffffe0008d96910 in ?? ()
#35 0xffffff82204bfc50 in ?? ()
#36 0xfffffe0008d94520 in ?? ()
#37 0xffffff82204bfc40 in ?? ()
#38 0xffffffff8080f0ef in fork_exit (callout=3D0xffffff82204bfb80,=20
    arg=3D0xfffffe0008d94550, frame=3D0xfffffe0008d94548)
    at /usr/src/sys/kern/kern_fork.c:995
Previous frame inner to this frame (corrupt stack?)
(kgdb) bt full
#0  doadump (textdump=3DVariable "textdump" is not available.
) at pcpu.h:224
No locals.
#1  0xffffffff8083b0b5 in kern_reboot (howto=3D260)
    at /usr/src/sys/kern/kern_shutdown.c:442
	_ep =3D Variable "_ep" is not available.
(kgdb) where
#0  doadump (textdump=3DVariable "textdump" is not available.
) at pcpu.h:224
#1  0xffffffff8083b0b5 in kern_reboot (howto=3D260)
    at /usr/src/sys/kern/kern_shutdown.c:442
#2  0xffffffff8083b561 in panic (fmt=3DVariable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:607
#3  0xffffffff82a82fe2 in KfLowerIrql (oldirql=3DVariable "oldirql" is not =
available.
)
    at /usr/src/sys/modules/ndis/../../compat/ndis/subr_hal.c:417
#4  0xfffffe0008dc90ce in ?? ()
#5  0xfffffe0008def280 in ?? ()
#6  0xfffffe000d239d20 in ?? ()
#7  0xfffffe0008def280 in ?? ()
#8  0xfffffe0008def270 in ?? ()
#9  0xffffff82204bfb80 in ?? ()
#10 0xfffffe0008dc804e in ?? ()
#11 0x0000000000000000 in ?? ()
#12 0x0000000000000000 in ?? ()
#13 0xfffffe00088f3460 in ?? ()
#14 0xfffffe0008d96910 in ?? ()
#15 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#16 0xfffffe000d239d00 in ?? ()
#17 0xffffff82204bfb80 in ?? ()
#18 0xffffffff82638f45 in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#19 0x0000000082a87970 in ?? ()
---Type <return> to continue, or q <return> to quit---
#20 0xfffffe000d239d20 in ?? ()
#21 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#22 0xfffffe0008d94520 in ?? ()
#23 0xffffffff82638efc in ndis_bcmwl564_sys_drv_data_start ()
   from /boot/modules/bcmwl564_sys.ko
#24 0xffffffff82a8c649 in x86_64_call2 ()
    at /usr/src/sys/modules/ndis/../../compat/ndis/winx64_wrap.S:138
#25 0xfffffe0008def280 in ?? ()
#26 0xfffffe000d239d20 in ?? ()
#27 0xfffffe0008def280 in ?? ()
#28 0xfffffe0008def270 in ?? ()
#29 0xffffff82204bfb80 in ?? ()
#30 0xfffffe0008dc804e in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0xfffffe00088f3460 in ?? ()
#34 0xfffffe0008d96910 in ?? ()
#35 0xffffff82204bfc50 in ?? ()
#36 0xfffffe0008d94520 in ?? ()
#37 0xffffff82204bfc40 in ?? ()
#38 0xffffffff8080f0ef in fork_exit (callout=3D0xffffff82204bfb80,=20
    arg=3D0xfffffe0008d94550, frame=3D0xfffffe0008d94548)
    at /usr/src/sys/kern/kern_fork.c:995
(kgdb) q
schism# ^D

Script done on Thu Nov 24 18:41:23 2011

--vEao7xgI/oilGqZ+--

--PHCdUe6m4AxPMzOu
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (FreeBSD)

iQEcBAEBCAAGBQJOzyLhAAoJEFJPDDeguUajBRgH/3YVOgMvNw2iufI808Bt/OIo
KP10HRTJ2NrG4rlXDNFS9NUjLMI6jG3QluNUeVazzMgST7M6r2qlaziSyGNXweQC
fEaPfu8BpRBwBVRN7Uqphfe3/sddeb9KywP4SUaIVusZQ2Fhtqw6KIBk6qr2EMKs
0scK7/NEqTTE2KBXr0Zz0ZKmIV7LQR2/XijSTEVH2QkYflFPpOJ7FJOYDR0KGkJZ
UgeOS9gZQKvKzobLStK5ol33E8faO294/9LFoseTy1Iq17K+14PijDlbu8jASmHQ
98RFkorp7AJU7fOgVzUxT1oqpwkxVNtcIE3g52EwYEy8YiEMct0Ovjz8aVGvBZw=
=Eebk
-----END PGP SIGNATURE-----

--PHCdUe6m4AxPMzOu--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111125050849.GC37731>