Date: Tue, 26 Sep 2006 06:37:06 GMT From: Igor Soumenkov <2igosha@gmail.com> To: freebsd-gnats-submit@FreeBSD.org Subject: i386/103664: kmem_map_too_small panic after about 7d uptime on 6.1-release Message-ID: <200609260637.k8Q6b6EQ064066@www.freebsd.org> Resent-Message-ID: <200609260640.k8Q6eIjW010449@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 103664
>Category: i386
>Synopsis: kmem_map_too_small panic after about 7d uptime on 6.1-release
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-i386
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Sep 26 06:40:17 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator: Igor Soumenkov
>Release: 6.1-RELEASE
>Organization:
Kaspersky Lab
>Environment:
FreeBSD mss.avp.ru 6.1-RELEASE FreeBSD 6.1-RELEASE #2: Tue Sep 19 10:17:23 MSD 2006 igosha@mss.avp.ru:/usr/obj/usr/src/sys/SMP i386
>Description:
I am running 6.1-RELEASE on a two-xeon machine with 2GB ram, acpi and HT disabled. The system is running the SMP kernel and each week it panics with "kmem_map too small" error.
Tried to increase the kmem_max value, it is now:
options VM_KMEM_SIZE_MAX=524288000 #500 MB
Also, increased the mbuf clusters value:
options NMBCLUSTERS=65536
But still got the panic after 7 days of uptime.
The machine is running mysql, apache, anti-spam, anti-virus and several applications that are forked from inetd. The load average is about 1.5 - 2.
There are two hdds mirrored using GEOM mirror.
I have read all those 'you should increase kmem_size_max' answers but when should I stop increasing it? How much memory does the system need?
It it possible that some system component leaks memory?
BTW the system does not reboot even while having "options PANIC_REBOOT_WAIT_TIME=16" in kernel.
--- the dmesg output
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 6.1-RELEASE #2: Tue Sep 19 10:17:23 MSD 2006
igosha@mss.avp.ru:/usr/obj/usr/src/sys/SMP
MPTable: <INTEL Lindenhurst >
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.13-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf41 Stepping = 1
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x641d<SSE3,RSVD2,MON,DS_CPL,CNTX-ID,CX16,<b14>>
AMD Features=0x20000000<LM>
Logical CPUs per core: 2
real memory = 2146893824 (2047 MB)
avail memory = 2095501312 (1998 MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 6
ioapic0: Assuming intbase of 0
ioapic1: Assuming intbase of 24
ioapic2: Assuming intbase of 48
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
kbd1 at kbdmux0
cpu0 on motherboard
cpu1 on motherboard
pcib0: <MPTable Host-PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
pci0: <unknown> at device 0.1 (no driver attached)
pci0: <base peripheral> at device 1.0 (no driver attached)
pcib1: <PCI-PCI bridge> irq 16 at device 2.0 on pci0
pci1: <PCI bus> on pcib1
pcib2: <MPTable PCI-PCI bridge> at device 0.0 on pci1
pci2: <PCI bus> on pcib2
em0: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0x2000-0x203f mem 0xdd200000-0xdd21ffff irq 30 at device 3.0 on pci2
em0: Ethernet address: 00:30:48:2d:f4:38
em1: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0x2040-0x207f mem 0xdd220000-0xdd23ffff irq 31 at device 3.1 on pci2
em1: Ethernet address: 00:30:48:2d:f4:39
pci1: <base peripheral, interrupt controller> at device 0.1 (no driver attached)
pcib3: <MPTable PCI-PCI bridge> at device 0.2 on pci1
pci3: <PCI bus> on pcib3
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0x3400-0x34ff,0x3000-0x30ff mem 0xdd300000-0xdd301fff irq 56 at device 4.0 on pci3
ahd0: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0x3c00-0x3cff,0x3800-0x38ff mem 0xdd302000-0xdd303fff irq 57 at device 4.1 on pci3
ahd1: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
pci1: <base peripheral, interrupt controller> at device 0.3 (no driver attached)
pcib4: <PCI-PCI bridge> irq 16 at device 4.0 on pci0
pci4: <PCI bus> on pcib4
pcib5: <PCI-PCI bridge> irq 16 at device 6.0 on pci0
pci5: <PCI bus> on pcib5
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0x1400-0x141f irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0x1420-0x143f irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powereduhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0x1440-0x145f irq 18 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0x1460-0x147f irq 16 at device 29.3 on pci0
uhci3: [GIANT-LOCKED]
usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
usb3: USB revision 1.0
uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub3: 2 ports with 2 removable, self powered
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem 0xdd001000-0xdd0013ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <Intel 82801EB/R (ICH5) USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub4: 8 ports with 8 removable, self powered
pcib6: <MPTable PCI-PCI bridge> at device 30.0 on pci0
pci6: <PCI bus> on pcib6
pci6: <display, VGA> at device 1.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 SATA150 controller> port 0x14c0-0x14c7,0x14b4-0x14b7,0x14b8-0x14bf,0x14b0-0x14b3,0x14a0-0x14af irq 18 at device 31.2 on pci0
ata2: <ATA channel 0> on atapci0
ata3: <ATA channel 1> on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1 at port 0x170-0x177,0x376 irq 15 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <PNP0c02> can't assign resources (memory)
unknown: <PNP0303> can't assign resources (port)
unknown: <PNP0f13> can't assign resources (irq)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0700> can't assign resources (port)
Timecounters tick every 1.000 msec
ad4: 238475MB <Seagate ST3250823AS 3.03> at ata2-master SATA150
ad6: 238475MB <Seagate ST3250823AS 3.06> at ata3-master SATA150
Waiting 5 seconds for SCSI devices to settle
GEOM_MIRROR: Device gm0 created (id=3296712808).
GEOM_MIRROR: Device gm0: provider ad4 detected.
GEOM_MIRROR: Device gm0: provider ad6 detected.
GEOM_MIRROR: Device gm0: provider ad6 activated.
GEOM_MIRROR: Device gm0: provider ad4 activated.
GEOM_MIRROR: Device gm0: provider mirror/gm0 launched.
SMP: AP CPU #1 Launched!
>How-To-Repeat:
Set up GEOM mirror on a two-xeon machine with 2 HDDs and 2 Gb RAM, install mysql, Kaspersky Anti-Virus for mailservers and Kaspersky Anti-Spam 3.0 and put them all under average load.
>Fix:
Not known yet.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200609260637.k8Q6b6EQ064066>