Date: Thu, 16 Jun 2005 19:15:14 +0400 From: dima <_pppp@mail.ru> To: dave baukus <dbaukus@chiaro.com> Cc: freebsd-net@freebsd.org Subject: Re[2]: (panic) Lots of network memory needed Message-ID: <E1Diw5K-00004d-00._pppp-mail-ru@f13.mail.ru> In-Reply-To: <42B07A31.70201@chiaro.com>
next in thread | previous in thread | raw e-mail | index | archive | help
>
> I'm fairly sure that you are out of kernel virtual memory.
> Look at kern/kern_malloc.c kmeminit (); you can play w/ VM_KMEM_SIZE_MAX
> or TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
>
> /*
> * Try to auto-tune the kernel memory size, so that it is
> * more applicable for a wider range of machine sizes.
> * On an X86, a VM_KMEM_SIZE_SCALE value of 4 is good, while
> * a VM_KMEM_SIZE of 12MB is a fair compromise. The
> * VM_KMEM_SIZE_MAX is dependent on the maximum KVA space
> * available, and on an X86 with a total KVA space of 256MB,
> * try to keep VM_KMEM_SIZE_MAX at 80MB or below.
> *
> * Note that the kmem_map is also used by the zone allocator,
> * so make sure that there is enough space.
> */
> vm_kmem_size = VM_KMEM_SIZE;
> mem_size = cnt.v_page_count * PAGE_SIZE;
Which version do you use? I see the following in my /sys/kern/kern_malloc.c:
vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
....
/*
* Limit kmem virtual size to twice the physical memory.
* This allows for kmem map sparseness, but limits the size
* to something sane. Be careful to not overflow the 32bit
* ints while doing the check.
*/
if (((vm_kmem_size / 2) / PAGE_SIZE) > cnt.v_page_count)
vm_kmem_size = 2 * cnt.v_page_count * PAGE_SIZE;
It's 5.4-p2
>
> #if defined(VM_KMEM_SIZE_SCALE)
> if ((mem_size / VM_KMEM_SIZE_SCALE) > vm_kmem_size)
> vm_kmem_size = mem_size / VM_KMEM_SIZE_SCALE;
> #endif
>
> #if defined(VM_KMEM_SIZE_MAX)
> if (vm_kmem_size >= VM_KMEM_SIZE_MAX)
> vm_kmem_size = VM_KMEM_SIZE_MAX;
> #endif
>
> /* Allow final override from the kernel environment */
> TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
>
>
> Karim Fodil-Lemelin wrote:
> > Thanks but the system still crashes (FreeBSD 4.9) with 131072. Here is a
> > backtrace showing just that:
> >
> > Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> > 321 }
> > (kgdb) bt
> > #0 Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> > #1 0xc016a230 in panic (
> > fmt=0xc02ea380 "pmap_enter: invalid page directory pdir=%#llx,
> > va=%#x\n")
> > at ../../kern/kern_shutdown.c:593
> > #2 0xc0283594 in pmap_enter (pmap=0xc0340460, va=4292141056, m=0xc24b2848,
> > prot=7 '\a', wired=1) at ../../i386/i386/pmap.c:1943
> > #3 0xc023ddd0 in vm_fault (map=0xc033322c, vaddr=4292141056,
> > fault_type=7 '\a', fault_flags=1) at ../../vm/vm_fault.c:841
> > #4 0xc023df0a in vm_fault_wire (map=0xc033322c, start=4292141056,
> > end=4292149248) at ../../vm/vm_fault.c:915
> > #5 0xc0240831 in vm_map_pageable (map=0xc033322c, start=4292141056,
> > real_end=4292149248, new_pageable=0) at ../../vm/vm_map.c:1817
> > #6 0xc023ea25 in kmem_alloc (map=0xc033322c, size=8192)
> > at ../../vm/vm_kern.c:213
> > #7 0xc024a46f in _zget (z=0xdb5c6e80) at ../../vm/vm_zone.c:425
> > #8 0xc024a269 in zalloc (z=0xdb5c6e80) at ../../vm/vm_zone.c:60
> > #9 0xc0196106 in namei (ndp=0xfa489ef4) at ../../kern/vfs_lookup.c:104
> > #10 0xc01614ca in execve (p=0xfa482e00, uap=0xfa489f90)
> > at ../../kern/kern_exec.c:165
> > #11 0xc01590a1 in start_init (dummy=0x0) at ../../kern/init_main.c:543
> > (kgdb) p nmbclusters
> > $1 = 131072
> > (kgdb)
> >
> > Goran Spirovski - MorEl On.net wrote:
> >
> >> AFAIK the number of mbufs (and consequently nmbclusters) has to be a
> >> power
> >> of 2, so you should set it to 131072
> >>
> >> MorEl
> >>
> >> ----- Original Message ----- From: "Karim Fodil-Lemelin" <kfl@xiphos.ca>
> >> To: <freebsd-net@freebsd.org>
> >> Sent: Wednesday, June 15, 2005 6:08 PM
> >> Subject: (panic) Lots of network memory needed
> >>
> >>
> >>
> >>
> >>> Hello,
> >>>
> >>> From kernel tuning page
> >>>
> >>
> >> (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kern
> >>
> >> el-limits.html)
> >>
> >>
> >>> Some sentence about nmbclusters says:
> >>>
> >>> "Under no circumstances should you specify an arbitrarily high value for
> >>>
> >>
> >> this parameter as it could lead to a boot time crash."
> >>
> >>
> >>> Now I want to push the limits where I need 4KB buffer for each of the
> >>>
> >>
> >> 32000 connections I want this server to handle. If I do the math:
> >>
> >>
> >>> (32000 (conns) * 4 (KB/buffer) * 2 (buffer/conn)) / 2048 (KB/cluster)
> >>>
> >>> = 128000 clusters
> >>>
> >>> So I set this arbitrary high value in loader.conf under
> >>>
> >>
> >> (kern.ipc.nmbclusters) and no surprises I get panic: pmap_enter
> >> invalid page
> >> directory pdir=0x3cb063, va=0xfff800
> >>
> >>
> >>> (va has a weird address here)
> >>>
> >>> I know I am pushing the limits here but I have plenty of memory (2GB) on
> >>>
> >>
> >> this system (after all its just 250MB for network memory ;) and this is
> >> mainly just experimentation.
> >>
> >>
> >>> I would like some pointers toward fixing this. Is there another variable
> >>>
> >>
> >> tied into this (I guess so)? Could anybody points me to a technical
> >> document
> >> that would explain the relationship with that (those) other(s) presumed
> >> variable(s)?
> >>
> >>
> >>> Thank you,
> >>>
> >>>
> >>> Karim
> >>>
> >>>
> >>> _______________________________________________
> >>> freebsd-net@freebsd.org mailing list
> >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
> --
> Dave Baukus
> dbaukus@chiaro.com
> Chiaro Networks Ltd.
> Richardson, Texas
> USA
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1Diw5K-00004d-00._pppp-mail-ru>