Date: Wed, 15 Jun 2005 13:57:53 -0500 From: dave baukus <dbaukus@chiaro.com> To: Karim Fodil-Lemelin <kfl@xiphos.ca> Cc: freebsd-net@freebsd.org Subject: Re: (panic) Lots of network memory needed Message-ID: <42B07A31.70201@chiaro.com> In-Reply-To: <42B06F3D.4050504@xiphos.ca> References: <42B0528D.9080308@xiphos.ca> <002301c571d0$ab6c60b0$680a0a0a@onnet> <42B06F3D.4050504@xiphos.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
I'm fairly sure that you are out of kernel virtual memory.
Look at kern/kern_malloc.c kmeminit (); you can play w/ VM_KMEM_SIZE_MAX
or TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
/*
* Try to auto-tune the kernel memory size, so that it is
* more applicable for a wider range of machine sizes.
* On an X86, a VM_KMEM_SIZE_SCALE value of 4 is good, while
* a VM_KMEM_SIZE of 12MB is a fair compromise. The
* VM_KMEM_SIZE_MAX is dependent on the maximum KVA space
* available, and on an X86 with a total KVA space of 256MB,
* try to keep VM_KMEM_SIZE_MAX at 80MB or below.
*
* Note that the kmem_map is also used by the zone allocator,
* so make sure that there is enough space.
*/
vm_kmem_size = VM_KMEM_SIZE;
mem_size = cnt.v_page_count * PAGE_SIZE;
#if defined(VM_KMEM_SIZE_SCALE)
if ((mem_size / VM_KMEM_SIZE_SCALE) > vm_kmem_size)
vm_kmem_size = mem_size / VM_KMEM_SIZE_SCALE;
#endif
#if defined(VM_KMEM_SIZE_MAX)
if (vm_kmem_size >= VM_KMEM_SIZE_MAX)
vm_kmem_size = VM_KMEM_SIZE_MAX;
#endif
/* Allow final override from the kernel environment */
TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
Karim Fodil-Lemelin wrote:
> Thanks but the system still crashes (FreeBSD 4.9) with 131072. Here is a
> backtrace showing just that:
>
> Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> 321 }
> (kgdb) bt
> #0 Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> #1 0xc016a230 in panic (
> fmt=0xc02ea380 "pmap_enter: invalid page directory pdir=%#llx,
> va=%#x\n")
> at ../../kern/kern_shutdown.c:593
> #2 0xc0283594 in pmap_enter (pmap=0xc0340460, va=4292141056, m=0xc24b2848,
> prot=7 '\a', wired=1) at ../../i386/i386/pmap.c:1943
> #3 0xc023ddd0 in vm_fault (map=0xc033322c, vaddr=4292141056,
> fault_type=7 '\a', fault_flags=1) at ../../vm/vm_fault.c:841
> #4 0xc023df0a in vm_fault_wire (map=0xc033322c, start=4292141056,
> end=4292149248) at ../../vm/vm_fault.c:915
> #5 0xc0240831 in vm_map_pageable (map=0xc033322c, start=4292141056,
> real_end=4292149248, new_pageable=0) at ../../vm/vm_map.c:1817
> #6 0xc023ea25 in kmem_alloc (map=0xc033322c, size=8192)
> at ../../vm/vm_kern.c:213
> #7 0xc024a46f in _zget (z=0xdb5c6e80) at ../../vm/vm_zone.c:425
> #8 0xc024a269 in zalloc (z=0xdb5c6e80) at ../../vm/vm_zone.c:60
> #9 0xc0196106 in namei (ndp=0xfa489ef4) at ../../kern/vfs_lookup.c:104
> #10 0xc01614ca in execve (p=0xfa482e00, uap=0xfa489f90)
> at ../../kern/kern_exec.c:165
> #11 0xc01590a1 in start_init (dummy=0x0) at ../../kern/init_main.c:543
> (kgdb) p nmbclusters
> $1 = 131072
> (kgdb)
>
> Goran Spirovski - MorEl On.net wrote:
>
>> AFAIK the number of mbufs (and consequently nmbclusters) has to be a
>> power
>> of 2, so you should set it to 131072
>>
>> MorEl
>>
>> ----- Original Message ----- From: "Karim Fodil-Lemelin" <kfl@xiphos.ca>
>> To: <freebsd-net@freebsd.org>
>> Sent: Wednesday, June 15, 2005 6:08 PM
>> Subject: (panic) Lots of network memory needed
>>
>>
>>
>>
>>> Hello,
>>>
>>> From kernel tuning page
>>>
>>
>> (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kern
>>
>> el-limits.html)
>>
>>
>>> Some sentence about nmbclusters says:
>>>
>>> "Under no circumstances should you specify an arbitrarily high value for
>>>
>>
>> this parameter as it could lead to a boot time crash."
>>
>>
>>> Now I want to push the limits where I need 4KB buffer for each of the
>>>
>>
>> 32000 connections I want this server to handle. If I do the math:
>>
>>
>>> (32000 (conns) * 4 (KB/buffer) * 2 (buffer/conn)) / 2048 (KB/cluster)
>>>
>>> = 128000 clusters
>>>
>>> So I set this arbitrary high value in loader.conf under
>>>
>>
>> (kern.ipc.nmbclusters) and no surprises I get panic: pmap_enter
>> invalid page
>> directory pdir=0x3cb063, va=0xfff800
>>
>>
>>> (va has a weird address here)
>>>
>>> I know I am pushing the limits here but I have plenty of memory (2GB) on
>>>
>>
>> this system (after all its just 250MB for network memory ;) and this is
>> mainly just experimentation.
>>
>>
>>> I would like some pointers toward fixing this. Is there another variable
>>>
>>
>> tied into this (I guess so)? Could anybody points me to a technical
>> document
>> that would explain the relationship with that (those) other(s) presumed
>> variable(s)?
>>
>>
>>> Thank you,
>>>
>>>
>>> Karim
>>>
>>>
>>> _______________________________________________
>>> freebsd-net@freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>>>
>>
>>
>>
>>
>>
>>
>
--
Dave Baukus
dbaukus@chiaro.com
Chiaro Networks Ltd.
Richardson, Texas
USA
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42B07A31.70201>