Date: Thu, 15 Oct 2009 18:15:18 -0400 From: grarpamp <grarpamp@gmail.com> To: freebsd-stable@freebsd.org Cc: freebsd-fs@freebsd.org Subject: Re: ZFS repeatable reboot 8.0-RC1 Message-ID: <d2e731a10910151515q601d9b90o66be76228a7d908a@mail.gmail.com> In-Reply-To: <43425103-6DAE-43FD-82E7-9B71BAB862B0@gothic.net.au> References: <d2e731a10910141555o604e0abewf8c5b5bdc948f4a0@mail.gmail.com> <alpine.BSF.2.00.0910142107230.72893@qvzrafvba.5c.ybpny> <d2e731a10910142347n52f60bey8e9058d2b76dccf7@mail.gmail.com> <43425103-6DAE-43FD-82E7-9B71BAB862B0@gothic.net.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Note: I tried setting vfs.zfs.arc_max=32M and zfs mem usage still grew past its limits and the machine rebooted. Forwarding a note I received... """"" >> Your machine is starving! > How can this be, there is over 500MiB free ram at all times? I'm sysctl vm.kmem_size_max I've got the following in my kernel configuration file options KVA_PAGES=512 and in /boot/loader.conf vm.kmem_size_max=1536M vm.kmem_size=1536M On two machines with 2G of RAM....both 8.0-RC1/i386 the ZFS tuning guide gives a better idea of how to play with things like that http://wiki.freebsd.org/ZFSTuningGuide """"" Some questions... Am I reading correctly that vm.kmem_size is how much ram the kernel initially allocates for itself on boot? And that vm.kmem_size_min and vm.kmem_size_max are the range that vm.kmem_size is allowed to float naturally within at runtime? Is KVA_PAGES the hard max space the kern is allowed/capable of addressing/using at runtime? Such that I could set kmem_size_max to the KVA_PAGES limit and then vm.kmem_size will grow into it as needed? With the caveat of course that with the below defaults and hardware, if I just bumped vm.kmem_size_max to 1GiB [as per KVA_PAGES limit] I'd have to add maybe another 1GiB ram so that this new vm.kmem_size_max kernel reservation wouldn't conflict with userland memory needs when vm.kmem_size grows to it? And KVA_PAGES is typically say 1/3 of installed ram? If vm.kmem_size starts out being under vm.kmem_size_max, can user apps use the unused space (vm.kmem_size_max - vm.kmem_size) until vm.kmem_size grows to vm.kmem_size_max and the kernel kills them off? Or can user apps only use (ram = user apps + [KVA_PAGES hard limit and/or vm.kmem_size_max])? What is the idea behind setting vm.kmem_size = vm.kmem_size_max? Should not just vm.kmem_size_max be set and allow vm.kmem_size [unset] to grow up to vm.kmem_size_max instead of allocating it all at boot with vm.kmem_size? Maybe someone can wikify these answers? I think I need to find more to read and then test one by one to see what changes. With untuned defaults and 1GiB ram I have: #define KVA_PAGES 256 # gives 1GiB kern address space vm.kmem_size_max: 335544320 # auto calculated by the kernel at boot? Less than KVA_PAGES? vm.kmem_size_min: 0 vm.kmem_size: 335544320 # amount in use at runtime? I'm still figuring out how to find and add all the kernel memory. Here's zfs: vfs.zfs.arc_meta_limit: 52428800 vfs.zfs.arc_meta_used: 56241732 # greater than meta_limit? vfs.zfs.arc_min: 26214400 vfs.zfs.arc_max: 209715200 vfs.zfs.vdev.cache.size: 10485760 vfs.zfs.vdev.cache.max: 16384 kstat.zfs.misc.arcstats.p: 20589785 # was 104857600 on boot kstat.zfs.misc.arcstats.c: 128292242 # was 209715200 on boot kstat.zfs.misc.arcstats.c_min: 26214400 kstat.zfs.misc.arcstats.c_max: 209715200 loader(8) vm.kmem_size Sets the size of kernel memory (bytes). This overrides the value determined when the kernel was compiled. Modifies VM_KMEM_SIZE. vm.kmem_size_min vm.kmem_size_max Sets the minimum and maximum (respectively) amount of ker- nel memory that will be automatically allocated by the ker- nel. These override the values determined when the kernel was compiled. Modifies VM_KMEM_SIZE_MIN and VM_KMEM_SIZE_MAX. sys/i386/include/pmap.h /* * Size of Kernel address space. This is the number of page table pages * (4MB each) to use for the kernel. 256 pages == 1 Gigabyte. * This **MUST** be a multiple of 4 (eg: 252, 256, 260, etc). * For PAE, the page table page unit size is 2MB. This means that 512 pages * is 1 Gigabyte. Double everything. It must be a multiple of 8 for PAE. */ #ifndef KVA_PAGES #ifdef PAE #define KVA_PAGES 512 #else #define KVA_PAGES 256 #endif #endif sys/i386/conf/NOTES # Change the size of the kernel virtual address space. Due to # constraints in loader(8) on i386, this must be a multiple of 4. # 256 = 1 GB of kernel address space. Increasing this also causes # a reduction of the address space in user processes. 512 splits # the 4GB cpu address space in half (2GB user, 2GB kernel). For PAE # kernels, the value will need to be double non-PAE. A value of 1024 # for PAE kernels is necessary to split the address space in half. # This will likely need to be increased to handle memory sizes >4GB. # PAE kernels default to a value of 512. # options KVA_PAGES=260
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d2e731a10910151515q601d9b90o66be76228a7d908a>