Date: Mon, 26 Jul 2010 21:19:22 +0300 From: Andriy Gapon <avg@freebsd.org> To: Matthew Fleming <mdf356@gmail.com>, freebsd-arch@freebsd.org Subject: Re: amd64: change VM_KMEM_SIZE_SCALE to 1? Message-ID: <4C4DD1AA.3050906@freebsd.org> In-Reply-To: <AANLkTikY%2BnPTgBtDWcphNkOrW-Aif5TRSCuCn8BsK3p7@mail.gmail.com> References: <4C4DB2B8.9080404@freebsd.org> <AANLkTikY%2BnPTgBtDWcphNkOrW-Aif5TRSCuCn8BsK3p7@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
on 26/07/2010 20:04 Matthew Fleming said the following: > On Mon, Jul 26, 2010 at 9:07 AM, Andriy Gapon <avg@freebsd.org> wrote: >> Anyone knows any reason why VM_KMEM_SIZE_SCALE on amd64 should not be set to 1? >> I mean things potentially breaking, or some unpleasant surprise for an >> administrator/user... > > As I understand it, it's merely a resource usage issue. amd64 needs > page table entries for the expected virtual address space, so allowing > more than e.g. 1/3 of physical memory means needing more PTEs. But > the memory overhead isn't all that large IIRC: each 4k physical memory > devoted to PTEs maps 512 4k virtual addresses, or 2MB, so e.g. it > takes about 4MB reserved as PTE pages to map 2GB of kernel virtual > address space. My understanding is that paging entries are only allocated when actual (physical) memory allocation is done. But I am not sure. > Having cut my OS teeth on AIX/PowerPC where virutal address space is > free and has no relation to the size of the hardware page table, the > FreeBSD architecture limiting the size of the kernel virtual space > seemed weird to me. However, since FreeBSD also does not page kernel > data to disk, there's a good reason to limit the size of the kernel's > virtual space, since that also limits the kernel's physical space. > > In other words, setting it to 1 could lead to the system being out of > memory but not trying to fail kernel malloc requests. I'm not > entirely sure this is a new problem since one could also chew through > physical memory with sub-page uma allocations as well on amd64. Well, personally I would prefer kernel eating a lot of memory over getting "kmem_map too small" panic. Unexpectedly large memory usage by kernel can be detected and diagnosed, and then proper limits and (auto-)tuning could be put in place. Panic at some random allocation is not that helpful. Besides, presently there are more and more workloads that require a lot of kernel memory - e.g. ZFS is gaining popularity. Hence, the question/suggestion. Of course, the things can be tuned by hand, but I think that VM_KMEM_SIZE_SCALE=1 would be a more reasonable default than current value. -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C4DD1AA.3050906>