Date: Thu, 11 Sep 2008 00:07:24 -0500 From: Alan Cox <alc@cs.rice.edu> To: Artem Belevich <fbsdlist@src.cx> Cc: freebsd-hackers@freebsd.org, Alan Cox <alc@cs.rice.edu> Subject: Re: Increasing KVM on amd64 Message-ID: <48C8A78C.6070608@cs.rice.edu> In-Reply-To: <ed91d4a80809101612t4e74a1e6y5ffffcf7ebf63d5d@mail.gmail.com> References: <ed91d4a80809101612t4e74a1e6y5ffffcf7ebf63d5d@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Artem Belevich wrote: >Alan, > >Thanks a lot for the patch. I've applied it to RELENG_7 and it seems >to work great - "make -j8 buildworld" succeeds, linux emulation seems >to work well enough to run linux-sun-jdk14 binaries, ZFS ARC size is >bigger, too. So far I didn't see any ZFS-related KVM shortages either. > >The only problem is that everything is fine as long as vm.kmem_size is >set to less or equal to 4096M. As soon as I set it to 4100M or >anything larger, kernel crashes on startup. I'm unable to capture >exact crash messages as they keep scrolling really fast on the screen >for a few seconds until the box reboots. Unfortunately the box does >not have built-in serial ports, so the messages are gone before I can >see them. :-( > > > There are two underlying causes. First, the size of the kmem map, which holds the kernel's heap, is recorded in a 32-bit int. So, setting vm.kmem_size to 4100M is leading to integer overflow. The following change addresses this issue: sys/kern/kern_malloc.c Revision *1.167*: download <http://www.freebsd.org/cgi/cvsweb.cgi/%7Echeckout%7E/src/sys/kern/kern_malloc.c?rev=1.167;content-type=text%2Fplain>; - view: text <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?rev=1.167;content-type=text%2Fplain>, markup <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?rev=1.167;content-type=text%2Fx-cvsweb-markup>, annotated <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?annotate=1.167>; - select for diffs <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?r1=1.167#rev1.167>; /Sat Jul 5 19:34:33 2008 UTC/ (2 months ago) by /alc/ Branches: MAIN <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?only_with_tag=MAIN>; CVS tags: HEAD <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c?only_with_tag=HEAD>; Diff to: previous 1.166: preferred <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c.diff?r1=1.166;r2=1.167>, colored <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_malloc.c.diff?r1=1.166;r2=1.167;f=h>; Changes since revision 1.166: +11 -11 lines SVN rev 180308 on 2008-07-05 19:34:33Z by alc Enable the creation of a kmem map larger than 4GB. Submitted by: Tz-Huan Huang Make several variables related to kmem map auto-sizing static. Found by: CScout Second, there is no room for a kmem map greater than 4GB unless the overall KVM size is greater than 6GB. Specifically, a >4GB kmem map isn't possible with 6GB KVM because the kmem map would overlap the kernel's code, data, and bss segment. If you're able to apply the above kern_malloc.c change to your kernel, then I should be able to describe how to increase your KVM beyond 6GB. Regards, Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48C8A78C.6070608>