From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 11 05:25:42 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 854801065670 for ; Thu, 11 Sep 2008 05:25:42 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id 4FE858FC29 for ; Thu, 11 Sep 2008 05:25:42 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 057232C2D2E; Thu, 11 Sep 2008 00:07:26 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 7KxbykW8z9YM; Thu, 11 Sep 2008 00:07:25 -0500 (CDT) Received: from [216.63.78.18] (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 2B8852C2D0F; Thu, 11 Sep 2008 00:07:25 -0500 (CDT) Message-ID: <48C8A78C.6070608@cs.rice.edu> Date: Thu, 11 Sep 2008 00:07:24 -0500 From: Alan Cox User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20080722 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Artem Belevich References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Alan Cox Subject: Re: Increasing KVM on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2008 05:25:42 -0000 Artem Belevich wrote: >Alan, > >Thanks a lot for the patch. I've applied it to RELENG_7 and it seems >to work great - "make -j8 buildworld" succeeds, linux emulation seems >to work well enough to run linux-sun-jdk14 binaries, ZFS ARC size is >bigger, too. So far I didn't see any ZFS-related KVM shortages either. > >The only problem is that everything is fine as long as vm.kmem_size is >set to less or equal to 4096M. As soon as I set it to 4100M or >anything larger, kernel crashes on startup. I'm unable to capture >exact crash messages as they keep scrolling really fast on the screen >for a few seconds until the box reboots. Unfortunately the box does >not have built-in serial ports, so the messages are gone before I can >see them. :-( > > > There are two underlying causes. First, the size of the kmem map, which holds the kernel's heap, is recorded in a 32-bit int. So, setting vm.kmem_size to 4100M is leading to integer overflow. The following change addresses this issue: sys/kern/kern_malloc.c Revision *1.167*: download - view: text , markup , annotated - select for diffs /Sat Jul 5 19:34:33 2008 UTC/ (2 months ago) by /alc/ Branches: MAIN CVS tags: HEAD Diff to: previous 1.166: preferred , colored Changes since revision 1.166: +11 -11 lines SVN rev 180308 on 2008-07-05 19:34:33Z by alc Enable the creation of a kmem map larger than 4GB. Submitted by: Tz-Huan Huang Make several variables related to kmem map auto-sizing static. Found by: CScout Second, there is no room for a kmem map greater than 4GB unless the overall KVM size is greater than 6GB. Specifically, a >4GB kmem map isn't possible with 6GB KVM because the kmem map would overlap the kernel's code, data, and bss segment. If you're able to apply the above kern_malloc.c change to your kernel, then I should be able to describe how to increase your KVM beyond 6GB. Regards, Alan