From owner-freebsd-fs@FreeBSD.ORG  Sat Jan 23 16:54:10 2010
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 97E441065670
	for <freebsd-fs@freebsd.org>; Sat, 23 Jan 2010 16:54:10 +0000 (UTC)
	(envelope-from mcdouga9@egr.msu.edu)
Received: from mx.egr.msu.edu (surfnturf.egr.msu.edu [35.9.37.164])
	by mx1.freebsd.org (Postfix) with ESMTP id 642088FC13
	for <freebsd-fs@freebsd.org>; Sat, 23 Jan 2010 16:54:09 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
	by mx.egr.msu.edu (Postfix) with ESMTP id 3A433101114;
	Sat, 23 Jan 2010 11:54:09 -0500 (EST)
X-Virus-Scanned: amavisd-new at egr.msu.edu
Received: from mx.egr.msu.edu ([127.0.0.1])
	by localhost (surfnturf.egr.msu.edu [127.0.0.1]) (amavisd-new,
	port 10024)
	with ESMTP id o8vSi8DY1qeI; Sat, 23 Jan 2010 11:54:09 -0500 (EST)
Received: from localhost (daemon.egr.msu.edu [35.9.44.65])
	by mx.egr.msu.edu (Postfix) with ESMTP id E4BAA101110;
	Sat, 23 Jan 2010 11:54:08 -0500 (EST)
Received: by localhost (Postfix, from userid 21281)
	id D451E8895C; Sat, 23 Jan 2010 11:54:08 -0500 (EST)
Date: Sat, 23 Jan 2010 11:54:08 -0500
From: Adam McDougall <mcdouga9@egr.msu.edu>
To: Artem Belevich <fbsdlist@src.cx>
Message-ID: <20100123165408.GR86054@egr.msu.edu>
References: <4B58BD2D.30803@rcn.com>
	<ed91d4a81001211421r4f7ba7a8n1c92bfc413e5feed@mail.gmail.com>
	<4B58D4D3.80009@egr.msu.edu> <20100122042843.GA8858@polands.org>
	<ed91d4a81001212209o3cf5955fw1eb0dd78703ad905@mail.gmail.com>
	<a8b9c9e35a14c4ada1c8f2602c91db63.squirrel@email.polands.org>
	<a4ab3938f95de65086a18607309178c4.squirrel@email.polands.org>
	<ed91d4a81001221211j2bc05b6bv22a51a8b2fcbcec1@mail.gmail.com>
	<1308c71eec426200d4c34b926bba8806.squirrel@email.polands.org>
	<ed91d4a81001221824u5c222e08l80395398c0a61c16@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ed91d4a81001221824u5c222e08l80395398c0a61c16@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: freebsd-fs@freebsd.org
Subject: Re: Repeatable ZFS "kmem map too small" panic on 8.0-STABLE
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Jan 2010 16:54:10 -0000

On Fri, Jan 22, 2010 at 06:24:36PM -0800, Artem Belevich wrote:

  > % sysctl hw.physmem vm.kmem_size vm.kmem_size_max vfs.zfs.arc_max
  >
  > hw.physmem:4102688768
  > vm.kmem_size: 2147483648
  
  Here's your problem -- kmem_size is for some reason only 2G.
  
  Argh! I ran into that before. The code in sys/kern/kern_malloc.c
  intentionally limits kmem_size to twice the physical memory size:
  
          /*
           * Limit kmem virtual size to twice the physical memory.
           * This allows for kmem map sparseness, but limits the size
           * to something sane. Be careful to not overflow the 32bit
           * ints while doing the check.
           */
          if (((vm_kmem_size / 2) / PAGE_SIZE) > cnt.v_page_count)
                  vm_kmem_size = 2 * cnt.v_page_count * PAGE_SIZE;
  
  So, either comment out these lines or just set vm.kmem_size to
  slightly below 8G.
  
  --Artem

That works for me and a friend of mine, I didn't realize that my kmem_size
was being capped.  I tested it on a 8.0-prerel server with 8G of ram,
kmem_size set to 20G in the loader but the resulting kmem_size was less than
4G before the patch.  Now it is 20G.  So, I'm not sure it calculates it properly?