From owner-freebsd-stable@FreeBSD.ORG Thu Jun 4 19:15:15 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 937CF1065677 for ; Thu, 4 Jun 2009 19:15:15 +0000 (UTC) (envelope-from ben@wanderview.com) Received: from mail.wanderview.com (mail.wanderview.com [66.92.166.102]) by mx1.freebsd.org (Postfix) with ESMTP id 328578FC1E for ; Thu, 4 Jun 2009 19:15:13 +0000 (UTC) (envelope-from ben@wanderview.com) Received: from [192.168.1.105] (static-70-108-250-162.res.east.verizon.net [70.108.250.162]) (authenticated bits=0) by mail.wanderview.com (8.14.3/8.14.3) with ESMTP id n54IZmCY053318 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 4 Jun 2009 18:35:54 GMT (envelope-from ben@wanderview.com) Message-Id: <3B36D8F8-7325-49A3-9F1F-42326C661E6D@wanderview.com> From: Ben Kelly To: Tim Chase In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Thu, 4 Jun 2009 14:35:42 -0400 References: X-Mailer: Apple Mail (2.935.3) X-Spam-Score: -0.72 () AWL X-Scanned-By: MIMEDefang 2.64 on 10.76.20.1 Cc: freebsd-stable@freebsd.org Subject: Re: kmem map too small panic after updating to STABLE-7 r192996 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 19:15:16 -0000 On Jun 4, 2009, at 10:48 AM, Tim Chase wrote: > vm.kmem_size="512M" > vm.kmem_size_max="512M" > vfs.zfs.arc_max="100M" > $1 = 0xc0792320 "kmem_malloc(131072): kmem_map too small: 325656576 > total allocated" It looks like you are suffering from fragmentation of your kmem address space. I've done some investigation of this on -current: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=909067+0+archive/2009/freebsd-current/20090503.freebsd-current The patch linked from that mail allows me to run with an untuned ARC size with kmem of 512 MB. It does this, however, by more aggressively limiting how the ARC grows so it doesn't need to trigger its aggressive memory reclamation algorithm. I haven't tested this patch against -stable or with Kip's more recent changes so I don't know if it will apply cleanly for you. I've had some luck avoiding the fragmentation by changing ZFS to use a separate memory pool with a 128KB slab allocator, but this isn't really ready for general usage. Unfortunately I've been quite busy at work lately and haven't had time to work on it in a few weeks. I think with the current code base your only options are to increase your kmem maximum or limit your ARC even further. I hope that helps. - Ben