From owner-freebsd-stable@FreeBSD.ORG Mon May 10 15:55:12 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 19D30106566B for ; Mon, 10 May 2010 15:55:12 +0000 (UTC) (envelope-from mandrews@bit0.com) Received: from magnum.bit0.com (magnum.bit0.com [207.246.88.226]) by mx1.freebsd.org (Postfix) with ESMTP id D8ACB8FC19 for ; Mon, 10 May 2010 15:55:11 +0000 (UTC) Received: from magnum.bit0.com (localhost [127.0.0.1]) by magnum.bit0.com (Postfix) with ESMTP id D26271105B for ; Mon, 10 May 2010 11:55:10 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=bit0.com; h=message-id :date:from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; s=boogity; bh=uDTXi9kma 9/YBys41W7fWNQlAbk6ZN99YCQOssth4QY=; b=krU0RqHnuNlTk7nqnteLpQqkI bLcs2EMq5SFrqR7UzceNV1bmxfOFsAtImxR4THnlN8rvqRp5Us/P1BB9OwiSwrpN oxwGiftj5vTXAEZrnGzviXWPxpcWUarIyHPZnWlv1XWQN1uUs/AE6OtG6XgS39UJ tofzxWLKN4uSMYs1tk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bit0.com; h=message-id:date :from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=boogity; b=xGZ lgEAdfeATV6V7D8jtivUda9r0qJYNzSZPqJsgg8fa1RfsoAoCHm0sEpidZQo7dR9 dVVOwUlqzneFTleOTfCgy8sHQPvquvrrsNpf4ywOGGoDgPvShAvu9VCetJ+KCcIz WTtABVuhJ5zxHwP4Q/QULVgVfy9lnSzYqHGfeh3Y= Received: from millenniumforce.int.bit0.com (207-246-87-56.dsl.static.blueone.net [207.246.87.56]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by magnum.bit0.com (Postfix) with ESMTPSA id 845AE11058 for ; Mon, 10 May 2010 11:55:10 -0400 (EDT) Message-ID: <4BE82C5D.1080806@bit0.com> Date: Mon, 10 May 2010 11:55:09 -0400 From: Mike Andrews User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4BDEA86E.3050109@zirakzigil.org> <20100503110100.GA93137@icarus.home.lan> <4BDEC106.3040807@zirakzigil.org> <4BE110E3.8040902@zirakzigil.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Freebsd 8.0 kmem map too small X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 May 2010 15:55:12 -0000 On 5/5/10 11:19 AM, Freddie Cash wrote: > On Tue, May 4, 2010 at 11:32 PM, Giulio Ferro wrote: > >> Giulio Ferro wrote: >> >>> Thanks, I'll try these settings. >>> >>> I'll keep you posted. >>> >> >> Nope, it's happened again... Now I've tried to rise vm.kmem_size to 6G... >> I'm really astounded at how unstable zfs is, it's causing me a lot of >> problem. >> Why isn't it stated in the handbook that zfs isn't up to production yet? >> > > As with everything related to computers, it all depends on your uses. Sorry to semi-hijack this, but... I'm also running into frequent "kmem_map too small" panics on 8-STABLE, such as: panic: kmem_malloc(131072): kmem_map too small: 2023780352 total allocated panic: kmem_malloc(131072): kmem_map too small: 2011525120 total allocated panic: kmem_malloc(114688): kmem_map too small: 1849356288 total allocated panic: kmem_malloc(114688): kmem_map too small: 1849356288 total allocated panic: kmem_malloc(114688): kmem_map too small: 1849356288 total allocated panic: kmem_malloc(131072): kmem_map too small: 2020409344 total allocated panic: kmem_malloc(536576): kmem_map too small: 2022957056 total allocated (those are over the course of 3-4 days) On this specific system, it has 32 GB physical memory and has vfs.zfs.arc_max="2G" and vm.kmem_size="64G" in /boot/loader.conf. The latter was added per earlier suggestions on this list, but appears to be ignored as "sysctl vm.kmem_size" returns about 2 GB (2172452864) anyway. Unfortunately I have not yet found a way to reliably reproduce the panic on demand, so it's hard to iteratively narrow down which date the potentially offending commit was on. It happened at least twice overnight, where several memory-intensive jobs run. Backing out to a previous 8-STABLE kernel from March 28 appears (so far) to have stabilized things, so the offending code was likely somewhere between then and May 5 or so. I do have KDB/DDB and serial console on this box, if there's any more info I can give to help troubleshoot other than just a one-month vague date range :) This is happening to more than just one system, but I figure it's easier to troubleshoot memory settings on the 32 GB i7 server instead of the 2 GB Atom one...