From owner-freebsd-current@FreeBSD.ORG Wed Sep 26 10:01:44 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C816016A420; Wed, 26 Sep 2007 10:01:44 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id 91EF713C457; Wed, 26 Sep 2007 10:01:44 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from compute1.internal (compute1.internal [10.202.2.41]) by out1.messagingengine.com (Postfix) with ESMTP id 82FDE2F4AC; Wed, 26 Sep 2007 06:01:43 -0400 (EDT) Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute1.internal (MEProxy); Wed, 26 Sep 2007 06:01:43 -0400 X-Sasl-enc: Cc8Gnxu0qAflUfU5vMawqPso89wzlGGq0qeL9DRwBS63 1190800903 Received: from [192.168.1.235] (64-142-85-108.dsl.dynamic.sonic.net [64.142.85.108]) by mail.messagingengine.com (Postfix) with ESMTP id B79CC233A4; Wed, 26 Sep 2007 06:01:42 -0400 (EDT) Message-ID: <46FA2E46.8020303@freebsd.org> Date: Wed, 26 Sep 2007 03:02:46 -0700 From: Darren Reed User-Agent: Thunderbird 2.0.0.0 (Windows/20070326) MIME-Version: 1.0 To: Kris Kennaway References: <20070921102946.T11189@borg> <46F415BF.9010500@FreeBSD.org> <20070921140550.D96923@thebighonker.lerctr.org> <46F41CFF.6080108@FreeBSD.org> <46F58799.1030702@freebsd.org> <46F58B21.8030307@FreeBSD.org> <20070924091558.GB32006@team.vega.ru> <46F78C59.1020801@FreeBSD.org> <20070924080347.O84223@thebighonker.lerctr.org> <20070924144210.GA82735@team.vega.ru> <46F7D7A4.5090007@samsco.org> <46F80A39.3050707@FreeBSD.org> <46F8951C.50904@freebsd.org> <46F8CE67.60206@FreeBSD.org> In-Reply-To: <46F8CE67.60206@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Larry Rosenman Subject: Re: panic: kmem_malloc(131072): kmem_map too small (AMD64) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Sep 2007 10:01:44 -0000 Kris Kennaway wrote: > Darren Reed wrote: >> Kris Kennaway wrote: >>> ... >>> Well yes, that is one hypothesis, but the evidence points elsewhere >>> as well. Prior the change you reference, some of my zfs machines >>> would run for weeks before hitting a load pattern that exhausted >>> their kmem_map and triggered the panic. Also unless I have missed >>> it I am not seeing the sudden flood of panic reports that may >>> indicate sudden breakage. It is quite possible that this particular >>> report has nothing to do with the recent change. >> >> Indeed. >> >> But here's something else to ponder... >> >> I've been using ZFS since it was internal beta at Sun, at first on >> i386 and later on amd64. >> I've never run into this kind of panic on Solaris. System can get >> very slow, yes, with >> ZFS hogging lots of memory, but it never panic'd because of it. >> >> We need to come up with a strategy here to solve this problem, be it >> fixing the kmem >> virtual memory or fixing zfs. > > Yes, Solaris does something architecturally different because it is > apparently acceptable for zfs to use gigabytes of memory by default. Well, if you were designing a file system for servers, is there any reason that you wouldn't try to use all of the RAM available? A similar thought process goes into having a unified buffer cache that uses all the free RAM that it can (on a 1.5GB NetBSD box, 1.4GB is file cache.) Even if I'm running a desktop workstation, if I'm not there, there's no reason that ZFS shouldn't be able to encourage the OS to swap out all of the applications (well as much as can be) if it so desires. The problem comes in deciding how strong ZFS's hold should be and how to apply pressure from other parts of the system that want to use the RAM as well. Now given that we have a ZFS tuning guide, surely the question we need to ask ourselves is why can't we take the recommendations from that and make up some code to implement the trends discussed? And how do we measure how much memory ZFS is using? Darren