From owner-freebsd-fs@FreeBSD.ORG  Mon Oct  8 16:33:51 2007
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 1B1E616A417;
	Mon,  8 Oct 2007 16:33:51 +0000 (UTC)
	(envelope-from cb@severious.net)
Received: from ion.gank.org (ion.gank.org [69.55.238.164])
	by mx1.freebsd.org (Postfix) with ESMTP id 0597F13C45A;
	Mon,  8 Oct 2007 16:33:50 +0000 (UTC)
	(envelope-from cb@severious.net)
Received: by ion.gank.org (Postfix, from userid 1001)
	id AF63611794; Mon,  8 Oct 2007 11:33:50 -0500 (CDT)
Date: Mon, 8 Oct 2007 11:33:49 -0500
From: Craig Boston <cb@severious.net>
To: Pawel Jakub Dawidek <pjd@FreeBSD.org>
Message-ID: <20071008163349.GB98555@nowhere>
Mail-Followup-To: Craig Boston <cb@severious.net>,
	Pawel Jakub Dawidek <pjd@FreeBSD.org>, freebsd-fs@FreeBSD.org,
	freebsd-current@FreeBSD.org
References: <20071005000046.GC92272@garage.freebsd.pl>
	<20071008121523.GM2327@garage.freebsd.pl>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20071008121523.GM2327@garage.freebsd.pl>
User-Agent: Mutt/1.4.2.3i
Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org
Subject: Re: ZFS kmem_map too small.
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 08 Oct 2007 16:33:51 -0000

On Mon, Oct 08, 2007 at 02:15:23PM +0200, Pawel Jakub Dawidek wrote:
> It's a hack, yes, but allows to mitigate the problem quite well. I'm
> looking for a solution that can be used for 7.0 before we find a better
> fix.

Um, tsleep()ing when M_NOWAIT is set?  Yes, I'd call that quite a hack
:)

Sorry to spam the thread again, but one thing I noticed is that zfs does
an awful lot of allocations of various sizes:

$ uptime
11:30AM  up 2 days, 18:06, 11 users, load averages: 0.05, 0.12, 0.15
(was idle most of the weekend)

$ vmstat -m | grep solaris
      solaris 83176 341456K       - 53216607  16,32,64,128,256,512,1024,2048,4096

I'm not completely up-to-date on what algorithm the kernel allocator is
using these days, but is it possible that kernel memory is getting
fragmented by all of those allocations?

Craig