From owner-freebsd-current@FreeBSD.ORG  Tue Mar 11 10:28:55 2008
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 8B0FF1065675;
	Tue, 11 Mar 2008 10:28:55 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id B67DF8FC29;
	Tue, 11 Mar 2008 10:28:53 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <47D65EE5.902@FreeBSD.org>
Date: Tue, 11 Mar 2008 11:28:53 +0100
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.12 (Macintosh/20080213)
MIME-Version: 1.0
To: Thomas Vogt <freebsdlists@bsdunix.ch>
References: <47D544B1.6070806@bsdunix.ch> <47D5D2B2.90202@FreeBSD.org>
	<EEB46FCB-A244-4204-82F2-5CB37448F6E7@bsdunix.ch>
In-Reply-To: <EEB46FCB-A244-4204-82F2-5CB37448F6E7@bsdunix.ch>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: freebsd-fs@FreeBSD.org, current@freebsd.org
Subject: Re: vm_thread_new: kstack allocation failed with many ZFS FS and
 NFSD
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Mar 2008 10:28:55 -0000

Thomas Vogt wrote:
> Hi Kris
> 
> Am 11.03.2008 um 01:30 schrieb Kris Kennaway:
>> Thomas Vogt wrote:
>>> Hi List(s)
>>> I try to simulate real workload for our environment in my lab. The idea
>>> was to create 10k+ ZFS fs with several thousand files on each fs and
>>> then measure daily workload performance. Maybe 10k fs sounds silly but
>>> if you need individual quota for every user on a system, 5-10k fs are
>>> not unusual for ZFS
>>> My script to cerate zfs fs
>>> #!/bin/sh
>>> i=0; while [ $i != 10000 ]; do zfs create tank/script$i; i=`expr $i +
>>> 1`; done
>>> My script stopped after creating ~4850 FS with:
>>> vm_thread_new: kstack allocation failed
>>> vm_thread_new: kstack allocation failed
>>> vm_thread_new: kstack allocation failed
>>> vm_thread_new: kstack allocation failed
>>> vm_thread_new: kstack allocation failed
>>> vm_thread_new: kstack allocation failed
>>
>> Your kernel has run out of memory.  If you cannot tune kmem_size 
>> further then it cannot handle this many ZFS filesystems.
> 
> Are there no limitation for vm.kmem_size* sysctls? I tried to increase 
> vm.kmem_size* with larger values than 1500M but the system paniced in 
> the boot process.

Yes, there is an upper bound somewhere around this point with the 
default kernel layout.

> Mark Tinguely told me maybe i can edit sys/amd64/include/pmap.h and 
> change the line:
> 
> - #define KPDPI         (NPDPEPG-2)    /* kernbase at -2GB */
> + #define KPDPI         (NPDPEPG-4)    /* kernbase at -4GB */
> 
> I will try this. Any idea if this is save?

I don't know, sorry.

Kris