From owner-freebsd-current@FreeBSD.ORG Sat Mar 15 05:52:54 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6C78106566C for ; Sat, 15 Mar 2008 05:52:54 +0000 (UTC) (envelope-from ender@enderzone.com) Received: from www.ksdhost.com (www.ksdhost.com [75.126.66.82]) by mx1.freebsd.org (Postfix) with ESMTP id B0AC88FC21 for ; Sat, 15 Mar 2008 05:52:54 +0000 (UTC) (envelope-from ender@enderzone.com) Received: (qmail 25730 invoked from network); 15 Mar 2008 00:26:13 -0500 Received: from 107.94.144.216.westtel.ky (HELO ?192.168.2.6?) (216.144.94.107) by www.ksdhost.com with SMTP; 15 Mar 2008 00:26:13 -0500 Message-ID: <47DB5DD2.2060502@enderzone.com> Date: Sat, 15 Mar 2008 01:25:38 -0400 From: Ender User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) MIME-Version: 1.0 To: Thomas Vogt References: <47D544B1.6070806@bsdunix.ch> <47D5D2B2.90202@FreeBSD.org> <47D65EE5.902@FreeBSD.org> In-Reply-To: <47D65EE5.902@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, current@freebsd.org Subject: Re: vm_thread_new: kstack allocation failed with many ZFS FS and NFSD X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2008 05:52:55 -0000 Kris Kennaway wrote: > Thomas Vogt wrote: >> Hi Kris >> >> Am 11.03.2008 um 01:30 schrieb Kris Kennaway: >>> Thomas Vogt wrote: >>>> Hi List(s) >>>> I try to simulate real workload for our environment in my lab. The >>>> idea >>>> was to create 10k+ ZFS fs with several thousand files on each fs and >>>> then measure daily workload performance. Maybe 10k fs sounds silly but >>>> if you need individual quota for every user on a system, 5-10k fs are >>>> not unusual for ZFS >>>> My script to cerate zfs fs >>>> #!/bin/sh >>>> i=0; while [ $i != 10000 ]; do zfs create tank/script$i; i=`expr $i + >>>> 1`; done >>>> My script stopped after creating ~4850 FS with: >>>> vm_thread_new: kstack allocation failed >>>> vm_thread_new: kstack allocation failed >>>> vm_thread_new: kstack allocation failed >>>> vm_thread_new: kstack allocation failed >>>> vm_thread_new: kstack allocation failed >>>> vm_thread_new: kstack allocation failed >>> >>> Your kernel has run out of memory. If you cannot tune kmem_size >>> further then it cannot handle this many ZFS filesystems. >> >> Are there no limitation for vm.kmem_size* sysctls? I tried to >> increase vm.kmem_size* with larger values than 1500M but the system >> paniced in the boot process. > > Yes, there is an upper bound somewhere around this point with the > default kernel layout. > >> Mark Tinguely told me maybe i can edit sys/amd64/include/pmap.h and >> change the line: >> >> - #define KPDPI (NPDPEPG-2) /* kernbase at -2GB */ >> + #define KPDPI (NPDPEPG-4) /* kernbase at -4GB */ >> >> I will try this. Any idea if this is save? > > I don't know, sorry. > > Kris > > It does not compile for me. With acpi removed: MAKE=make sh /usr/src/sys/conf/newvers.sh NFSD1 cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mcmodel=kernel -mno-red-zone -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror vers.c linking kernel locore.o(.text+0x19): In function `btext': : relocation truncated to fit: R_X86_64_32S .bss rijndael-alg-fst.o(.text+0xc0): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0xd0): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0xe1): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0xf2): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x105): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x1ce): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x1da): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x1e8): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x209): In function `rijndaelKeySetupEnc': : relocation truncated to fit: R_X86_64_32S .rodata rijndael-alg-fst.o(.text+0x213): In function `rijndaelKeySetupEnc': : additional relocation overflows omitted from the output *** Error code 1 Stop in /usr/obj/usr/src/sys/NFSD1. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. nfsd1# With acpi in the kernel config it just fails on it, same type of R_X86_64_32S error i believe