From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 12 08:43:38 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0FB1065688 for ; Sun, 12 Oct 2008 08:43:38 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA03.emeryville.ca.mail.comcast.net (qmta03.emeryville.ca.mail.comcast.net [76.96.30.32]) by mx1.freebsd.org (Postfix) with ESMTP id 119258FC0A for ; Sun, 12 Oct 2008 08:43:37 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA01.emeryville.ca.mail.comcast.net ([76.96.30.11]) by QMTA03.emeryville.ca.mail.comcast.net with comcast id RkjW1a0010EPchoA3kjdN4; Sun, 12 Oct 2008 08:43:37 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA01.emeryville.ca.mail.comcast.net with comcast id Rkjc1a0062P6wsM8MkjcNK; Sun, 12 Oct 2008 08:43:37 +0000 X-Authority-Analysis: v=1.0 c=1 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=eL-bSH8qi-lIiaZTkJsA:9 a=ZG7prpRk9s827enNUn2ru2NZ9X0A:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 580FCC9419; Sun, 12 Oct 2008 01:43:36 -0700 (PDT) Date: Sun, 12 Oct 2008 01:43:36 -0700 From: Jeremy Chadwick To: Nate Eldredge Message-ID: <20081012084336.GA84786@icarus.home.lan> References: <200810112330.53214.ken@mthelicon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org, Pegasus Mc Cleaft Subject: Re: ZFS boot X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Oct 2008 08:43:38 -0000 On Sat, Oct 11, 2008 at 04:21:55PM -0700, Nate Eldredge wrote: > On Sat, 11 Oct 2008, Pegasus Mc Cleaft wrote: > >>> FWIW, my system is amd64 with 1 G of memory, which the page implies is >>> insufficient. Is it really? >> >> This may be purely subjective, as I have never bench marked the speeds, but >> when I was first testing zfs on a i386 machine with 1gig ram, I thought the >> performance was mediocre. However, when I loaded the system on a quad core - >> core2 with 8 gigs ram, I was quite impressed. I put localized changes in my >> /boot/loader.conf to give the kernel more breathing room and disabled the >> prefetch for zfs. >> >> #more loader.conf >> vm.kmem_size_max="1073741824" >> vm.kmem_size="1073741824" >> vfs.zfs.prefetch_disable=1 > > I was somewhat confused by the suggestions on the wiki. Do the kmem_size > sysctls affect the allocation of *memory* or of *address space*? The Wiki is somewhat vague and doesn't give you all the knowledge you need. The kmem_* sysctls do not define pre-allocated amounts. They define the amount of memory which can be used by the kernel for allocation. I strongly advocate tuning two other sysctls, which can help greatly in ensuring no system lock-ups and no kmem exhaustion panics: vfs.zfs.arc_min vfs.zfs.arc_max The following values are what I use, but others have reported better performance with arc_max set to 128M: vfs.zfs.arc_min="16M" vfs.zfs.arc_max="64M" > It seems a bit much to reserve 1 G of memory solely for the use of the > kernel, expecially in my case when that's all I have :) But on amd64, > it's welcome to have terabytes of address space if it will help. ZFS is a memory hog, period. That's just the nature of the beast. You probably should not be using it on a system with 1GB. I'll remind you that memory right now is *incredibly* cheap; you can get 4GB of brand-name lifetime-warranty RAM for around US$40-50. Secondly, with regards to amd64: RELENG_6 and RELENG_7 amd64 cannot handle more than 2GB of kmem. Yes, you read that correct; it's not a typo. It's an implementation issue which cannot be easily solved on those releases. CURRENT can address up to 512GB. I've fully documented this on my Wiki, see section Kernel. http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |