From owner-freebsd-hardware@FreeBSD.ORG Fri Oct 31 04:08:01 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2F72106567A; Fri, 31 Oct 2008 04:08:01 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id 826F68FC14; Fri, 31 Oct 2008 04:08:01 +0000 (UTC) (envelope-from fbsd@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1KvlIb-000JJS-BH; Fri, 31 Oct 2008 14:08:00 +1000 Message-ID: <490A849C.7030009@dannysplace.net> Date: Fri, 31 Oct 2008 14:07:56 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Jeremy Chadwick References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> In-Reply-To: <20081031033208.GA21220@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-10-31 14:07:50 X-Connected-IP: 203.206.171.212:1636 X-Message-Linecount: 84 X-Body-Linecount: 70 X-Message-Size: 3160 X-Body-Size: 2565 X-Received-Count: 1 X-Recipient-Count: 3 X-Local-Recipient-Count: 3 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: koitsu@FreeBSD.org, freebsd-hardware@freebsd.org, freebsd-fs@freebsd.org X-SA-Exim-Mail-From: fbsd@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fbsd@dannysplace.net List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2008 04:08:01 -0000 Jeremy Chadwick wrote: > I think these sets of tests are good. There are some others I'd like to > see, but they'd only be applicable if the 1231-ML has hardware cache. I > can mention what those are if the card does have hardware caching. The card comes standard with 256Mb of cache. >> I do have some concern about the size of the eventual array and ZFS' use >> of system memory. Are there guidelines available that give advice on >> how much memory a box should have with large ZFS arrays? > > The general concept is: "the more RAM the better". However, if you're > using RELENG_7, then there's not much point (speaking solely about ZFS) > to getting more than maybe 3 or 4GB; you're still limited to a 2GB kmap > maximum. > > Regarding size of the array vs. memory usage: as long as you tune kmem > and ZFS ARC, you shouldn't have much trouble. There have been some > key people reporting lately that they run very large ZFS arrays without > issue, with proper tuning. I followed the recommendations here: http://wiki.freebsd.org/ZFSTuningGuide vm.kmem_size="1024M" vm.kmem_size_max="1024M" vfs.zfs.debug=1 And : kern.maxvnodes=400000 I have not added the following because they were listed in the i386 section. (These values were quoted for a machine with 768Mb of ram) vfs.zfs.arc_max="40M" vfs.zfs.vdev.cache.size="5M" Am I right in assuming these do not apply to amd64? The article was not specific. > > Also, just a reminder: do not pick a value of 2048M for kmem_size or > kmem_size_max; the machine won't boot/work. You shouldn't go above > something like 1536M, although some have tuned slightly above that > with success. (You need to remember that there is more to kernel > memory allocation than just this, so you don't want to exhaust it all > assigning it to kmap. Hope that makes sense...) It makes sense. I'm using 1024 at the moment, but I've never really looked into what memory is actually being used. Tuning advice here would be well received :-) >> Can an AMD64 kernel make use of memory above 2g? > > Only on CURRENT; 7.x cannot, and AFAIK, will never be able to, as the > engineering efforts required to fix it are too great. > > I look forward to seeing your numbers. Someone here might be able to > compile them into some graphs and other whatnots to make things easier > for future readers. Ahhh, well, that will eventually decide my upgrade path when RELENG_8 is released and stable. > Thanks for doing all of this! No worries, hopefully it will be useful information to future google searches :-P -D