From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 18 11:43:43 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7A68816A4CE for ; Thu, 18 Dec 2003 11:43:43 -0800 (PST) Received: from mail.advantagecom.net (mail.advantagecom.net [65.103.151.155]) by mx1.FreeBSD.org (Postfix) with ESMTP id 309D343D55 for ; Thu, 18 Dec 2003 11:43:32 -0800 (PST) (envelope-from andykinney@advantagecom.net) Received: from SCSI-MONSTER (scsi-monster.advantagecom.net [207.109.186.200]) by mail.advantagecom.net (8.11.6/8.11.6) with ESMTP id hBIJgkq10177; Thu, 18 Dec 2003 11:42:46 -0800 From: "Andrew Kinney" Organization: Advantagecom Networks, Inc. To: Julian Elischer Date: Thu, 18 Dec 2003 11:45:46 -0800 MIME-Version: 1.0 Message-ID: <3FE1936A.10308.75BF6ECA@localhost> Priority: normal In-reply-to: <3FE0EA71.10807@vicor.com> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body cc: hackers@freebsd.org Subject: Re: Machines with >= 4GB of RAM X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: andykinney@advantagecom.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Dec 2003 19:43:43 -0000 On 17 Dec 2003 at 15:44, Julian Elischer wrote: > options KVA_PAGES=512 > > may be a start, but is it still required, and do I have to change > anything else to match it? (where does the Makefile work out where to > link the kernel for?) > Is a value of 512 enough for a machine with 16GB of RAM? > > Any hints, (even a better google search string) appreciated. > > Julian > > Julian, We have a 4GB machine running 4.8-RELEASE, so we aren't using PAE, but we had to make changes similar to what you're asking about for a different reason. Your requirements will vary depending on the version of FreeBSD you're running, but in general, increasing KVA_PAGES will help considerably with stability on large memory machines. It should be noted that releases prior to 4.8 required more changes than just KVA_PAGES, but the documentation is a bit muddied on that subject. I don't know if it is required, but we rebuilt the world after changing KVA_PAGES just to make sure that any hidden dependencies on that value were handled in things other than the kernel. As far as 512 being a large enough setting for a 16GB machine, that depends entirely on what you plan to do with the machine and its usage pattern of various system resources. For instance, on our 4GB machine, it does a lot of heavy web serving, databases, and email. We needed the 2GB KVA on that machine because of large numbers of files, large network buffers, and some weirdness relating to Apache and pv entries. If your usage patterns were similar and you wanted to make full use of the 16GB without getting trap 12 panics, then 2GB KVA may be inadequate. My suggestion is to try it out and keep an eye on sysctl vm.kvm_free. When that starts to run low, it should be a pretty good indicator that you need a larger KVA_PAGES value. Regarding the remark by another fellow that AMD64 is a solution, it's only a solution when AMD64 support on FreeBSD is a bit more stable and complete. Also, there is the issue that big companies like Dell don't (and probably won't) offer any AMD powered systems due to probable contractual obligations to Intel. At companies like the one I work at, Dell provides generous financing terms and that weighs a bit on who gets our server dollars. I love the Opteron architecture, though, and one day hope to be able to plop down cash for a big ol' quad opteron man-machine with uber specs. At present, we're just waiting for OS support to catch up, though. Sincerely, Andrew Kinney President and Chief Technology Officer Advantagecom Networks, Inc. http://www.advantagecom.net