From owner-freebsd-database@FreeBSD.ORG Sat Jul 26 00:19:25 2003 Return-Path: Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E08A737B401; Sat, 26 Jul 2003 00:19:25 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB16543F85; Sat, 26 Jul 2003 00:19:24 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-2ivfj2b.dialup.mindspring.com ([165.247.204.75] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19gJKi-0004SK-00; Sat, 26 Jul 2003 00:19:13 -0700 Message-ID: <3F222B36.CFAD4391@mindspring.com> Date: Sat, 26 Jul 2003 00:18:15 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Tom Samplonius References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a468c5c12d8272b9e579ad62b00e49fc6ba2d4e88014a4647c350badd9bab72f9c350badd9bab72f9c cc: Sean Chittenden cc: freebsd-database@freebsd.org cc: freebsd-performance@freebsd.org cc: Christopher Weimann cc: Paul Pathiakis Subject: Re: Tuning for PostGreSQL Database X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 07:19:26 -0000 Tom Samplonius wrote: > On Fri, 25 Jul 2003, Terry Lambert wrote: > > Ideally, you would use memory mapped files for this, and not System V > > shared memory, so that the OS could implement swapping policies as it > > saw fit, and could actually swap the data, if nevcessary, instead of > > it sucking up huge amounts of wired memory. > > PostgreSQL is from the good old days of RDMSes when the they would > System V shared memory for everything, and store databases on raw devices > in an effort to utilize as little of the OS as possible, in effort to be > fast and reliable. > > But it does give PostgreSQL the advantage of working with large tables > and databases. Mmapping a file over 4GB in size would likely exhaust the > VM on a x86. Or, is it possible to map 4+GB with PAE? It's not possible. PAE only provides the ability to utilize a larger-than-4G amount of RAM consecutively in different processes. It doesn't buy you the ability to have 4+GB System V shared memory segments, either. The limitation is based on pointer size in user space, which is, in turn, based on the size of the hardware registers. PAE is basically good for allowing you to utilize large amounts of RAM for seperate process insteances and/or RAM disks (not even cache), and is most often used merely to run a lot of processes without getting bottlenecked by disk I/O doing swapping. To get better than that, you need 64 bit pointers, which means a 64 bit architecture. System V shared memory segments are basically *subtracted* from your available memory. Memory mappes files don't benefit you (or hurt you) in terms of available address space for your process. But they *are* able to be swapped, and as long as you use madvise() and/or can live with "LRU" as your paging policy, they *will* give you more physical RAM to play with for other things, if it gets down to it, than using System V shared memory will. Maybe you could reimplement the code to allocate pageable RAM for shared memory segments; I don't really see how, without an alomost total rewrite of the System V shared memory code, though. -- Terry