From owner-freebsd-hackers Sat Mar 21 13:48:56 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA07850 for freebsd-hackers-outgoing; Sat, 21 Mar 1998 13:48:56 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA07836 for ; Sat, 21 Mar 1998 13:48:47 -0800 (PST) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id OAA24514; Sat, 21 Mar 1998 14:48:44 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp02.primenet.com, id smtpd024494; Sat Mar 21 14:48:42 1998 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id OAA06529; Sat, 21 Mar 1998 14:48:28 -0700 (MST) From: Terry Lambert Message-Id: <199803212148.OAA06529@usr09.primenet.com> Subject: Re: How do you increase available SYSV shared memory? To: dec@phoenix.its.rpi.edu (David E. Cross) Date: Sat, 21 Mar 1998 21:48:27 +0000 (GMT) Cc: eivind@yes.no, shimon@simon-shapiro.org, kgor@inetspace.com, roberto@keltia.freenix.fr, freebsd-hackers@FreeBSD.ORG In-Reply-To: from "David E. Cross" at Mar 20, 98 04:29:34 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > > > I have been thinking of changing Postgres to use mmapped files instead > > > > of SYSV shared memory. I think this should allow for larger postgres > > > > > > This will be a disaster. It assumes that PostgreSQL uses files for data > > > storage. While this is the default mode, it is NOT the only storage > > > meanager. In PostgreSQL, like most true RDBMS, the storage of data is > > > decoupled from the logic of the relational model, etc. I am building a > > > storage manager that uses a totally different (distributed) storage model > > > than Unix files. A memory based storage manager already exists in > > > PostgreSQL. Please do not break these. > > > > I don't think you're quite getting him (or I'm not getting you at all). > > mmap()ing /dev/zero is a common way of getting hold of shared memory, > > instead of using the SYSV SHMEM extension. mmap'ing usually works better. > > > > This is just replacing one technique for getting hold of shared memory with > > another; it does nothing to the storage manager. > > This is very common indeed, it is how the dynamic linker on solaris works. As a point of interest, virtual memory *without* a backing object works *significantly* faster than virtual memory *with* a backing object. This should be pretty obvious from simple observation of what a backing object means: it means that you must maintain write-through cache coherency for the object, going through the VFS code to do it. There are a number of large wins you could wring out of the VFS code that FreeBSD may be on the road to wringing, but a backing object still tends to nail flodding after the high water-mark to the speed of the pager (either the swap pager (which is fast) or the vnode pager (which is less fast). I think John Dyson did significant work to speed up anonymous shared memory. I *think* it should work with /dev/zero, but of course, you would have to implement some mechanism of region sharing on top of that (/dev/zero pages are copy-on-write, and would only be shared by child processes, and then only in specific rfork circumstances). That said, I believe there are currently reasons, until you can use a procfs call to adopt a copy-on-write region as a non-copy-on-write region in another process (ie: copy the pages if they are /dev/zero pages, but share them otherwise), that SYSV SHMEM is actually a better (in terms of performance) technology. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message