From owner-freebsd-stable@FreeBSD.ORG Fri Oct 27 16:47:05 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BD0016A492 for ; Fri, 27 Oct 2006 16:47:05 +0000 (UTC) (envelope-from decibel@decibel.org) Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5581443D62 for ; Fri, 27 Oct 2006 16:47:01 +0000 (GMT) (envelope-from decibel@decibel.org) Received: by noel.decibel.org (Postfix, from userid 1001) id 0F44F5645B; Fri, 27 Oct 2006 11:46:48 -0500 (CDT) Received: (hashcash-sendmail, from uid 1001); Fri, 27 Oct 2006 11:46:44 -0500 Date: Fri, 27 Oct 2006 11:46:44 -0500 From: "Jim C. Nasby" To: Bucky Jordan Message-ID: <20061027164643.GB11985@decibel.org> References: <20061027044403.GK26892@decibel.org> <78ED28FACE63744386D68D8A9D1CF5D4209DF4@MAIL.corp.lumeta.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <78ED28FACE63744386D68D8A9D1CF5D4209DF4@MAIL.corp.lumeta.com> X-Operating-System: FreeBSD 6.1-RELEASE-p10 amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.13 (2006-08-11) X-Hashcash: 1:20:061027:bjordan@lumeta.com::ACtzpX7hWmwWlrgN:0000000000000000000 0000000000000000000000004V48 X-Hashcash: 1:20:061027:dmagda@ee.ryerson.ca::XbUb3djfQKfKNRWy:00000000000000000 0000000000000000000000005v4V X-Hashcash: 1:20:061027:mikej@rogers.com::PiJIUHZXumLvjkig:01qdt X-Hashcash: 1:20:061027:stable@freebsd.org::DLxZfX5Lu4wfsiaY:0000000000000000000 00000000000000000000000009fW Cc: Mike Jakubik , stable@freebsd.org, David Magda Subject: Re: Running large DB's on FreeBSD X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 16:47:05 -0000 On Fri, Oct 27, 2006 at 10:06:30AM -0400, Bucky Jordan wrote: > > -----Original Message----- > > From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd- > > stable@freebsd.org] On Behalf Of Jim C. Nasby > > Sent: Friday, October 27, 2006 12:44 AM > > To: David Magda > > Cc: Mike Jakubik; stable@freebsd.org > > Subject: Re: Running large DB's on FreeBSD > > > > On Mon, Oct 23, 2006 at 08:15:04PM -0400, David Magda wrote: > > > As for Postgres on FreeBSD, FlighAware seems to be using it some > some > > > decent amount of data: > > > > > > >. Receiving the data and processing it puts them about 6 minutes > > > >behind real time > > > >. Generating one map can be done in about 160 milliseconds of CPU > time > > > >. Capable of generating several million maps a day > > > >. About 1 TB of stored data > > > >. Approximately 40 million position updates on air craft per day > > > > > > http://joseph.randomnetworks.com/archives/2006/05/12/flightaware- > > > freebsd-and-postgresql/ > > > > And that's on a dual opteron with 12G of memory and a run of the mill > > RAID10 (for the database that is). > > Yes.. but how many disks (size/type/rpm?) are in that RAID 10? I'm > guessing it's an external enclosure... > > Also, I know 10k rpm vs 15 doesn't make much of a difference for > sequential, but random IO seems to be significantly improved. Granted, > it's not as dramatic as adding more spindles... IIRC it's a 6 drive array of SATA. Nothing all that fancy. > I think the other point that may be relevant is the active section of > the data that you're accessing, and how good your design is in terms of > being able to access that directly. You could have a 1TB database, but > only have a portion that is frequently accessed/updated. In that case, > you might just need lots of storage, which is fairly inexpensive these > days. Also, your money might be better spent on more RAM- if you can fit > most of the active data in memory, that will also have a positive impact > on performance. > > As pointed out, 10GB isn't really that much, especially when you can buy > relatively inexpensive servers with 8 or 16 GB of ram. Fitting over half > your db in memory is quit a luxury. Well, what's most important is your system architecture. If you have a poor design to start with, you'll never get good performance out of it. -- Jim C. Nasby, Database Architect decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"