From owner-freebsd-hardware@FreeBSD.ORG Wed Aug 4 16:07:24 2004 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB85F16A4CE for ; Wed, 4 Aug 2004 16:07:24 +0000 (GMT) Received: from mproxy.gmail.com (rproxy.gmail.com [64.233.170.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E79A43D5E for ; Wed, 4 Aug 2004 16:07:24 +0000 (GMT) (envelope-from william.mansfield@gmail.com) Received: by mproxy.gmail.com with SMTP id 75so186617rnl for ; Wed, 04 Aug 2004 09:07:23 -0700 (PDT) Received: by 10.38.71.13 with SMTP id t13mr46046rna; Wed, 04 Aug 2004 09:07:23 -0700 (PDT) Message-ID: <95fd7dc704080409077deea4e9@mail.gmail.com> Date: Wed, 4 Aug 2004 12:07:23 -0400 From: William Mansfield To: Kathy Quinlan In-Reply-To: <4110C9C7.6080506@kaqelectronics.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <4110C9C7.6080506@kaqelectronics.dyndns.org> cc: freebsd-hardware@freebsd.org Subject: Re: Big Problem X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Aug 2004 16:07:25 -0000 I had a similar problem with a database environment a while ago. Throughput on the DB (5-6TB) was aweful due to some really poor coding, so we needed options to speed up the indexes and such. This was on a 16way Itanium2 Unisys box attached to EMC 8830 storage to give people an idea how bad the DB and the queries were set up. We eventually scrapped the project due to overruns in cost, time, and efficiency. The only real solution we could come up with was a product from Texas Memory Systems called TerraRAMSan. It's a bit spendy and power hungry, but allows you to have ram based LUNs addressable via the SAN. Stripe 12 or so of these together into a volume and you should be able to get some performance gain. http://www.superssd.com/products/tera-ramsan/ On Wed, 04 Aug 2004 19:34:31 +0800, Kathy Quinlan wrote: > Hi Guys and Gals, > > First off I am not a troll, this is a serious email. I can not go into > to many fine points as I am bound by an NDA. > > The problem: > > I need to hold a text file in ram, the text file in the forseable future > could be up to 10TB in size. > > My Options: > > Design a computer (probably multiple AMD 64's) to handle 10TB of memory > (+ a few extra Gb of ram for system overhead) and hold the file in one > physical computer system. > > Build a server farm and have each server hold a portion eg 4GB each > Server (250 servers (plus a few extra for system overhead) > > The reason the file needs to be in ram is that I need speed of search > for paterns in the data (less than 1 second to pull out relevent chunks) > > I am sure I have missed some options, right now I am just kicking ideas > around, the software will be based on FreeBSD with some major > modifications to address the large amount of ram (probably set it up as > a virtual drive with one file) > > Regards, > > Kat. > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" >