From owner-freebsd-questions@FreeBSD.ORG Thu Jun 9 13:52:24 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B3C7F16A41C for ; Thu, 9 Jun 2005 13:52:24 +0000 (GMT) (envelope-from Frantisek.Rysanek@post.cz) Received: from mail.fccps.cz (mail.fccps.cz [195.146.112.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1B9CE43D4C for ; Thu, 9 Jun 2005 13:52:23 +0000 (GMT) (envelope-from Frantisek.Rysanek@post.cz) Received: from localhost (localhost [127.0.0.1]) by mail.fccps.cz (Postfix) with ESMTP id C3B4277C85; Thu, 9 Jun 2005 15:52:22 +0200 (CEST) Received: from mail.fccps.cz ([127.0.0.1]) by localhost (mail [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 15267-06; Thu, 9 Jun 2005 15:52:21 +0200 (CEST) Received: from frr (frr.in.fccps.cz [192.168.2.14]) by mail.fccps.cz (Postfix) with ESMTP id B85CE77C7C; Thu, 9 Jun 2005 15:52:20 +0200 (CEST) From: "Frantisek Rysanek" To: Alex Zbyslaw Date: Thu, 09 Jun 2005 15:57:57 +0200 MIME-Version: 1.0 Message-ID: <42A86705.29771.9A91FF01@localhost> Priority: normal In-reply-to: <42A801CB.20604@dial.pipex.com> References: <42A7FAD3.5051.98EB5B15@localhost> X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at fccps.cz Cc: freebsd-questions@freebsd.org Subject: Re: Summary: 12TB GEOM stripe, newfs, then fsck: cannot alloc 768053748 bytes for blockmap X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jun 2005 13:52:24 -0000 On 9 Jun 2005 at 9:46, Alex Zbyslaw wrote: > >A local friend has suggested to increase the block size > > > It ought also make sense if you are serving up *large* files (didn't you > say video/audio?). ... > I don't have anything on your scale (23Gb > of document database pales into insignificance against 12Tb :-) ) > I myself have never met that sort of data volume, either. I actually just work for a reseller / assembly shop. We needed to test these three units before shipping them. FreeBSD 5 is the only free UNIX known to me that handles >2TB "out of the box" on a 32bit x86 PC. (I didn't bother to install a 64bit OS on the dual Nocona box used for testing.) While I was at it, I thought I could give GEOM a try. Never used it before. Took me 5 minutes to find the example in the manpage. Then it took two commands in the shell and the block device was up. Unbelievable. Me being a FreeBSD illiterate. I seem to recall that the boxes will be used for medium-term video archival at their final destination, separately. Thanks again for your help :-) Frank Rysanek P.S.: It makes you wonder. A disk's transfer rate grows roughly with the square root of the disk's capacity (== storage density), if other conditions are unchanged (RPM, number of heads etc.) It used to take three minutes to read my first 100MB hard drive. It takes 40 minutes to over 1h to read the current desktop drives - alone, at their respective maximum sustained rate. The RAID controllers have a lower total throughput than the sum of their drives - maybe 150 MBps per unit of 16 drives. Using three separate SCSI busses for the three RAID units, maybe I could crank them up to 400 MBps of sustained transfer rate. That's about 8 hours just to read the whole 12TB thing. Think of sipping a swimming pool with a straw.