From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 16 22:53:50 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8B02216A4CE; Thu, 16 Sep 2004 22:53:50 +0000 (GMT) Received: from VARK.homeunix.com (SYDNEYPACIFIC-FOUR-EIGHTY-SIX.MIT.EDU [18.95.6.231]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D9FC43D3F; Thu, 16 Sep 2004 22:53:50 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.homeunix.com (localhost [127.0.0.1]) by VARK.homeunix.com (8.13.1/8.12.10) with ESMTP id i8GMrn04001119; Thu, 16 Sep 2004 18:53:49 -0400 (EDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.homeunix.com (8.13.1/8.12.10/Submit) id i8GMrnUq001118; Thu, 16 Sep 2004 18:53:49 -0400 (EDT) (envelope-from das@FreeBSD.ORG) Date: Thu, 16 Sep 2004 18:53:49 -0400 From: David Schultz To: Kris Kennaway Message-ID: <20040916225349.GA892@VARK.homeunix.com> Mail-Followup-To: Kris Kennaway , Sam , freebsd-hackers@FreeBSD.ORG References: <20040916211837.GE70401@hub.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040916211837.GE70401@hub.freebsd.org> cc: freebsd-hackers@FreeBSD.ORG cc: Sam Subject: Re: ZFS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2004 22:53:50 -0000 > On Thu, Sep 16, 2004 at 10:31:57AM -0500, Sam wrote: > > Let's suppose you generate an exabyte of storage per year. Filling a > > 64-bit filesystem would take you approximately 8 million years. I suggest that you review your calculations. > > I'm not saying we'll never get there, [...] > > It's a_single filesystem_. If you want another 8192 ZB, just make another. A goal for ZFS is to eliminate that kind of nonsense. On Thu, Sep 16, 2004, Kris Kennaway wrote: > The detectors in the particle accelerator at Fermilab produce raw data > at a rate of 100 TB/sec (yes, 100 terabytes per second). They have to > use a three-tiered system of hardware filters to throw away most of > this and try to pick out the events that might actually be > interesting, to get it down to a "slow" data rate of 100 MB/sec that > can actually be written out to storage. If the hardware and software > was up to it I'm sure they'd want to keep much more of the data than > this. > > Now, over a year of runtime, the raw data amounts to (according to > Google Calculator): > > (100 (terabytes / sec)) * 1 year = 3.4697207 10^21 bytes > > or just over 2^71 bytes in a year. A UC Berkeley study has some interesting statistics on total storage sold per year, including a breakdown by medium: http://www.sims.berkeley.edu/research/projects/how-much-info-2003/printable_magnetic.pdf They place the total storage sold in 2003 at 2^68 bytes and the amount of original data produced at 2^62 bytes.