Date: Fri, 23 Nov 2001 10:36:40 -0800 From: "Ted Mittelstaedt" <tedm@toybox.placo.com> To: "Robert Suetterlin" <robert@mpe.mpg.de> Cc: <freebsd-questions@FreeBSD.ORG> Subject: RE: why are You asking here Message-ID: <000001c1744d$ca806340$1401a8c0@tedm.placo.com> In-Reply-To: <20011123175912.B1170@robert2.mpe-garching.mpg.de>
next in thread | previous in thread | raw e-mail | index | archive | help
>-----Original Message----- >From: owner-freebsd-questions@FreeBSD.ORG >[mailto:owner-freebsd-questions@FreeBSD.ORG]On Behalf Of Robert >Suetterlin >Sent: Friday, November 23, 2001 8:59 AM >To: Ted Mittelstaedt >Cc: freebsd-questions@FreeBSD.ORG >Subject: Re: why are You asking here > >> >> 1) The power consumption and all that, while it's important, is not >> something that FreeBSD can solve for you. This is purely a hardware >> issue. Now, if you have some hardware already in mind that you think >> can do it and your just looking for an OS to run it - well that's >> something else, tell us what it is. If not - well I think your first >> stop is the hardware mailing lists. >Yes. No. Perhaps. >Good idea that You mentioned the hardware page... >Still the hardwarepage is not the first stop if the application that >solves my problem is bound to a specific operating system, which then >turns out to be bound to some specific hardware. > it's a lot easier to modify software to work with existing hardware instead of modifying hardware to work with existing software. Find the hardware first then worry about working out drivers for it later. > >> I am not at all convinced that tape storage is any kind of answer for >> you. Besides the fragility of the medium, there's the fragility of the >> recording mechanism itself. All tape drives, being mechanical things, >> will wear and drift out of alignment. I simply cannot imagine any kind >> of data tape drive manufactured today lasting for 5-10 years with any >> reasonable amount of use, [...] >1) Well why tape storage? Did I indicate tapes in my mail? yes you did > Sorry for >beeing so specific, I do not want to specify 'tapes' already. >2) Our datacenter is using tapes for data storage and so far (under very >heavy use over a very long time) everything is still working. Of course >we must plan for wear of the machinery. And we must have a solution >that can be serviced by astronauts / kosmonauts easily. > what hardware are they already using and how much data is it storing? Maybe someone can write a FreeBSD driver for it. > >> 3) What exactly is a HSM going to do for you? >HSM is useful if You have a hirarchy of storage with different >qualities. So it will help in any application where You apply several >storage solutions that have differnt qualities --- i.e. density, price, >robustness, mobility, bandwidth, etc. --- to specific problems. >Like Ram, Flash >Memory, Static Memory, Harddrives, Tapes, DVDs, what do I know. > That's what I said. You indicated that your doing archival only - ie: "one quality" in your terms. Is this not true? >Our experiment should run 24/7 if it would be possible. But at the >least it should be possible to run it continously for 45 Minutes at peak >data rates. The total experiment time is estimated to at least 3 hours >per week. This means aprox. 150TB of data per year. If there is a >shuttle mission each month, they will have to transport 10TB of data >each time. > Speaking in GByte Flashcards that is $5.000.000 in >Flashmemory every two >months. Ok we could recycle some of that memory... And Flashmemory >will also get severe problems with hard radiation. Redundancy will >increase price by aprox. 30%. So we talk about $6.5M > If we will get larger data volumes in the future (because >experimentalists always want higher data rates and longer experiment >time) then prices will scale accordingly. So then have someone design an optical solution for you. > Speaking of a Solid State Solution. First I cannot guess easily >the price of 10TB Static Ram. Second, if You compare the size of a Ram >Chip to a 100GByte Tape, I would say overall datadensity is similar. Of >course You can get Memory much denser if You remove Packaging but not by >an order of magnitude. Of course using a solid state storage to >store for example 45 minutes of data could be very reasonable. > Harddrives are a problem, because they disturb microgravity. As >will all rotating objects, because of the enormous torque they can >generate. > >So what can an HSM do for us? I guess it depends on the capabilities of >the HSM. I believe, that our system will need a hirarchy of storage >systems, specialised in specific tasks. Why? What will the experiment be doing? All you have indicated is archival, there's no varying data quality need here. >I also would like to use the >same general technique today that we will launch in five years (and then >use for up to ten more). So I would like to use standard hardware and >software solutions. These will most likely adept to the future >automatically. Any standard solution will be hopelessly obsolete 10 years from now and getting parts for it will cost astronomically. > An HSM would be an important building block in my >concept of a solution to the data storage problem, as it relieves me of >all the problems connected with hirarchical storage management. No, it relieves you of having to redesign a storage solution sometime in the future when you START needing HSM. In short your trying to double-dip and get a two-fer, you want to pay for tomorrow's data storage solution needs today. > This does of course not prove that I need HSM at all. I might >have a complete misconception of how to handle this amount of data at >all. > I think that your grossly underestimating the technical problems of handling 10TB of data a month in a 494x435x248mm space, with power of 200W (at 28VDC). Consider if a shuttle is delayed due to weather that's not leaving you any overhead. You could end up with a need for 50TB of data if a long delay happened, and that's with no redundancy. There's another thing I think your missing here. The Space Program and even the ISS is still mainly funded by US Taxpayer dollars. Well, we are funding it for the express reason of GENERATING new technical advances due to the specialized needs of space exploration. You have an application here that's crying out for a custom-built high-capacity data storage device that takes small space and little power and your screwing around looking for an off the shelf HSM array that will do it. Such an animal doesen't exist. You need to take that R&D budget you have and put it towards R&D with one of the major technical players - perhaps IBM - that can supplement your budget. Of course your going to end up with a custom-built solution but that's the entire point of the exercise as then such solution can be translated into a real product. Instead of attempting to get some sort of standard now, what you need to do is trust in the Capatalistic system to use whatever work your R&D budget creates and make commercial products out of it. Then 10 years from now such high-capacity arrays will cost $100 and be available from Costco. Attempting to use off-the-shelf products now does not help anybody realize any R&D leverage from your budget - which is the entire point of the money spent on ISS. The dollars are available from the hardware manufacturers. Your project would have high marketing value to them. Leverage that value! Ted Mittelstaedt tedm@toybox.placo.com Author of: The FreeBSD Corporate Networker's Guide Book website: http://www.freebsd-corp-net-guide.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?000001c1744d$ca806340$1401a8c0>