From owner-freebsd-questions@FreeBSD.ORG Thu Feb 4 16:57:37 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BA4E1065694 for ; Thu, 4 Feb 2010 16:57:37 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (gate6.infracaninophile.co.uk [IPv6:2001:8b0:151:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9D8808FC18 for ; Thu, 4 Feb 2010 16:57:36 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.4/8.14.4) with ESMTP id o14GvUxr060077 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 4 Feb 2010 16:57:30 GMT (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp.infracaninophile.co.uk o14GvUxr060077 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1265302650; bh=N12ICvX83TDvq6AXoZ0dlzI5Q2I/MWJDo/LCJfcZKwA=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding:Cc:Content-Type: Date:From:In-Reply-To:Message-ID:Mime-Version:References:To; z=Message-ID:=20<4B6AFC7A.9040200@infracaninophile.co.uk>|Date:=20T hu,=2004=20Feb=202010=2016:57:30=20+0000|From:=20Matthew=20Seaman= 20|Organization:=20Infracaninophi le|User-Agent:=20Mozilla/5.0=20(Macintosh=3B=20U=3B=20Intel=20Mac= 20OS=20X=2010.6=3B=20en-GB=3B=20rv:1.9.1.7)=20Gecko/20100111=20Thu nderbird/3.0.1|MIME-Version:=201.0|To:=20"Svein=20Skogen=20(Listma il=20Account)"=20|CC:=20freebsd-que stions@freebsd.org|Subject:=20Re:=20Backup=20and=20FreeBSD/ZFS|Ref erences:=20<4B694A52.4080306@stillbilde.net>=09<4B69BD02.5060305@g mail.com>=09<4B69CCC8.2060105@stillbilde.net>=09=20<4B6AE93C.5090803@st illbilde.net>|In-Reply-To:=20<4B6AE93C.5090803@stillbilde.net>|X-E nigmail-Version:=201.0|Content-Type:=20text/plain=3B=20charset=3DU TF-8|Content-Transfer-Encoding:=207bit; b=EAN+D8+Cap/uMnOdD3LIRnd6Kn1BUP73620aygmGhXOuRobPporJVlHwsSz2guz8z f35pxYSUJwggtBjD3cRJ2I1NOkiqSKmkq8YvtktkvhIdPOScHutZphplHPAUFPUFAU 1IVH/1ZvQZKwwNRvvCPXwTNhzrjnxYE9optx0ueE= Message-ID: <4B6AFC7A.9040200@infracaninophile.co.uk> Date: Thu, 04 Feb 2010 16:57:30 +0000 From: Matthew Seaman Organization: Infracaninophile User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: "Svein Skogen (Listmail Account)" References: <4B694A52.4080306@stillbilde.net> <4B69BD02.5060305@gmail.com> <4B69CCC8.2060105@stillbilde.net> <4B6AE93C.5090803@stillbilde.net> In-Reply-To: <4B6AE93C.5090803@stillbilde.net> X-Enigmail-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.95.3 at happy-idiot-talk.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VERIFIED,SPF_FAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on happy-idiot-talk.infracaninophile.co.uk Cc: freebsd-questions@freebsd.org Subject: Re: Backup and FreeBSD/ZFS X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2010 16:57:37 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/02/2010 15:35, Svein Skogen (Listmail Account) wrote: > On a monthly rotation the tapes are placed in a firetolerant safe. Since > the most critical thing here is the terabyte (and growing!) of original > photographs, I'm not thinking about just day-to-day diskfailure or > pebcaks (proper raid and snapshotting handles that rather well). However > snapshotting and raid solutions handles the house being on fire rather > poorly, or should we say "Data integrity and fires, get along like a > house on fire"? ;) fire tolerant? That doesn't sound amazingly effective to me. Would it stand up to temperatures in excess of 600degC for more than about 20 minutes? That's going to be fairly typical in a house fire... A safe like that is a good idea for local storage of backup media while it waits to go into the tape library or off-site. It's a bad idea for storing your entire archive. You really want your backups to be stored in an off-site location. Preferably by a company that has the right sort of secure archive facilities. 'Off-site' means 'sufficiently far away that any conceivable disaster can't affect them.' The gold standard for 'conceivable disasters' is a fully laden and fuelled plane crashing onto your premises. I did once have a setup where 'off site' was a storage company only a couple of streets over, but as their archive was in a former World War II Bunker some 100 feet underground, that was acceptable. Usually you'ld be looking at several miles away at minimum. > And I think ... everyone on this list can agree that data not properly > backed up, is a fancy way of saying data not yet lost. ;) > > This is why I'm willing to (and have already) cough up for such > solutions as autoloaders for my home storage server, however my last > wrestle with ZFS (on freebsd RELENG_7) left me rather less than > enthusiastic about the backup options. Tape libraries are horribly expensive since they're not mass market items. They are also intrinsically prone to breaking down or failing to work quite as well as the salesman implied. They're the only viable solution when your storage volumes get really huge, but what is considered huge nowadays is rather more than terabyte scale. If you can get away with just a single tape drive you'll save yourself a lot of money. LTO4 tapes are rated at 800--1600GB depending on achievable compression, so they might be big enough on their own. As image formats are already internally compressed, I'd expect them to come in at the low end of that, which might be tight. Worth trying out if you can get a drive on evaluation. You might want to evaluate getting a bunch of 1TB (or larger) hard dives - -- either USB or hot-swap SATA. They don't need to perform particularly well, but they'd have to be rated for a lot of spin-up/spin-down cycles (so something aimed at the mobile PC market). One other thing you should seriously consider is on-line backup. There are quite a lot of providers out there, and they should be at least competitive with running your own dedicated backup system. They also generally have the advantage of being instantly available if you need to recover anything in a hurry. > Someone told me that Amanda should handle this, and I'm looking into it > now (especially reading up on what I'd need to do to handle disaster > recovery), but other options are welcome as well, including the option > of going Solaris (if someone can point me to proper documentation on how > to get Solaris to do what I want). Also checkout Bacula. I've found Bacula quite a lot easier to manage than Amanda, especially with tape libraries. > The box itself is a C2D E7500 with 8GB ram, Asus P5Q Premium (the > "deluxe" version with fewer NICs is on the BigAdmin HCL, basically an > intel P45 chipset with sufficient number of pci-express slots, and four > Marvell Yukon gigabit nics with Marvell Alaska PHY), backed by LSI > SAS-MPT for the autoloader and SAS-MFI for the disks, and will handle > SMB/CIFS, NFS, and iSCSI services (and the backups of that data). > Nothing fancy here, meaning it should hardwarewise be no biggie to get > it up and running in FreeBSD, Solaris (or leave it on Windows Storage > server if that's the best solution, even if that means the > iSCSI-target-service has ... less than stellar performance). > > So, I'm basically looking for pointers on what solutions to consider, > not looking for a pre-cooked solution. I have sufficient external > diskspace (still with redundancy) to handle the move-to-new-os-and-fs > issue... > > Thanks again for taking the time to help me out here. ;) Hard to know what to advise OS-wise. FreeBSD will do the job, although I'm not sure the iSCSI-target stuff is the best available. So will Solaris for that matter, although more likely to suffer from hardware incompatibilites. I really haven't got a clue about how well Windows would perform although I personally would avoid it simply because it was Windows... Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktq/HoACgkQ8Mjk52CukIzDHQCfZCFh3+b1r1+rlM4gXg/dKLHi 3EwAn1qyfnPzpvObGUvB0MBMXw3glLdi =e3iw -----END PGP SIGNATURE-----