Date: Wed, 3 Jul 2013 16:36:31 -0700 From: Jeremy Chadwick <jdc@koitsu.org> To: Berend de Boer <berend@pobox.com> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: EBS snapshot backups from a FreeBSD zfs file system: zpool freeze? Message-ID: <20130703233631.GA74698@icarus.home.lan> In-Reply-To: <87k3l748gb.wl%berend@pobox.com> References: <87ehbg5raq.wl%berend@pobox.com> <20130703055047.GA54853@icarus.home.lan> <6488DECC-2455-4E92-B432-C39490D18484@dragondata.com> <CADBaqmihCB5JP01hLwXTWHoZiJJ5-jkT-Ro=oDwOcKZT_zvEKA@mail.gmail.com> <A5A66641-5EF9-454E-A767-009480EE404E@dragondata.com> <871u7g57rl.wl%berend@pobox.com> <op.wznad7th34t2sn@tech304.office.supranet.net> <87mwq34emp.wl%berend@pobox.com> <20130703200241.GB60515@in-addr.com> <87k3l748gb.wl%berend@pobox.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jul 04, 2013 at 09:52:52AM +1200, Berend de Boer wrote: > >>>>> "Gary" == Gary Palmer <gpalmer@freebsd.org> writes: > > Gary> Other than using SAN (FC or iSCSI), I know of no reason to > Gary> do backups at the raw disk level, nor any real demand. > > Probably the hundreds of thousands of businesses that use Amazon AWS > disagree :-) > > > Gary> I've worked with people who have done LUN based backups in > Gary> the past and they have one drawback - they tend to back up > Gary> the entire LUN, irrespective of whether it is an allocated > Gary> block or not. Modern systems that implement some kind of > Gary> TRIM emulation (or cheat and sniff the filesystem block > Gary> allocation maps) may alleviate that problem. > > That's not how EBS does a back up. It only backs up allocated blocks > for the first time, and for subsequent backups only back up the > changed blocks. > > > Gary> However, in the vast majority of cases, people back up from > Gary> above the FS, not below. This makes your use case probably > Gary> more tied to EBS than you may otherwise think. > > People generally didn't have a choice I would say. Now millions of > servers run on top of block storage. > > Disks are just software. That's the new world. I understand what you're trying to say by this statement, but you're stretching it big time. I was opting to stay out of the thread until I saw your last line. It doesn't matter how many layers of I/O abstraction there are, eventually physical hardware for storage is involved. It doesn't matter what type of disk (mechanical vs. solid-state vs. something custom/proprietary) or what type of controller -- it does eventually end up on bare metal. I say this well-aware of the relationship between software and hardware (ex. disk firmware (software) controlling the underlying hardware (drive motor IC, underlying I/O controller, etc.)). The problem with these "software solutions" (cloud, etc.) -- I'm not sure what to call them because it varies -- are many. One of those problems is that there is a great disconnect between the user of the "solution" and the actual bare metal. And quite often the topology -- meaning the actual innards/how it all works/what transpires even on a protocol level -- is never documented or made public to the user. Hell, my experience in the enterprise world shows that quite often even support personnel don't know how it works. Why this matters: when it breaks -- and it will break, believe me -- that information becomes critical/key to troubleshooting and providing a solution. I've even encountered one "enterprise-grade storage solution" where when the product broke (as in all filesystems inaccessible), multiple levels of support engineers had no idea where the actual problem was because of how much abstraction there was between the appliance itself and the bare metal. How many engineers does it take to turn a light bulb? Apparently too many. As politely as I can: It sounds like you may have spent too much time with these types of setups, or believe them to be "magical" in some way, in turn forgetting the realities of bare metal and instead thinking "everything is software". Bzzt. And while generally I don't see eye-to-eye with Richard Stallman, storage **is** the one area where I do: http://www.guardian.co.uk/technology/2008/sep/29/cloud.computing.richard.stallman KISS principle goes a long, long way when applied to storage. And no, I am not saying "get away from this EBS/AWS stuff!" -- I'm simply saying that your statement "disks are software in the new world" is utter nonsense. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130703233631.GA74698>