Date: Tue, 07 Oct 2008 02:57:24 +0300 From: Evren Yurtesen <yurtesen@ispro.net.tr> To: Jeremy Chadwick <koitsu@FreeBSD.org> Cc: questions@freebsd.org Subject: Re: continuous backup solution for freebsd? Message-ID: <48EAA5E4.8030805@ispro.net.tr> In-Reply-To: <20081006175940.GB26368@icarus.home.lan> References: <48E9E146.9040308@ispro.net.tr> <20081006112030.GA18670@icarus.home.lan> <48EA2270.2080405@ispro.net.tr> <20081006152430.GA23608@icarus.home.lan> <48EA45D2.70502@ispro.net.tr> <20081006175940.GB26368@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeremy Chadwick wrote: > What I'm saying is that Linux has the upper hand here. More eyes, more > people, more developers, larger community, larger vendor support, and > much **much** faster turn-around time on fixes/bugs. We can sit here > and argue about those facts all we want (it's the equivalent of doing > burn-outs in an AMC Pacer in a parking lot -- wasted time, zero gain), > but nothing changes the facts. Sorry, I had to remove the whole bunch of text that you wrote :) but I get the point. I think it is a funny historical fact that BSD was commercially licensed way too long to allow Linux to be developed at first place. If BSD was not commercial at that times, Linus Torvalds probably wouldnt have started writing the Linux kernel. Thus we wouldnt be having this sort of conversation now and it might as well be that Microsoft wouldnt have become so huge. If we look at this from that point of view then eventually all BSD and Linux etc. are bound to disappear in time and Microsoft will stand all alone. But things can change one step at a time. I prefer(or try) to look at this positively. I thought it wouldnt hurt to ask for help if somebody could contact r1soft and perhaps ask a pile of money to develop a driver. It would have been a win-win situation eh? > Right. We're definitely talking about snapshots, at least in concept. > > The fact that you're able to restore data within *minutes* is pretty > impressive. I'm curious what sort of disk requirements are needed > though (I guess it depends on how often changes happen on the > filesystem). Well it is not so fine grained (5 to 10 minutes intervals as mentioned). http://www.r1soft.com/CDP.html (there is more information in the link above, with links to outside sources on the concept such as wikipedia articles etc.) I know some large hosters who use this technology with Linux servers. As a matter of fact the only reason they went with Linux instead of FreeBSD is because they cant get CDP with FreeBSD. I can ask how much space it is using and return back to you. But if you think about it for a second, a traditional backup program would copy the whole file even if there was 1 byte changed in it. Lets say 10mbyte file and 1 byte is changed. R1soft copies only 1 byte. Sure enough the tables can turn around if the filesystem was modified really a lot. But it looks like this type of solution is mostly effective (at least I didnt see anywhere that anybody is complaining that it is using too much disk space yet). The best is, all it would take for FreeBSD users to be able to utilize this technology is a driver to interface with r1soft agent and buy a license. Now I am not expecting anybody to write this for free or nobody is obligated to help. I just dont know anybody who can help so I thought I would drop in a line here so... > I for one have never correlated snapshots and backup restorations > (bare-metal recovery). I consider them completely separate things, and > handled *very* differently. I have a feeling that no one's done this on > FreeBSD because the amount of effort required is quite large. Someone > did mention HAMMER on DragonflyBSD, but I have no knowledge of it or > what it provides -- that said, Matt (Dillon)'s stuff is usually very, > very good. I also dont know much about HAMMER either. But it doesnt look like it will make mainstream usage anytime soon on FreeBSD if it ever does. Actually I found a nice document here: http://www.dragonflybsd.org/hammer/hammer.pdf http://www.dragonflybsd.org/hammer/index.shtml > It depends on how the filesystem is done. For example, with UFS2+SU > snapshots, snapshot generation can take literally hours: completely > unreasonable. While with ZFS, snapshot generation usually takes 2-3 > seconds -- even on massive changes (e.g. take a snapshot, then rm a > 600MB ISO image, then compare present vs. snapshot -- the diff is > something like 40KBytes). Yes, but r1soft backup can restore a single file at a consistent state without restoring the whole filesystem from a graphical user interface and can restore mysql databases at a table level. While I agree that there might be different solutions that I dont know about, it just takes a driver to get this functionality on current FreeBSD systems without everybody to change to ZFS or HAMMER. One has to think, would people change their filesystems or install a driver? :) I would rather pay license fee to a backup program and use the driver. The price of the software is very well justified if I can return back to 5min before in my backups. The data I might loose is much more expensive. >>> I'm sorry for sounding anti-FreeBSD, but the reality is that people >>> should use whatever solutions work best for them -- if that's using >>> Windows, Solaris, or Linux, great! Remember that open-source is about >>> choice: and choice means supporting the possibility that someone chooses >>> something else. Blind one-sided advocacy is very damaging to the >>> open-source model and concept. >> I agree, and please dont shoot the messenger :) I just have a bunch of >> customers who would use FreeBSD but not using only because of this >> problem. In addition to that I myself would like to use near continuous >> backups as well. > > Understood. I now realise the full importance of what it is you've > described, and what R1Soft has developed. Thank you for taking the time > to educate me -- I appreciate it! There are other companies with similar products for Linux ( you can search for continuous backup in google to find ) . However R1Soft is the most active one. Besides the possibility of continous backup, they have been partnering up with hosting control panel software vendors, for example they support cPanel or H-Sphere where the users can restore their own content from backups etc. Since most of the servers on the internet are used for such services, it makes much sense. While there are other backup or backup like solutions ranging from HAMMER and ZFS built in features or Bacula etc. as far as I know these do not allow such flexibility. >> I was just trying to inform the FreeBSD community here so if somebody >> can have some time to divert to giving the right advices to r1soft then >> we all could benefit from it. It doesnt even have to be free even, with >> a reasonable price they can probably hire somebody to work for building >> the basics of this feature. >> >> So the real question is, is there anybody who is willing and have the >> experience to help on this issue? > > The response you're going to get is: "why don't you state how much > you're willing to pay for this feature, so that anyone who IS interested > can decide if that amount of money is worth the time required?" It is a possibility but then I am not working for R1Soft. People who has the knowledge to be able to pull this off can ask this to R1Soft. I am probably reporting a job possibility to an idle developer if there is such thing as an idle developer. R1soft is even willing to develop this themselves as far as I can tell. Maybe they just need the right information to be pointed out to them to start development only. > My response is different: this sort of thing should definitely be pawned > off onto the FreeBSD Foundation. IMHO, this is the sort of thing the > Foundation *should* be handling. There is money there, and this sounds > like a project which could benefit FreeBSD as a whole. It's possible > that R1Soft, if paid, would take up such a challenge, assuming some key > folks (like Kirk McKusick; not volunteering him, just saying he has > experience with filesystems) could help with the development process and > learning curve. I can't speak for the Foundation, but it really sounds > like that's the way to go with this. I don't think you'll get any > responses from interested parties on freebsd-questions. :-) > Thanks, I like your response :) But I dont know anybody in FreeBSD Foundation or anybody who can help. That is exactly why I came to here to mention the issue. If somebody knows somebody who can help then please pass along the word. I am in no way saying that anybody is obligated to do anything. Just informing that this great technology exists but FreeBSD users are not able to use it yet because of development issues. My personal opinion is that backups are an important issue for a serious company and it is a key selection criteria when choosing components and design for a system. I might be right or wrong of course. I just got pissed off enough that lately people I work with just wouldnt use FreeBSD only because of this problem... :( if you check r1soft forums, you can also see that I was bugging r1soft about this issue a lot and they ended up saying that it is probably not possible in FreeBSD to create such kernel module. At least I feel satisfied that I was doing or trying to do something in my skill levels to help to FreeBSD community and I am happy that some people are listening. But it is obvious that I dont know the right people to directly contact and inform about these stuff so please go ahead and forward this information to whomever you feel can help. Thanks, Evren
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48EAA5E4.8030805>