From owner-freebsd-stable@FreeBSD.ORG Thu Mar 26 15:46:23 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 087031065674 for ; Thu, 26 Mar 2009 15:46:23 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id C0FDC8FC18 for ; Thu, 26 Mar 2009 15:46:22 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QFkLbc001091 for ; Thu, 26 Mar 2009 09:46:22 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 09:46:22 2009 Message-ID: <49CBA34B.9070708@denninger.net> Date: Thu, 26 Mar 2009 10:46:19 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Jake Scott References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> In-Reply-To: Content-Type: multipart/mixed; boundary="------------030704080901050100070208" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , Peter Schuller , fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:46:24 -0000 This is a multi-part message in MIME format. --------------030704080901050100070208 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Jake Scott wrote: > >> That said there are plenty of other reasos to use proper dump tools >> (data portability, confirming the ability to actually read all rows >> from a table, using a more often exercised code path and perhaps less >> likely to have edge case bugs, etc). > > Absolutely. You really must use a tool that interacts with the > database to perform the backup. Most commercial DBs have hooks that > allow the backup routines to call out to custom snapshot facilities. > One would usually request a backup through the database, which would > then freeze IO to its data files and maybe log files, deal with > flushing caches etc and then call your snapshot routine. I'm not > aware that MySQL and Postgres do though so the best you can do is a dump. > > Jake VERY careful thought has to go into backup strategy with production databases. Hooks that call out and snapshot are not necessarily good enough although they're "necessary" to get a dump that restores without the database going into log-replay mode. It is not difficult to do this with Postgresql; you can quiesce the database, snapshot and then release it, then dump the snapshots. This gives you transaction-complete dumps (as opposed to "crashed and rolled forward" dumps). The latter ("crashed and rolled forward"), if its sufficient, is trivially able to be done by having Postgresql (and most other databases) keep a sufficient number of log segments that a rollover cannot happen during the dump process itself, and either snapshotting both filesystems at once, keeping both on the same filesystem (undesirable for performance reasons) or dumping the database first and XLOG second. However, whether either of these approaches is sufficient is another matter. One of the real problems with live transaction processing systems is a means to know when there is a failure exactly what you lost. This is not a trivial problem to solve and requires plenty of thought before implementation, especially if you cannot afford the outage time necessary to take the snapshots - in some cases even that (relatively) short outage time is unacceptable. -- -- Karl Denninger karl@denninger.net --------------030704080901050100070208--