From owner-freebsd-questions@FreeBSD.ORG Tue Aug 12 19:34:24 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9FFE9BF9 for ; Tue, 12 Aug 2014 19:34:24 +0000 (UTC) Received: from smtp-vbr12.xs4all.nl (smtp-vbr12.xs4all.nl [194.109.24.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37103258B for ; Tue, 12 Aug 2014 19:34:23 +0000 (UTC) Received: from slackbox.erewhon.home (slackbox.xs4all.nl [83.162.243.5]) by smtp-vbr12.xs4all.nl (8.13.8/8.13.8) with ESMTP id s7CJYJqw052238; Tue, 12 Aug 2014 21:34:19 +0200 (CEST) (envelope-from rsmith@xs4all.nl) Received: by slackbox.erewhon.home (Postfix, from userid 1001) id ABD9C12611; Tue, 12 Aug 2014 21:34:19 +0200 (CEST) Date: Tue, 12 Aug 2014 21:34:19 +0200 From: Roland Smith To: Andrew Hamilton-Wright Subject: Re: Problems with dump and restore Message-ID: <20140812193419.GB7166@slackbox.erewhon.home> Mail-Followup-To: Andrew Hamilton-Wright , freebsd-questions@freebsd.org References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Yylu36WmvOXNoKYn" Content-Disposition: inline In-Reply-To: X-GPG-Fingerprint: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 X-GPG-Key: http://www.xs4all.nl/~rsmith/pubkey.txt X-GPG-Notice: If this message is not signed, don't assume I sent it! User-Agent: Mutt/1.5.23 (2014-03-12) X-Virus-Scanned: by XS4ALL Virus Scanner Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 19:34:24 -0000 --Yylu36WmvOXNoKYn Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 12, 2014 at 01:07:06PM -0300, Andrew Hamilton-Wright wrote: >=20 > I was attempting to restore my /usr partition today, and have encountered > some rather terrifying issues using restore. >=20 >=20 > Some background ... >=20 > I have used dump/restore for several years, very happily, to maintain > backups on my machine. >=20 > I have a level 0 dump of each file system, and then a cron-based script > that does higher level dumps on a regular basis. I therefore have dumps > at the following levels for this filesystem at the moment: 0, 2, 3, 5 >=20 > These were created using snapshots, so the level 0 was created via > dump 0uLCf 32 - /usr > and higher level dumps were created similarly. In 2011, a problem was found with snapshots in combination with soft updates *and* journaling (SU+J) hanging the machine. At that time the recommendation was to switch off journaling. According to https://wiki.freebsd.org/NewFAQs: If you want to use snapshot (dump -L) then disable the soft updates journal for that filesystem. This bug was fixed toward the end of 2011; https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D160662 Personally I make dumps *only* from filesystems that are unmounted or mount= ed read-only, so never from a =E2=80=9Clive=E2=80=9D filesystem, just to be on= the safe side. > My uname info is: > FreeBSD qemg.org 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8= 06:37:44 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/= sys/GENERIC amd64 > I wanted to restore the /usr partition to the state it was in at the last > (level 5) backup. My expected steps to achieve this are: > o go to single user (I did this through a full reboot) > o create a replacement filesystem on the drive: > newfs -O 2 -U -a 4 -b 32768 -d 32768 -e 4096 -f 4096 \ > -g 16384 -h 64 -i 8192 -k 0 -m 8 -o time \ > -s 415236096 /dev/ada0e > o mount the drive as /usr, and change directory to the mount point > o restore the level 0 dump > restore ruf /backup/dumps/current/usr.dump > * this is the first sign of trouble, as restore output the warning > expected next file 19266003, got 19100935 This is mentioned in the restore's manpage; expected next file , got A file that was not listed in the directory showed up. This c= an occur when using a dump created on an active file system. > o restore the level 2 dump > restore ruf /backup/dumps/current/l1d0/l2d0/usr.dump > * this failed, indicating that the restore was corrupt (unfortunately > I do not have the full text of the errors received, but a complaint > that an entry was "not a leaf" was in the first message) >=20 > Frankly, this terrifies me. If dump and restore cannot be trusted > as a robust backup solution, I don't know where to turn to. >=20 > Some questions then: > - is anyone else using dump/restore as their main backup method? Yes, operating system filesystems like /, /usr and /var, which can contain flags and hard links and such. These filesystem's aren't all that big, so dumps are relatively quick. For my large /home filesystem I rather use rsync, because it copies less and so is much faster. > Are you using snapshots? No, because of the aforementioned bug that surfaced in 2011. > If so, have you seen anything like this when running restore? I've had hangs and corrupted dumps when dumping live filesystems. > - is there any means of validating the dump file, other than the -N > option (which returns no warnings on any of these files)? Not that I know of. I generally make and verify checksums when copying dumps to other machines or external harddrives. > - does anyone have any advice that may help determine what may have > gone wrong? Try using restore's =E2=80=9Cdegraded=E2=80=9D mode (using the =E2=80=98-D= =E2=80=99 option) and use the =E2=80=98-y=E2=80=99 option. Roland --=20 R.F.Smith http://rsmith.home.xs4all.nl/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 5753 3324 1661 B0FE 8D93 FCED 40F6 D5DC A38A 33E0 (keyID: A38A33E0) --Yylu36WmvOXNoKYn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJT6mw7AAoJEED21dyjijPgeZcP/jF3rLJRzZANnUMJqn28pg7K YR7vJXjfAOUf3ISLziZwMr+bFrgnzx5WtmMQmXmg9kRjGNnv8OaD3tXrjWSz71Hs VJRX4Y4Jt1YNTjSiwSrIB4/XwaGZiX3G7CjhVLGstzSJWXF9cMn/JYRXig8j8kNS gKD894Lx4ghP9EJKfWshkSOTSX714a4bg7eRRlBKhRgXPfFpwJ7oCQn3aeb4Fhyk csYFoG2Ez8Puk6wc/ZygmBLIv+86G26TvGYYk+V4ei3M6UjAznqOzmXGE3vK5o6P +DHgB+QaMA4olqChMZy+7/q2TyfEFDmEZvaOvQX9wFzn2oft/WYVhTm87bhSB5Ug m08wvYoS2/Z4QkD6AdSQjkqmAk2SJPAu8/tJ+m0wrW2iSkmjnBY8KjAYiGFDkKgh A7/G2LKJ19SYMhg8lbMkRG0IlbLlSOzSvrAQbkJyFKb9opE6V0z9N32D9qCIpp3a PW6XkmpiNC9x3BiaYKJXkky1t7kETLe6UuSaEtjn87w7jto7riyQqJ7Dz0XJ9jhy 0ncyCerE2OpPk2V0KZ2D17hi/jh6pi3/zLSbOWy/9WUcEn3Al6iNkwhfDq0nbH0l 2n8oV4idFkxwNbIXySU5QtJBF7PS+dXbQYP4va7Q9BgqqDSxZKGMvgkZCBPWt04w LQM0g3QhZn7eFUt0dyQL =vw+R -----END PGP SIGNATURE----- --Yylu36WmvOXNoKYn--