From owner-freebsd-bugs@FreeBSD.ORG Tue May 2 19:11:02 2006 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FE9E16A82E for ; Tue, 2 May 2006 19:11:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id F1C9C43D7B for ; Tue, 2 May 2006 19:10:09 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k42JA9T2080066 for ; Tue, 2 May 2006 19:10:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k42JA9xf080065; Tue, 2 May 2006 19:10:09 GMT (envelope-from gnats) Resent-Date: Tue, 2 May 2006 19:10:09 GMT Resent-Message-Id: <200605021910.k42JA9xf080065@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Ulrich Spoerlein Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DFC4216A924 for ; Tue, 2 May 2006 19:06:43 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from nproxy.gmail.com (nproxy.gmail.com [64.233.182.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28FB543D77 for ; Tue, 2 May 2006 19:06:21 +0000 (GMT) (envelope-from uspoerlein@gmail.com) Received: by nproxy.gmail.com with SMTP id m18so714640nfc for ; Tue, 02 May 2006 12:06:21 -0700 (PDT) Received: by 10.49.17.13 with SMTP id u13mr777747nfi; Tue, 02 May 2006 11:59:59 -0700 (PDT) Received: from roadrunner.q.local ( [217.185.118.14]) by mx.gmail.com with ESMTP id l22sm279626nfc.2006.05.02.11.59.42; Tue, 02 May 2006 11:59:59 -0700 (PDT) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.6/8.13.6) with ESMTP id k42IwokP007779; Tue, 2 May 2006 20:58:50 +0200 (CEST) (envelope-from q@roadrunner.q.local) Received: (from q@localhost) by roadrunner.q.local (8.13.6/8.13.6/Submit) id k42H8wGW007281; Tue, 2 May 2006 19:08:58 +0200 (CEST) (envelope-from q) Message-Id: <200605021708.k42H8wGW007281@roadrunner.q.local> Date: Tue, 2 May 2006 19:08:58 +0200 (CEST) From: Ulrich Spoerlein To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: phk@FreeBSD.org Subject: bin/96677: Improvements for src/tools/tools/recoverdisk X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ulrich Spoerlein List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 19:11:02 -0000 >Number: 96677 >Category: bin >Synopsis: Improvements for src/tools/tools/recoverdisk >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: update >Submitter-Id: current-users >Arrival-Date: Tue May 02 19:10:09 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Ulrich Spoerlein >Release: FreeBSD 6.1-RC i386 >Organization: >Environment: >Description: Already sent this to current@ for review. Only positive feedback so far. I also whipped up a preliminary man-page, though it would still require some work. I'm not sure about the 128kB DMA reads and my mdoc-fu is probably rather lousy. >How-To-Repeat: >Fix: --- recoverdisk.diff begins here --- --- recoverdisk.c.orig Thu Dec 15 19:41:42 2005 +++ recoverdisk.c Tue May 2 18:16:01 2006 @@ -1,4 +1,4 @@ -/* +/*- * ---------------------------------------------------------------------------- * "THE BEER-WARE LICENSE" (Revision 42): * wrote this file. As long as you retain this notice you @@ -14,15 +14,20 @@ #include #include #include +#include +#include +#include #include #include #include #include #include -#define BIGSIZE (1024 * 1024) -#define MEDIUMSIZE (64 * 1024) -#define MINSIZE (512) +#define MIN(a,b) (((a)<(b))?(a):(b)) + +static size_t bigsize = 1024 * 1024; +static size_t medsize = 64 * 1024; +static size_t minsize = 512; struct lump { off_t start; @@ -48,29 +53,120 @@ TAILQ_INSERT_TAIL(&lumps, lp, list); } +static struct lump *lp; +static char *wworklist = NULL; +static char *rworklist = NULL; + +/* Save the worklist, if -w was given */ +static void +save_worklist(__unused int sig) +{ + FILE *file; + + if (wworklist != NULL) { + (void)fprintf(stderr, "\nSaving worklist ..."); + fflush(stderr); + + file = fopen(wworklist, "w"); + if (file == NULL) + err(1, "Error opening file %s", wworklist); + + for (;;) { + lp = TAILQ_FIRST(&lumps); + if (lp == NULL) + break; + fprintf(file, "%jd %jd %d\n", + (intmax_t)lp->start, (intmax_t)lp->len, lp->state); + TAILQ_REMOVE(&lumps, lp, list); + } + (void)fprintf(stderr, " done.\n"); + } + exit(0); +} + +static off_t +read_worklist(off_t t) +{ + off_t s, l, d; + int state, lines; + FILE *file; + + (void)fprintf(stderr, "Reading worklist ..."); + fflush(stderr); + file = fopen(rworklist, "r"); + if (file == NULL) + err(1, "Error opening file %s", rworklist); + + lines = 0; + d = t; + for (;;) { + ++lines; + if (3 != fscanf(file, "%jd %jd %d\n", &s, &l, &state)) { + if (!feof(file)) + err(1, "Error parsing file %s at line %d", + rworklist, lines); + else + break; + } + + new_lump(s, l, state); + d -= l; + } + (void)fprintf(stderr, " done.\n"); + + /* + * Return the number of bytes already read + * (at least not in worklist). + */ + return (d); +} + +static void +usage(void) +{ + (void)fprintf(stderr, + "usage: recoverdisk [-r worklist] [-w worklist] source-drive [destination]\n"); + exit(EX_USAGE); +} + int -main(int argc, const char **argv) +main(int argc, char * const argv[]) { + int ch; int fdr, fdw; - struct lump *lp; - off_t t, d; + off_t t, d; size_t i, j; int error, flags; u_char *buf; - u_int sectorsize, minsize; + u_int sectorsize; time_t t1, t2; struct stat sb; + while ((ch = getopt(argc, argv, "r:w:")) != -1) { + switch (ch) { + case 'w': + wworklist = strdup(optarg); + if (wworklist == NULL) + err(1, "Cannot allocate enough memory"); + break; + case 'r': + rworklist = strdup(optarg); + if (rworklist == NULL) + err(1, "Cannot allocate enough memory"); + break; + default: + usage(); + } + } + argc -= optind; + argv += optind; - if (argc < 2) - errx(1, "Usage: %s source-drive [destination]", argv[0]); + if (argc < 1 || argc > 2) + usage(); - buf = malloc(BIGSIZE); - if (buf == NULL) - err(1, "Cannot allocate %d bytes buffer", BIGSIZE); - fdr = open(argv[1], O_RDONLY); + fdr = open(argv[0], O_RDONLY); if (fdr < 0) - err(1, "Cannot open read descriptor %s", argv[1]); + err(1, "Cannot open read descriptor %s", argv[0]); error = fstat(fdr, &sb); if (error < 0) @@ -80,46 +176,62 @@ error = ioctl(fdr, DIOCGSECTORSIZE, §orsize); if (error < 0) err(1, "DIOCGSECTORSIZE failed"); + + /* + * Make medsize roughly 64kB, depending on native sector + * size. bigsize has to be a multiple of medsize. + * For media with 2352 sectors, this will + * result in 2352, 63504, and 1016064 bytes. + */ minsize = sectorsize; + medsize = (medsize / sectorsize) * sectorsize; + bigsize = medsize * 16; error = ioctl(fdr, DIOCGMEDIASIZE, &t); if (error < 0) err(1, "DIOCGMEDIASIZE failed"); } else { - sectorsize = 1; t = sb.st_size; - minsize = MINSIZE; flags |= O_CREAT | O_TRUNC; } - if (argc > 2) { - fdw = open(argv[2], flags, DEFFILEMODE); + buf = malloc(bigsize); + if (buf == NULL) + err(1, "Cannot allocate %jd bytes buffer", (intmax_t)bigsize); + + if (argc > 1) { + fdw = open(argv[1], flags, DEFFILEMODE); if (fdw < 0) - err(1, "Cannot open write descriptor %s", argv[2]); + err(1, "Cannot open write descriptor %s", argv[1]); } else { fdw = -1; } - new_lump(0, t, 0); - d = 0; + if (rworklist != NULL) { + d = read_worklist(t); + } else { + new_lump(0, t, 0); + d = 0; + } + + signal(SIGINT, save_worklist); t1 = 0; + printf("%13s %7s %13s %5s %13s %13s %9s\n", + "start", "size", "len", "state", "done", "remaining", "% done"); for (;;) { lp = TAILQ_FIRST(&lumps); if (lp == NULL) break; - TAILQ_REMOVE(&lumps, lp, list); while (lp->len > 0) { - i = BIGSIZE; - if (lp->len < BIGSIZE) - i = lp->len; + i = MIN(lp->len, bigsize); if (lp->state == 1) - i = MEDIUMSIZE; + i = MIN(lp->len, medsize); if (lp->state > 1) - i = minsize; + i = MIN(lp->len, minsize); time(&t2); - if (t1 != t2 || lp->len < BIGSIZE) { - printf("\r%13jd %7zu %13jd %3d %13jd %13jd %.8f", + if (t1 != t2 || lp->len < bigsize) { + printf("\r%13jd %7zu %13jd %5d %13jd %13jd %.7f", (intmax_t)lp->start, i, (intmax_t)lp->len, @@ -152,9 +264,9 @@ lp->start += i; lp->len -= i; } + TAILQ_REMOVE(&lumps, lp, list); free(lp); } printf("\nCompleted\n"); - exit (0); + return (0); } - --- recoverdisk.diff ends here --- --- recoverdisk.1 begins here --- .\" Insert license here .\" $FreeBSD$ .\" .Dd May 2, 2006 .Dt RECOVERDISK 1 .Os .Sh NAME .Nm recoverdisk .Nd recover data from hard disk or optical media .Sh SYNOPSIS .Nm .Op Fl r Ar rlist .Op Fl w Ar wlist .Ar special .Op file .Sh DESCRIPTION The .Nm utility reads data from the .Pa special file until all blocks could be successfully read. It starts reading in multiples of the sector size. Whenever a block fails, it is put to the end of the working queue and will be read again, possibly with a smaller read size. .Pp It uses block sizes of roughly 1 MB, 64kB, and the native sector size (usually 512 bytes). These figures are adjusted slightly, for devices whose sectorsize is not a power of 2, e.g., audio CDs with a sector size of 2352 bytes. .Pp The options are as follows: .Bl -tag -width indent .It Fl r Ar rlist Read the list of blocks and block sizes to read from the specified file. .It Fl w Ar wlist Write the list of remaining blocks to read to the specified file if .Nm is aborted via SIGINT. .El .Pp The .Fl r and .Fl w option can be used in combination. Especially, they can point to the same file, which will be updated on abort. .Sh OUTPUT .Nm prints several columns, detailing the progress .Bl -tag -width remaining .It start Starting offset of the current block. .It size Read size of the current block. .It len Length of the current block. .It state Is increased for every failed read. .It done Number of bytes already read. .It remaining Number of bytes remaining. .It % done Percent complete. .El .Sh EXAMPLES # recover data from failing hard drive ad3 .Dl $ touch /data/lots_of_space .Dl $ recoverdisk /dev/ad3 /data/lots_of_space .Pp # clone a hard disk .Dl $ recoverdisk /dev/ad3 /dev/ad4 .Pp # read an ISO image from a CD-ROM .Dl $ touch /data/cd.iso; recoverdisk /dev/acd0 /data/cd.iso .Pp # continue reading from a broken CD and update the existing worklist .Dl $ recoverdisk -r worklist -w worklist /dev/acd0 /data/cd.iso .Sh SEE ALSO .Xr dd 1 .Sh HISTORY The .Nm command first appeared in .Fx 6.2 . .Sh BUGS Reading from media where the sectorsize is not a power of 2 will make all 1 MB reads fail. This is due to the DMA reads being split up into blocks of at most 128kB. XXX I have no clue, verify this! These reads then fail if the sectorsize is not a divisor of 128kB. When reading a full raw audio CD, this leads to roughly 700 error messages flying by. This is harmless. .Sh AUTHORS .An -nosplit The original implementation was done by .An Poul-Henning Kamp Aq phk@freebsd.org with minor improvements from .An Ulrich Sp\(:orlein Aq uspoerlein@gmail.com . --- recoverdisk.1 ends here --- >Release-Note: >Audit-Trail: >Unformatted: