From owner-freebsd-fs@FreeBSD.ORG Tue May 24 20:12:03 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B77E716A41C for ; Tue, 24 May 2005 20:12:03 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5CC0143D49 for ; Tue, 24 May 2005 20:12:03 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from zappa.Chelsea-Ct.Org (pool-151-199-7-31.ROA.east.verizon.net [151.199.7.31]) by gromit.dlib.vt.edu (8.13.3/8.13.3) with ESMTP id j4OKBvsJ071875 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 24 May 2005 16:12:02 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from zappa.Chelsea-Ct.Org (localhost.Chelsea-Ct.Org [127.0.0.1]) by zappa.Chelsea-Ct.Org (8.13.3/8.13.3) with ESMTP id j4OKBnwo050586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 24 May 2005 16:11:49 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: (from paul@localhost) by zappa.Chelsea-Ct.Org (8.13.3/8.13.3/Submit) id j4OKBm1o050585; Tue, 24 May 2005 16:11:48 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) X-Authentication-Warning: zappa.Chelsea-Ct.Org: paul set sender to paul@gromit.dlib.vt.edu using -f From: Paul Mather To: bv@wjv.com In-Reply-To: <20050524184257.GB84311@wjv.com> References: <1116955638.48224.27.camel@zappa.Chelsea-Ct.Org> <20050524184257.GB84311@wjv.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Tue, 24 May 2005 16:11:48 -0400 Message-Id: <1116965508.50272.24.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Evolution 2.2.2 FreeBSD GNOME Team Port Cc: freebsd-fs@freebsd.org Subject: Re: Recovering UFS2 content via Sleuthkit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2005 20:12:03 -0000 On Tue, 2005-05-24 at 14:42 -0400, Bill Vermillion wrote: > > Has anyone managed successfully to recover deleted file content > > using, say, Sleuthkit, or is deleted UFS2 content recovery > > not feasible (aside from sifting manually with a disk sector > > editor)? > > I've not used Sleuthkit but I've managed to recover files > using TCT - The Coroners Tookkit and the program 'lasarus' in > conjunction with 'unrm'. Thanks for the suggestion. Unfortunately, my desktop system runs FreeBSD 6.0-CURRENT, and I discovered the following when trying to build TCT from ports: ===> tct-1.14 is marked as broken: Does not build on FreeBSD >= 6.x. I do have a 4.11-STABLE system, and did build the TCT port there, mainly to have a look at the documentation. Does unrm understand UFS2? I can only see references to UFS. > > I tried Sleuthkit from the ports collection, and although I can > > find deleted content using it, it's not possible to recover > > that content because too much important information has been > > lost from the inode: specifically, although information like > > the owner and timestamp information appears to be preserved, > > vital data such as the size, direct blocks, etc. are all zeroed, > > rendering the deleted content unreachable (or, rather, reducing > > the problem back to a manual search). > > > So, am I right in thinking that even if the inodes and blocks > > belonging to a deleted file have not yet been reallocated or > > used again, it's still not feasible to recover the deleted > > content easily because of the data loss inflicted upon the > > deleted file's inode(s)? In other words, that the only data > > recovery possible is via manual means (searching for signatures > > and trying to piece together fragments)? > > 'lazarus' has options to build files or build an html structure. > 'unrm' is preferable to use over 'dd' because you use it to > search the disk only for unallocated blocks. > > What you will often find in the recovered files are parts of > two files at times, or only part of one large file. This is because > the 'unrm' goes block by block through the drive, and files are not > neccesarily contiguous. > > When I ran 'unrm' and 'lazarus' after stupidly not checking a > command line [I accidentally remove files like this about every 6 > or 8 years :-( ] I saved the output of the unrm to another > partition, then ran lazarus to parse those pieces found by 'unrm'. > Then when I got the whole segment done, I just moved that entire > tree into my XP machine and used the 'search' utility to look for > strings. Looking at lazarus, it seems like it is trying to semi-automate a manual search/classification of data blocks. Is this a consequence of what I observed originally (i.e., too much vital information is scrubbed in the inode when a file is deleted, so, even if the inode[s] and data blocks have not been reallocated, there's still no direct linkage information for data recovery software to chase)? In my case, I'll probably not bother with lazarus. According to Sleuthkit, the deleted content seems to consist mainly of some PDF files and some multimedia files (FLAC, SHN, AVI, and MPG). Unfortunately, these would seem to be poor candidates for manual inspection recovery, because it's hard to see what is a valid part of the file, and whether everything is in the correct order. With the multimedia files, in particular, these are likely to be large and hence much more likely to be non-contiguously laid out. > Nothing is easy when it comes to recovering lost files. But you > learn more about the system, and more importantly it reinforces > your checking to keep from doing this again - hopefully. In my case, it was a matter of executing the right command in the wrong directory too late at night on too little sleep. ;-) I did manage to ctrl-c the command fairly quickly, when I realised the Awful Truth of what was happening. :-) In retrospect, I should probably have bashed the reset button before SoftUpdates had a chance to start committing things to disk. :-) > > Also, I wonder why some, but not all, information is scrubbed when a > > file becomes deleted (especially information in the inode). > > Probably for performance problems. There has been talk about > buildding an FS that will zero blocks when you remove the indoes, > and in that case you will never recover anything. An 'rm' just > essentially frees up the inodes, and doesn't do a thing with the > on-disk blocks. I can understand not scrubbing the on-disk blocks (as you say, for performance reasons), but it seemed curious that only some parts of the dinode appeared to be scrubbed, because it's all going to be written back to disk anyway (so why not scrub it all?). (Perhaps it's just a sadistic way of mocking those who would attempt data recovery: a sort of "you're close, but not close enough..." [cue echoing mocking laughter, et al.] :-)) > Best of luck on your attempts. 'tis painful and I've done it. > If you run lazarus to output html you may be surprised to find what > else lurrks on your disk that you thought you had gotten rid of. Sleuthkit was very revealing when it came to finding deleted content (and Autopsy makes browsing very convenient). It just seemed no good for actually recovering it, at least as far as I could tell. It does support UFS2, though. Thanks for the help and advice! Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa