From owner-freebsd-questions@freebsd.org Tue Feb 2 03:04:21 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 086464FAFE2 for ; Tue, 2 Feb 2021 03:04:21 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.134]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DV8mM4x1dz3tSb for ; Tue, 2 Feb 2021 03:04:19 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from r56.edvax.de ([178.5.95.157]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPA (Nemesis) id 1Mdva2-1lhURl2jKA-00b6It; Tue, 02 Feb 2021 04:04:15 +0100 Date: Tue, 2 Feb 2021 04:04:14 +0100 From: Polytropon To: "Matt Emmerton" Cc: Subject: Re: Help recovering damaged drive - fsck segfaults, read-only mount looks ok Message-Id: <20210202040414.d0227107.freebsd@edvax.de> In-Reply-To: <016b01d6f8f3$d8853c00$898fb400$@gsicomp.on.ca> References: <012a01d6f81e$3103d390$930b7ab0$@gsicomp.on.ca> <20210201155842.1e529018.freebsd@edvax.de> <016b01d6f8f3$d8853c00$898fb400$@gsicomp.on.ca> Reply-To: Polytropon Organization: EDVAX X-Mailer: Sylpheed 3.1.1 (GTK+ 2.24.5; i386-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:98DkwH3Qy48CktsarZcUiSO2cWL1dsHNs+bKuj7g2PiJ52vxISG y20QKABI3EIc7/4SSOEU8vUHI0cCQlPTJuSgQ/BdrZ35VDn7DKsv4hOstXD0/RgtdLHZyAA /Fg9vtD4UuIG4Ri6dzWFyqE2USDhtNHapLWW/PGapBpPxBR9UHk70Hk6qzYZfwrebdMfEKQ 4NV9aiiKq4Awf/vdXECiA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:LX4HJJJZgw4=:54ayKZXbyOjC1nz70jyzpH MSpVBlDg0FKeJS0d+A5J9xgWRm388R+WwXJj9pCmR4mtDlVms+2KB5+tpKNz6i6LVM2jsxNTv xl1sTRw36IaYHaPgLpTUgv2MvxLzigMKO4bZbSuJccLgp1kQeMWY4EwbQvuxyrIdPODRYp8t/ fmVvhTq5kZ2jPsKZ7AsRQDxQDZxGFurIX9fAqGb1nr+608gfCNQ2FWXRWjYWOQqxG/NHBJmNH wSlcPAaq+SzGd6qRSuxsWyPTzkKwlnM/e1eCzXOU0rcXR6NwpsbwfLb77Wb0YEKn+0g4RQJji K1iTQYy8j9uD30PGE4HKhwvkB3VkNCEDk9ngl+s/X6fXNCYTZgcoX/bYCiZZKDcd1wa8r+RlA BK+Ne8qeNFwQu9lkDTk0w+ip5A97MeT1oa30qJQpOeTWNkDnrSvtuQ834+nAX X-Rspamd-Queue-Id: 4DV8mM4x1dz3tSb X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd@edvax.de has no SPF policy when checking 212.227.126.134) smtp.mailfrom=freebsd@edvax.de X-Spamd-Result: default: False [-0.60 / 15.00]; HAS_REPLYTO(0.00)[freebsd@edvax.de]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; HAS_ORG_HEADER(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; RECEIVED_SPAMHAUS_PBL(0.00)[178.5.95.157:received]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[212.227.126.134:from]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[edvax.de]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[212.227.126.134:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_CONTAINS_FROM(1.00)[]; RCVD_IN_DNSWL_NONE(0.00)[212.227.126.134:from]; R_SPF_NA(0.00)[no SPF record]; RWL_MAILSPIKE_POSSIBLE(0.00)[212.227.126.134:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 03:04:21 -0000 On Mon, 1 Feb 2021 18:42:04 -0500, Matt Emmerton wrote: > On Sun, 31 Jan 2021 17:12:40 -0500, Matt Emmerton wrote: > > > > If I force a mount in readonly mode, I can inspect the drive and at > > > first glance, everything seems valid. Since this machine is used for > > > backups, I have lots of other medata (eg, checksums) and I'm slowly > > > working through to see if anything important is damaged. > > > > At this point: STOP. > > If your data is important to you, get a copy of it NOW. > > Yep, I've been down this road before, which is why I have backups in the > first place. Still working through which is the "best" place to seed my new > backups from - the backup system (maybe damaged) or the source system (still > healthy). Probably the source system. In worst case you can compare (at least via checksums) to see if everything you have is what you expect it to be. Nothing is worse than assuming you got your file back - same location as before, same name, same size, just discovering that is has been filled with \0 bytes. That happened to me a few times on a buggy system that froze. Luckily, with "grepdisk" I was able to find the expected data "elsewhere" on the disk... :-) > The thing is - I don't care about the deleted files. What I care about is > the fact that trying to "fix" the partially deleted directory tree is > sending fsck down paths which might not be right, which has me worried. So that is not a problem - any unallocated stuff will be in lost+found/ and can then be deleted from there (if fsck doesn't already perform the pending removal). Everything else should go unaffected, except - and that's the thing where guessing, hoping and maybe praying starts - some other aspect of the filesystem has been damaged. The case you're seeing that fsck segfaults is a reminder that there is a severe (!) problem with the filesystem, significant in a way that even fsck cannot repair it. In my (probably different) case that problem was a directory called .snap that stopped fsck from doing its job. I don't know why or how, but after I removed it forcedly (on the still unclean filesystem, that is), fsck would run two times, and everything was back to almost-normal. > Preen mode doesn't do anything useful (basically exits immediately). > Force mode sounds scary. And I agree that fsck -y might not do the right > thing. The -f flag is especially used for the case when you want to check a filesystem that fsck assumes to be clean and therefore does not perform the check. The -y flag will answer "YES" to all questions, even if "NO" would have been your choice for a particular case, e. g., salvaging files, removing entries, filling files at expected size with \0 bytes, or truncating files to zero size. > > And _that_ is how I finally got my files back (the initial "severe data > > loss problem more than 10 years ago): With ls -i, I determined the > > inode of an offending directory, then used fsdb (which I found out > > about reading a reference manual about a GDR UNIX system) to > > remove it, and _then_ (!) fsck was able, after two runs, to bring > > the filesystem back to a consistent state. > > Ahah! fsdb! That's the tool I remember poking around with a long time ago. Yes, this is the tool to "delete on an unclean filesystem" so that fsck will then pick up the deletion and clean up what's remaining. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...