Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 11 Apr 2019 06:36:20 +0200
From:      Peter Holm <peter@holm.cc>
To:        Jamie Landeg-Jones <jamie@catflap.org>
Cc:        freebsd-stable@freebsd.org, Kirk McKusick <mckusick@mckusick.com>
Subject:   Re: Replicable file-system corruption due to fsck/ufs
Message-ID:  <20190411043620.GA87473@x2.osted.lan>
In-Reply-To: <201904110347.x3B3lhpA034197@donotpassgo.dyslexicfish.net>
References:  <201904110347.x3B3lhpA034197@donotpassgo.dyslexicfish.net>

next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Apr 11, 2019 at 04:47:43AM +0100, Jamie Landeg-Jones wrote:
> I've noticed a replicable disk corruption by fsck_ufs/ffs on sparse files.
> 
> This is on amd/64 12-stable-20190409, but I first noticed it on
> 12-stable-20190326.
> 
> I didn't notice it on my previous build of 12-stable-20190107, but I
> may not have had any relevant sparse files at the time, so I don't know
> if that version was affected. 12-release worked OK.
> 
> Here is a simplified replicable example. Thinking about it just now, I
> suspect it's triggered by files which end in sparseness.
> 
> Can anyone else replicate this, or has my machine gone nuts?
> 
> Cheers, Jamie
> 
>  | root@thompson# l
>  | total 12
>  | 4 drwxr-x---   2 root  wheel  -   512 11 Apr 04:08 ./
>  | 4 drwxr-xr-x  16 root  wheel  - 1,024 11 Apr 04:08 ../
>  | 4 -rw-r-----   1 root  wheel  -    43 11 Apr 04:08 typescript
>  |
>  | root@thompson# dd if=/dev/zero bs=1m count=2048 of=test.img
>  | 2048+0 records in
>  | 2048+0 records out
>  | 2147483648 bytes transferred in 4.127411 secs (520298036 bytes/sec)
>  |
>  | root@thompson# l
>  | total 2097708
>  |       4 drwxr-x---   2 root  wheel  -           512 11 Apr 04:08 ./
>  |       4 drwxr-xr-x  16 root  wheel  -         1,024 11 Apr 04:08 ../
>  | 2097696 -rw-r-----   1 root  wheel  - 2,147,483,648 11 Apr 04:08 test.img
>  |       4 -rw-r-----   1 root  wheel  -            43 11 Apr 04:08 typescript
>  |
>  | root@thompson# mdconfig test.img
>  | md1
>  |
>  | root@thompson# newfs /dev/md1
>  | /dev/md1: 2048.0MB (4194304 sectors) block size 32768, fragment size 4096
>  |         using 4 cylinder groups of 512.03MB, 16385 blks, 65664 inodes.
>  | super-block backups (for fsck_ffs -b #) at:
>  |  192, 1048832, 2097472, 3146112
>  |
>  | root@thompson# md mnt
>  | mnt
>  |
>  | root@thompson# mount /dev/md1 mnt
>  |
>  | root@thompson# cd mnt/
>  | ~/x/mnt ~/x
>  |
>  | root@thompson# df .
>  | Filesystem 1K-blocks Used     Avail Capacity  Mounted on
>  | /dev/md1   2,031,132    8 1,868,636     0%    /root/x/mnt
>  |
>  | root@thompson# l
>  | total 12
>  | 4 drwxr-xr-x  3 root  wheel     - 512 11 Apr 04:09 ./
>  | 4 drwxr-x---  3 root  wheel     - 512 11 Apr 04:09 ../
>  | 4 drwxrwxr-x  2 root  operator  - 512 11 Apr 04:09 .snap/
>  |
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test
>  |
>  | root@thompson# l
>  | total 652
>  |   4 drwxr-xr-x  3 root  wheel     -           512 11 Apr 04:14 ./
>  |   4 drwxr-x---  3 root  wheel     -           512 11 Apr 04:09 ../
>  |   4 drwxrwxr-x  2 root  operator  -           512 11 Apr 04:09 .snap/
>  | 640 -rw-r-----  1 root  wheel     - 9,663,676,605 11 Apr 04:14 test
>  |
>  | root@thompson# sha256 -r test > sha256.out
>  |
>  | root@thompson# cd ..
>  | ~/x ~/x/mnt
>  |
>  | root@thompson# umount mnt
>  |
>  | root@thompson# fsck /dev/md1
>  | ** /dev/md1
>  | ** Last Mounted on /root/x/mnt
>  | ** Phase 1 - Check Blocks and Sizes
>  | INODE 4: FILE SIZE 9663676605 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 1342210048
>  | ADJUST? [yn] y
>  |
>  | ** Phase 2 - Check Pathnames
>  | ** Phase 3 - Check Connectivity
>  | ** Phase 4 - Check Reference Counts
>  | ** Phase 5 - Check Cyl groups
>  | 4 files, 163 used, 507620 free (20 frags, 63450 blocks, 0.0% fragmentation)
>  |
>  | ***** FILE SYSTEM IS CLEAN *****
>  |
>  | ***** FILE SYSTEM WAS MODIFIED *****
>  |
>  | root@thompson# fsck /dev/md1
>  | ** /dev/md1
>  | ** Last Mounted on /root/x/mnt
>  | ** Phase 1 - Check Blocks and Sizes
>  | PARTIALLY TRUNCATED INODE I=4
>  | SALVAGE? [yn] y
>  |
>  | INCORRECT BLOCK COUNT I=4 (1280 should be 256)
>  | CORRECT? [yn] y
>  |
>  | INODE 4: FILE SIZE 1342210048 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 268468224
>  | ADJUST? [yn] y
>  |
>  | ** Phase 2 - Check Pathnames
>  | ** Phase 3 - Check Connectivity
>  | ** Phase 4 - Check Reference Counts
>  | ** Phase 5 - Check Cyl groups
>  | FREE BLK COUNT(S) WRONG IN SUPERBLK
>  | SALVAGE? [yn] y
>  |
>  | SUMMARY INFORMATION BAD
>  | SALVAGE? [yn] y
>  |
>  | BLK(S) MISSING IN BIT MAPS
>  | SALVAGE? [yn] y
>  |
>  | 4 files, 35 used, 507748 free (20 frags, 63466 blocks, 0.0% fragmentation)
>  |
>  | ***** FILE SYSTEM IS CLEAN *****
>  |
>  | ***** FILE SYSTEM WAS MODIFIED *****
>  |
>  | root@thompson# fsck /dev/md1
>  | ** /dev/md1
>  | ** Last Mounted on /root/x/mnt
>  | ** Phase 1 - Check Blocks and Sizes
>  | PARTIALLY TRUNCATED INODE I=4
>  | SALVAGE? [yn] y
>  |
>  | INCORRECT BLOCK COUNT I=4 (256 should be 128)
>  | CORRECT? [yn] y
>  |
>  | INODE 4: FILE SIZE 268468224 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 134610944
>  | ADJUST? [yn] y
>  |
>  | ** Phase 2 - Check Pathnames
>  | ** Phase 3 - Check Connectivity
>  | ** Phase 4 - Check Reference Counts
>  | ** Phase 5 - Check Cyl groups
>  | FREE BLK COUNT(S) WRONG IN SUPERBLK
>  | SALVAGE? [yn] y
>  |
>  | SUMMARY INFORMATION BAD
>  | SALVAGE? [yn] y
>  |
>  | BLK(S) MISSING IN BIT MAPS
>  | SALVAGE? [yn] y
>  |
>  | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation)
>  |
>  | ***** FILE SYSTEM IS CLEAN *****
>  |
>  | ***** FILE SYSTEM WAS MODIFIED *****
>  |
>  | root@thompson# fsck /dev/md1
>  | ** /dev/md1
>  | ** Last Mounted on /root/x/mnt
>  | ** Phase 1 - Check Blocks and Sizes
>  | ** Phase 2 - Check Pathnames
>  | ** Phase 3 - Check Connectivity
>  | ** Phase 4 - Check Reference Counts
>  | ** Phase 5 - Check Cyl groups
>  | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation)
>  |
>  | ***** FILE SYSTEM IS CLEAN *****
>  |
>  | root@thompson# fsck /dev/md1
>  | ** /dev/md1
>  | ** Last Mounted on /root/x/mnt
>  | ** Phase 1 - Check Blocks and Sizes
>  | ** Phase 2 - Check Pathnames
>  | ** Phase 3 - Check Connectivity
>  | ** Phase 4 - Check Reference Counts
>  | ** Phase 5 - Check Cyl groups
>  | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation)
>  |
>  | ***** FILE SYSTEM IS CLEAN *****
>  |
>  | root@thompson# mount /dev/md1 mnt
>  |
>  | root@thompson# cd mnt/
>  | ~/x/mnt ~/x
>  |
>  | root@thompson# l
>  | total 80
>  |  4 drwxr-xr-x  3 root  wheel     -         512 11 Apr 04:14 ./
>  |  4 drwxr-x---  3 root  wheel     -         512 11 Apr 04:09 ../
>  |  4 drwxrwxr-x  2 root  operator  -         512 11 Apr 04:09 .snap/
>  |  4 -rw-r-----  1 root  wheel     -          70 11 Apr 04:14 sha256.out
>  | 64 -rw-r-----  1 root  wheel     - 134,610,944 11 Apr 04:14 test
>  |
>  | root@thompson# cat sha256.out
>  | 76b042e7fbb3ed1914cf600a0b5ed8e10b8d917a006dbbff774a996c9bbce941 test
>  |
>  | root@thompson# sha256 -r test
>  | 6b1a548d057244632b5d2897f8c17177236c262c6af54cc0a9db5ddc8285fbd4 test
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"

I see this even with a single truncate on HEAD.

$ ./truncate10.sh
96 -rw-r--r--  1 root  wheel  1073741824 11 apr. 06:33 test
** /dev/md10a
** Last Mounted on /mnt
** Phase 1 - Check Blocks and Sizes
INODE 3: FILE SIZE 1073741824 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 268435456
ADJUST? yes


-- 
Peter



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190411043620.GA87473>