From owner-freebsd-stable@freebsd.org Thu Apr 11 04:36:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0145615784B5 for ; Thu, 11 Apr 2019 04:36:34 +0000 (UTC) (envelope-from pho@holm.cc) Received: from relay05.pair.com (relay05.pair.com [216.92.24.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91E5674486 for ; Thu, 11 Apr 2019 04:36:33 +0000 (UTC) (envelope-from pho@holm.cc) Received: from x2.osted.lan (87-58-223-204-dynamic.dk.customer.tdc.net [87.58.223.204]) by relay05.pair.com (Postfix) with ESMTP id 50CC31A2D39; Thu, 11 Apr 2019 00:36:31 -0400 (EDT) Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.15.2/8.15.2) with ESMTPS id x3B4aLLX088302 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 11 Apr 2019 06:36:21 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.15.2/8.15.2/Submit) id x3B4aKLJ088301; Thu, 11 Apr 2019 06:36:20 +0200 (CEST) (envelope-from pho) Date: Thu, 11 Apr 2019 06:36:20 +0200 From: Peter Holm To: Jamie Landeg-Jones Cc: freebsd-stable@freebsd.org, Kirk McKusick Subject: Re: Replicable file-system corruption due to fsck/ufs Message-ID: <20190411043620.GA87473@x2.osted.lan> References: <201904110347.x3B3lhpA034197@donotpassgo.dyslexicfish.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201904110347.x3B3lhpA034197@donotpassgo.dyslexicfish.net> User-Agent: Mutt/1.11.1 (2018-12-01) X-Rspamd-Queue-Id: 91E5674486 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.987,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 04:36:34 -0000 On Thu, Apr 11, 2019 at 04:47:43AM +0100, Jamie Landeg-Jones wrote: > I've noticed a replicable disk corruption by fsck_ufs/ffs on sparse files. > > This is on amd/64 12-stable-20190409, but I first noticed it on > 12-stable-20190326. > > I didn't notice it on my previous build of 12-stable-20190107, but I > may not have had any relevant sparse files at the time, so I don't know > if that version was affected. 12-release worked OK. > > Here is a simplified replicable example. Thinking about it just now, I > suspect it's triggered by files which end in sparseness. > > Can anyone else replicate this, or has my machine gone nuts? > > Cheers, Jamie > > | root@thompson# l > | total 12 > | 4 drwxr-x--- 2 root wheel - 512 11 Apr 04:08 ./ > | 4 drwxr-xr-x 16 root wheel - 1,024 11 Apr 04:08 ../ > | 4 -rw-r----- 1 root wheel - 43 11 Apr 04:08 typescript > | > | root@thompson# dd if=/dev/zero bs=1m count=2048 of=test.img > | 2048+0 records in > | 2048+0 records out > | 2147483648 bytes transferred in 4.127411 secs (520298036 bytes/sec) > | > | root@thompson# l > | total 2097708 > | 4 drwxr-x--- 2 root wheel - 512 11 Apr 04:08 ./ > | 4 drwxr-xr-x 16 root wheel - 1,024 11 Apr 04:08 ../ > | 2097696 -rw-r----- 1 root wheel - 2,147,483,648 11 Apr 04:08 test.img > | 4 -rw-r----- 1 root wheel - 43 11 Apr 04:08 typescript > | > | root@thompson# mdconfig test.img > | md1 > | > | root@thompson# newfs /dev/md1 > | /dev/md1: 2048.0MB (4194304 sectors) block size 32768, fragment size 4096 > | using 4 cylinder groups of 512.03MB, 16385 blks, 65664 inodes. > | super-block backups (for fsck_ffs -b #) at: > | 192, 1048832, 2097472, 3146112 > | > | root@thompson# md mnt > | mnt > | > | root@thompson# mount /dev/md1 mnt > | > | root@thompson# cd mnt/ > | ~/x/mnt ~/x > | > | root@thompson# df . > | Filesystem 1K-blocks Used Avail Capacity Mounted on > | /dev/md1 2,031,132 8 1,868,636 0% /root/x/mnt > | > | root@thompson# l > | total 12 > | 4 drwxr-xr-x 3 root wheel - 512 11 Apr 04:09 ./ > | 4 drwxr-x--- 3 root wheel - 512 11 Apr 04:09 ../ > | 4 drwxrwxr-x 2 root operator - 512 11 Apr 04:09 .snap/ > | > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | root@thompson# echo "testing 1...2...3..." >> test ; truncate -s +1g test > | > | root@thompson# l > | total 652 > | 4 drwxr-xr-x 3 root wheel - 512 11 Apr 04:14 ./ > | 4 drwxr-x--- 3 root wheel - 512 11 Apr 04:09 ../ > | 4 drwxrwxr-x 2 root operator - 512 11 Apr 04:09 .snap/ > | 640 -rw-r----- 1 root wheel - 9,663,676,605 11 Apr 04:14 test > | > | root@thompson# sha256 -r test > sha256.out > | > | root@thompson# cd .. > | ~/x ~/x/mnt > | > | root@thompson# umount mnt > | > | root@thompson# fsck /dev/md1 > | ** /dev/md1 > | ** Last Mounted on /root/x/mnt > | ** Phase 1 - Check Blocks and Sizes > | INODE 4: FILE SIZE 9663676605 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 1342210048 > | ADJUST? [yn] y > | > | ** Phase 2 - Check Pathnames > | ** Phase 3 - Check Connectivity > | ** Phase 4 - Check Reference Counts > | ** Phase 5 - Check Cyl groups > | 4 files, 163 used, 507620 free (20 frags, 63450 blocks, 0.0% fragmentation) > | > | ***** FILE SYSTEM IS CLEAN ***** > | > | ***** FILE SYSTEM WAS MODIFIED ***** > | > | root@thompson# fsck /dev/md1 > | ** /dev/md1 > | ** Last Mounted on /root/x/mnt > | ** Phase 1 - Check Blocks and Sizes > | PARTIALLY TRUNCATED INODE I=4 > | SALVAGE? [yn] y > | > | INCORRECT BLOCK COUNT I=4 (1280 should be 256) > | CORRECT? [yn] y > | > | INODE 4: FILE SIZE 1342210048 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 268468224 > | ADJUST? [yn] y > | > | ** Phase 2 - Check Pathnames > | ** Phase 3 - Check Connectivity > | ** Phase 4 - Check Reference Counts > | ** Phase 5 - Check Cyl groups > | FREE BLK COUNT(S) WRONG IN SUPERBLK > | SALVAGE? [yn] y > | > | SUMMARY INFORMATION BAD > | SALVAGE? [yn] y > | > | BLK(S) MISSING IN BIT MAPS > | SALVAGE? [yn] y > | > | 4 files, 35 used, 507748 free (20 frags, 63466 blocks, 0.0% fragmentation) > | > | ***** FILE SYSTEM IS CLEAN ***** > | > | ***** FILE SYSTEM WAS MODIFIED ***** > | > | root@thompson# fsck /dev/md1 > | ** /dev/md1 > | ** Last Mounted on /root/x/mnt > | ** Phase 1 - Check Blocks and Sizes > | PARTIALLY TRUNCATED INODE I=4 > | SALVAGE? [yn] y > | > | INCORRECT BLOCK COUNT I=4 (256 should be 128) > | CORRECT? [yn] y > | > | INODE 4: FILE SIZE 268468224 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 134610944 > | ADJUST? [yn] y > | > | ** Phase 2 - Check Pathnames > | ** Phase 3 - Check Connectivity > | ** Phase 4 - Check Reference Counts > | ** Phase 5 - Check Cyl groups > | FREE BLK COUNT(S) WRONG IN SUPERBLK > | SALVAGE? [yn] y > | > | SUMMARY INFORMATION BAD > | SALVAGE? [yn] y > | > | BLK(S) MISSING IN BIT MAPS > | SALVAGE? [yn] y > | > | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation) > | > | ***** FILE SYSTEM IS CLEAN ***** > | > | ***** FILE SYSTEM WAS MODIFIED ***** > | > | root@thompson# fsck /dev/md1 > | ** /dev/md1 > | ** Last Mounted on /root/x/mnt > | ** Phase 1 - Check Blocks and Sizes > | ** Phase 2 - Check Pathnames > | ** Phase 3 - Check Connectivity > | ** Phase 4 - Check Reference Counts > | ** Phase 5 - Check Cyl groups > | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation) > | > | ***** FILE SYSTEM IS CLEAN ***** > | > | root@thompson# fsck /dev/md1 > | ** /dev/md1 > | ** Last Mounted on /root/x/mnt > | ** Phase 1 - Check Blocks and Sizes > | ** Phase 2 - Check Pathnames > | ** Phase 3 - Check Connectivity > | ** Phase 4 - Check Reference Counts > | ** Phase 5 - Check Cyl groups > | 4 files, 19 used, 507764 free (20 frags, 63468 blocks, 0.0% fragmentation) > | > | ***** FILE SYSTEM IS CLEAN ***** > | > | root@thompson# mount /dev/md1 mnt > | > | root@thompson# cd mnt/ > | ~/x/mnt ~/x > | > | root@thompson# l > | total 80 > | 4 drwxr-xr-x 3 root wheel - 512 11 Apr 04:14 ./ > | 4 drwxr-x--- 3 root wheel - 512 11 Apr 04:09 ../ > | 4 drwxrwxr-x 2 root operator - 512 11 Apr 04:09 .snap/ > | 4 -rw-r----- 1 root wheel - 70 11 Apr 04:14 sha256.out > | 64 -rw-r----- 1 root wheel - 134,610,944 11 Apr 04:14 test > | > | root@thompson# cat sha256.out > | 76b042e7fbb3ed1914cf600a0b5ed8e10b8d917a006dbbff774a996c9bbce941 test > | > | root@thompson# sha256 -r test > | 6b1a548d057244632b5d2897f8c17177236c262c6af54cc0a9db5ddc8285fbd4 test > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" I see this even with a single truncate on HEAD. $ ./truncate10.sh 96 -rw-r--r-- 1 root wheel 1073741824 11 apr. 06:33 test ** /dev/md10a ** Last Mounted on /mnt ** Phase 1 - Check Blocks and Sizes INODE 3: FILE SIZE 1073741824 BEYOND END OF ALLOCATED FILE, SIZE SHOULD BE 268435456 ADJUST? yes -- Peter