Date: Fri, 29 Dec 2023 21:42:25 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 276002] nfscl: data corruption using copy_file_range Message-ID: <bug-276002-227@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276002 Bug ID: 276002 Summary: nfscl: data corruption using copy_file_range Product: Base System Version: 15.0-CURRENT Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org I can reliably generate corruption on 15.0-CURRENT's NFS client using fsx, = with copy_file_range enabled. I can't reproduce it on ZFS, UFS, tmpfs, or msdos= fs.=20 I can't reproduce it on Linux's NFS client either (though I'm not positive = that it's using server-side-copying). Environment =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D NFS Server: FreeBSD 14.0-RELEASE-p3, amd64 NFS Clients: Both FreeBSD 14.0-RELEASE-p3 and 15.0-CURRENT @ 20-Dec.=20 /etc/fstab is configured like this: 192.168.1.2:/home /usr/home nfs rw,nfsv4,minorversion=3D2 0= =20=20=20=20=20=20 0 Steps To Reproduce =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) Check out the fsx-rs git repository (I'm working on getting the latest version into ports) git clone git@github.com:asomers/fsx-rs.git 2) Install a Rust toolchain, if you don't have one already. pkg install rust 3) cd fsx-rs 4) cat > copy_file_range.toml <<HERE nomsyncafterwrite =3D true [weights] truncate =3D 1 fsync =3D 1 fdatasync =3D 1 punch_hole =3D 0 sendfile =3D 1 write =3D 10 read =3D 10 copy_file_range =3D 10 HERE 5) Run the tool. Assuming /home is NFS_mounted, run like this: cargo run --release -- -N 1024 -S 3381155135556591634 -f copy_file_range.toml -m 149264:149265 -v ~/tmp/foo.bin Expected Output =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ... All operations completed A-OK! Actual Output =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [INFO fsx] 3 mapwrite 0x3d369 .. 0x3ffff ( 0x2c97 bytes) [INFO fsx] 4 mapread 0x3b828 .. 0x3ffff ( 0x47d8 bytes) [INFO fsx] 5 write 0x18a59 .. 0x21671 ( 0x8c19 bytes) [INFO fsx] 6 write 0x3407e .. 0x3d057 ( 0x8fda bytes) [INFO fsx] 7 sendfile 0x141b5 .. 0x1da23 ( 0x986f bytes) [INFO fsx] 8 copy_file_range [0x326b6:0x357d2] =3D> [0x1c39b:0x1f4b7] ( 0x311d bytes) [INFO fsx] 9 copy_file_range [0x36e5e:0x3feb4] =3D> [ 0xa813:0x13869] ( 0x9057 bytes) [INFO fsx] 10 read 0x18e3a .. 0x20a9c ( 0x7c63 bytes) [WARN fsx] 11 write 0x20935 .. 0x2a296 ( 0x9962 bytes) [INFO fsx] 12 mapwrite 0x3eadf .. 0x3ffff ( 0x1521 bytes) [INFO fsx] 13 copy_file_range [0x3b9d9:0x3e7bf] =3D> [0x31338:0x3411e] ( 0x2de7 bytes) [WARN fsx] 14 mapwrite 0x22763 .. 0x2b5af ( 0x8e4d bytes) [INFO fsx] 15 write 0x12126 .. 0x1666c ( 0x4547 bytes) [WARN fsx] 16 mapread 0x1bc78 .. 0x2ad3d ( 0xf0c6 bytes) [INFO fsx] 17 write 0x29386 .. 0x36d8b ( 0xda06 bytes) [INFO fsx] 18 write 0x3cff4 .. 0x3ffff ( 0x300c bytes) [INFO fsx] 19 copy_file_range [0x3c882:0x3dc94] =3D> [ 0x6fbe: 0x83d0] ( 0x1413 bytes) [WARN fsx] 20 truncate 0x40000 =3D> 0x14e87 [INFO fsx] 21 mapwrite 0x194a2 .. 0x19bbb ( 0x71a bytes) [INFO fsx] 22 read 0x10595 .. 0x19bbb ( 0x9627 bytes) [INFO fsx] 23 mapread 0x11ebf .. 0x14df9 ( 0x2f3b bytes) [INFO fsx] 24 mapread 0x1887 .. 0x11174 ( 0xf8ee bytes) [INFO fsx] 25 mapread 0x13b75 .. 0x19bbb ( 0x6047 bytes) [INFO fsx] 26 write 0x1867c .. 0x19f7a ( 0x18ff bytes) [INFO fsx] 27 mapread 0x119df .. 0x13ecb ( 0x24ed bytes) [INFO fsx] 28 mapwrite 0x19c2d .. 0x1ec85 ( 0x5059 bytes) [INFO fsx] 29 mapwrite 0x580f .. 0x14e19 ( 0xf60b bytes) [INFO fsx] 30 mapwrite 0x2e746 .. 0x3b542 ( 0xcdfd bytes) [INFO fsx] 31 copy_file_range [0x1ae6f:0x1e92f] =3D> [0x173ae:0x1ae6e] ( 0x3ac1 bytes) [INFO fsx] 32 read 0x2e63a .. 0x3af8d ( 0xc954 bytes) [INFO fsx] 33 write 0x3313d .. 0x38155 ( 0x5019 bytes) [INFO fsx] 34 mapread 0x332a5 .. 0x35e49 ( 0x2ba5 bytes) [INFO fsx] 35 copy_file_range [0x17dc9:0x2206d] =3D> [0x2e626:0x388ca] ( 0xa2a5 bytes) [INFO fsx] 36 fdatasync [INFO fsx] 37 read 0x2dad6 .. 0x3b3c8 ( 0xd8f3 bytes) [INFO fsx] 38 sendfile 0x27769 .. 0x346a8 ( 0xcf40 bytes) [INFO fsx] 39 mapwrite 0x38473 .. 0x3ffff ( 0x7b8d bytes) [INFO fsx] 40 read 0x14cb6 .. 0x157ce ( 0xb19 bytes) [INFO fsx] 41 sendfile 0x10a33 .. 0x1f57b ( 0xeb49 bytes) [INFO fsx] 42 fsync [INFO fsx] 43 copy_file_range [ 0xa490: 0xb45f] =3D> [ 0xfb8: 0x1f87] (= =20 0xfd0 bytes) [INFO fsx] 44 mapread 0x31956 .. 0x3ffff ( 0xe6aa bytes) [INFO fsx] 45 copy_file_range [ 0xc1ad: 0xc474] =3D> [ 0x7e98: 0x815f] (= =20 0x2c8 bytes) [INFO fsx] 46 mapread 0x8bf .. 0xc9a9 ( 0xc0eb bytes) [INFO fsx] 47 write 0x28d35 .. 0x2d27e ( 0x454a bytes) [INFO fsx] 48 mapwrite 0x3f5d8 .. 0x3ffff ( 0xa28 bytes) [INFO fsx] 49 fsync [INFO fsx] 50 copy_file_range [0x2b709:0x2e6dc] =3D> [0x32312:0x352e5] ( 0x2fd4 bytes) [INFO fsx] 51 write 0x183d9 .. 0x1f309 ( 0x6f31 bytes) [INFO fsx] 52 copy_file_range [0x2c928:0x31221] =3D> [0x31222:0x35b1b] ( 0x48fa bytes) [INFO fsx] 53 fsync [INFO fsx] 54 write 0x39603 .. 0x3ffff ( 0x69fd bytes) [INFO fsx] 55 write 0x30895 .. 0x39042 ( 0x87ae bytes) [INFO fsx] 56 write 0x3c384 .. 0x3ffff ( 0x3c7c bytes) [INFO fsx] 57 mapwrite 0x33ab6 .. 0x39fa0 ( 0x64eb bytes) [WARN fsx] 58 write 0x23f7e .. 0x2b871 ( 0x78f4 bytes) [INFO fsx] 59 mapwrite 0x1a29b .. 0x1c938 ( 0x269e bytes) [INFO fsx] 60 write 0x29da5 .. 0x2effc ( 0x5258 bytes) [INFO fsx] 61 read 0x29919 .. 0x3099d ( 0x7085 bytes) [WARN fsx] 62 mapread 0x1f81d .. 0x2d882 ( 0xe066 bytes) [INFO fsx] 63 write 0x32fd5 .. 0x3479f ( 0x17cb bytes) [INFO fsx] 64 copy_file_range [ 0xf5b5:0x13756] =3D> [0x3be5e:0x3ffff] ( 0x41a2 bytes) [WARN fsx] 65 sendfile 0x1fd25 .. 0x2cffb ( 0xd2d7 bytes) [WARN fsx] 66 copy_file_range [0x30269:0x380a6] =3D> [0x201d9:0x28016] ( 0x7e3e bytes) [INFO fsx] 67 read 0x3f602 .. 0x3ffff ( 0x9fe bytes) [WARN fsx] 68 read 0x23bd4 .. 0x25fc1 ( 0x23ee bytes) [ERROR fsx] miscompare: offset=3D 0x23bd4, size =3D 0x23ee [ERROR fsx] OFFSET GOOD BAD RANGE=20=20 [ERROR fsx] 0x24710 0x39 0x37 0x18b2 [ERROR fsx] Step# (mod 256) for a misdirected write may be 55 [ERROR fsx] Using seed 3381155135556591634 [ERROR fsx] LOG DUMP [ERROR fsx] 1 SKIPPED (read) [ERROR fsx] 2 SKIPPED (mapread) [ERROR fsx] 3 MAPWRITE 0x3d369 =3D> 0x40000 ( 0x2c97 bytes) HOLE [ERROR fsx] 4 MAPREAD 0x3b828 =3D> 0x40000 ( 0x47d8 bytes) [ERROR fsx] 5 WRITE 0x18a59 =3D> 0x21672 ( 0x8c19 bytes) [ERROR fsx] 6 WRITE 0x3407e =3D> 0x3d058 ( 0x8fda bytes) [ERROR fsx] 7 SENDFILE 0x141b5 =3D> 0x1da24 ( 0x986f bytes) [ERROR fsx] 8 COPY_FILE_RANGE [0x326b6,0x357d3] =3D> [0x1c39b,0x1f4b8] ( 0x311d bytes) [ERROR fsx] 9 COPY_FILE_RANGE [0x36e5e,0x3feb5] =3D> [ 0xa813,0x1386a] ( 0x9057 bytes) [ERROR fsx] 10 READ 0x18e3a =3D> 0x20a9d ( 0x7c63 bytes) [ERROR fsx] 11 WRITE 0x20935 =3D> 0x2a297 ( 0x9962 bytes) [ERROR fsx] 12 MAPWRITE 0x3eadf =3D> 0x40000 ( 0x1521 bytes) [ERROR fsx] 13 COPY_FILE_RANGE [0x3b9d9,0x3e7c0] =3D> [0x31338,0x3411f] ( 0x2de7 bytes) [ERROR fsx] 14 MAPWRITE 0x22763 =3D> 0x2b5b0 ( 0x8e4d bytes) [ERROR fsx] 15 WRITE 0x12126 =3D> 0x1666d ( 0x4547 bytes) [ERROR fsx] 16 MAPREAD 0x1bc78 =3D> 0x2ad3e ( 0xf0c6 bytes) [ERROR fsx] 17 WRITE 0x29386 =3D> 0x36d8c ( 0xda06 bytes) [ERROR fsx] 18 WRITE 0x3cff4 =3D> 0x40000 ( 0x300c bytes) [ERROR fsx] 19 COPY_FILE_RANGE [0x3c882,0x3dc95] =3D> [ 0x6fbe, 0x83d1] ( 0x1413 bytes) [ERROR fsx] 20 TRUNCATE DOWN from 0x40000 to 0x14e87 [ERROR fsx] 21 MAPWRITE 0x194a2 =3D> 0x19bbc ( 0x71a bytes) HOLE [ERROR fsx] 22 READ 0x10595 =3D> 0x19bbc ( 0x9627 bytes) [ERROR fsx] 23 MAPREAD 0x11ebf =3D> 0x14dfa ( 0x2f3b bytes) [ERROR fsx] 24 MAPREAD 0x1887 =3D> 0x11175 ( 0xf8ee bytes) [ERROR fsx] 25 MAPREAD 0x13b75 =3D> 0x19bbc ( 0x6047 bytes) [ERROR fsx] 26 WRITE 0x1867c =3D> 0x19f7b ( 0x18ff bytes) EXTEND [ERROR fsx] 27 MAPREAD 0x119df =3D> 0x13ecc ( 0x24ed bytes) [ERROR fsx] 28 MAPWRITE 0x19c2d =3D> 0x1ec86 ( 0x5059 bytes) EXTEND [ERROR fsx] 29 MAPWRITE 0x580f =3D> 0x14e1a ( 0xf60b bytes) [ERROR fsx] 30 MAPWRITE 0x2e746 =3D> 0x3b543 ( 0xcdfd bytes) HOLE [ERROR fsx] 31 COPY_FILE_RANGE [0x1ae6f,0x1e930] =3D> [0x173ae,0x1ae6f] ( 0x3ac1 bytes) [ERROR fsx] 32 READ 0x2e63a =3D> 0x3af8e ( 0xc954 bytes) [ERROR fsx] 33 WRITE 0x3313d =3D> 0x38156 ( 0x5019 bytes) [ERROR fsx] 34 MAPREAD 0x332a5 =3D> 0x35e4a ( 0x2ba5 bytes) [ERROR fsx] 35 COPY_FILE_RANGE [0x17dc9,0x2206e] =3D> [0x2e626,0x388cb] ( 0xa2a5 bytes) [ERROR fsx] 36 FDATASYNC [ERROR fsx] 37 READ 0x2dad6 =3D> 0x3b3c9 ( 0xd8f3 bytes) [ERROR fsx] 38 SENDFILE 0x27769 =3D> 0x346a9 ( 0xcf40 bytes) [ERROR fsx] 39 MAPWRITE 0x38473 =3D> 0x40000 ( 0x7b8d bytes) EXTEND [ERROR fsx] 40 READ 0x14cb6 =3D> 0x157cf ( 0xb19 bytes) [ERROR fsx] 41 SENDFILE 0x10a33 =3D> 0x1f57c ( 0xeb49 bytes) [ERROR fsx] 42 FSYNC [ERROR fsx] 43 COPY_FILE_RANGE [ 0xa490, 0xb460] =3D> [ 0xfb8, 0x1f88] (= =20 0xfd0 bytes) [ERROR fsx] 44 MAPREAD 0x31956 =3D> 0x40000 ( 0xe6aa bytes) [ERROR fsx] 45 COPY_FILE_RANGE [ 0xc1ad, 0xc475] =3D> [ 0x7e98, 0x8160] (= =20 0x2c8 bytes) [ERROR fsx] 46 MAPREAD 0x8bf =3D> 0xc9aa ( 0xc0eb bytes) [ERROR fsx] 47 WRITE 0x28d35 =3D> 0x2d27f ( 0x454a bytes) [ERROR fsx] 48 MAPWRITE 0x3f5d8 =3D> 0x40000 ( 0xa28 bytes) [ERROR fsx] 49 FSYNC [ERROR fsx] 50 COPY_FILE_RANGE [0x2b709,0x2e6dd] =3D> [0x32312,0x352e6] ( 0x2fd4 bytes) [ERROR fsx] 51 WRITE 0x183d9 =3D> 0x1f30a ( 0x6f31 bytes) [ERROR fsx] 52 COPY_FILE_RANGE [0x2c928,0x31222] =3D> [0x31222,0x35b1c] ( 0x48fa bytes) [ERROR fsx] 53 FSYNC [ERROR fsx] 54 WRITE 0x39603 =3D> 0x40000 ( 0x69fd bytes) [ERROR fsx] 55 WRITE 0x30895 =3D> 0x39043 ( 0x87ae bytes) [ERROR fsx] 56 WRITE 0x3c384 =3D> 0x40000 ( 0x3c7c bytes) [ERROR fsx] 57 MAPWRITE 0x33ab6 =3D> 0x39fa1 ( 0x64eb bytes) [ERROR fsx] 58 WRITE 0x23f7e =3D> 0x2b872 ( 0x78f4 bytes) [ERROR fsx] 59 MAPWRITE 0x1a29b =3D> 0x1c939 ( 0x269e bytes) [ERROR fsx] 60 WRITE 0x29da5 =3D> 0x2effd ( 0x5258 bytes) [ERROR fsx] 61 READ 0x29919 =3D> 0x3099e ( 0x7085 bytes) [ERROR fsx] 62 MAPREAD 0x1f81d =3D> 0x2d883 ( 0xe066 bytes) [ERROR fsx] 63 WRITE 0x32fd5 =3D> 0x347a0 ( 0x17cb bytes) [ERROR fsx] 64 COPY_FILE_RANGE [ 0xf5b5,0x13757] =3D> [0x3be5e,0x40000] ( 0x41a2 bytes) [ERROR fsx] 65 SENDFILE 0x1fd25 =3D> 0x2cffc ( 0xd2d7 bytes) [ERROR fsx] 66 COPY_FILE_RANGE [0x30269,0x380a7] =3D> [0x201d9,0x28017] ( 0x7e3e bytes) [ERROR fsx] 67 READ 0x3f602 =3D> 0x40000 ( 0x9fe bytes) [ERROR fsx] 68 READ 0x23bd4 =3D> 0x25fc2 ( 0x23ee bytes) --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-276002-227>