Date: Sun, 16 Jul 2023 18:17:09 -0700 From: Kevin Oberman <rkoberman@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: Current FreeBSD <freebsd-current@freebsd.org> Subject: Re: rsync use with -tmsdosfs mounted file system? file has vanished: . . . Message-ID: <CAN6yY1vsUddvZcrUDrD_cMZL28to5MxDcNjfQNUrsxuCaUHiOw@mail.gmail.com> In-Reply-To: <485EB974-5EF4-4569-9B40-81A474983F33@yahoo.com> References: <485EB974-5EF4-4569-9B40-81A474983F33.ref@yahoo.com> <485EB974-5EF4-4569-9B40-81A474983F33@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000b7b2ec0600a48f90 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Jul 16, 2023 at 1:57=E2=80=AFPM Mark Millard <marklmi@yahoo.com> wr= ote: > I used a sequence that looked like: > > mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /CA72optM2efi-media/ \ > && rsync -x --delete -aAUHhh --info=3Dprogress2 /boot/efi/ > /CA72optM2efi-media/ > > that got: > > file has vanished: "/CA72optM2efi-media/BCM271~5.DTB" > file has vanished: "/CA72optM2efi-media/BCM271~6.DTB" > 73.77K 0% 1.63MB/s 0:00:00 (xfr#7, to-chk=3D0/493) rsy= nc > warning: some files vanished before they could be transferred (code 24) a= t > main.c(1357) [sender=3D3.2.7] > > After that, activity reported the likes of: > > rsync: [generator] delete_file: unlink(overlays/VC4-KM~8.DTB) failed: > Read-only file system (30) > and: > rsync: [receiver] mkstemp "/CA72optM2efi-media/.fixup4.dat.2Wonu9" failed= : > Read-only file system (30) > > More than rsync was odd at that point: > > # ls -Tld /CA72optM2efi-media/*.DTB > ls: /CA72optM2efi-media/BCM271~5.DTB: No such file or directory > ls: /CA72optM2efi-media/BCM271~6.DTB: No such file or directory > > # rm /CA72optM2efi-media/*/*.DTB > override rwxr-xr-x root/wheel uarch for > /CA72optM2efi-media/overlays/SDHOST~1.DTB? y > rm: /CA72optM2efi-media/overlays/SDHOST~1.DTB: Read-only file system > . . . > > But: > > # mount | grep media > /dev/gpt/CA72optM2efi on /CA72optM2efi-media (msdosfs, local, noatime) > > So the mount itself was not the source of the read-only status so far. > > I then tried: > > # umount /CA72optM2efi-media > # newfs_msdos /dev/da0p1 > # mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /mnt > # cp -aRx /boot/efi/ /mnt/ > cp: utimensat: /mnt: Invalid argument > > (which is normal). > > # umount /mnt > > No more oddities , so far after that. > > > For reference: > > # uname -apKU > FreeBSD CA72-16Gp-ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #99 > main-n264171-2a0c0aea4209-dirty: Fri Jul 14 21:00:44 PDT 2023 > root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/ar= m64.aarch64/sys/GENERIC-NODBG-CA72 > arm64 aarch64 1400093 1400093 > > # pkg info rsync > rsync-3.2.7 > Name : rsync > Version : 3.2.7 > Installed on : Sat Jul 15 14:53:48 2023 PDT > Origin : net/rsync > Architecture : FreeBSD:14:aarch64 > . . . > Annotations : > FreeBSD_version: 1400092 > build_timestamp: 2023-07-02T06:57:44+0000 > built_by : poudriere-git-3.3.99.20220831 > cpe : cpe:2.3:a:samba:rsync:3.2.7:::::freebsd14:aarch64 > port_checkout_unclean: no > port_git_hash : f45cd5bd9d4b > ports_top_checkout_unclean: yes > ports_top_git_hash: 880f72e54deb > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.co <http://yahoo.com>; This looks a bit like an issue I was hitting on a 4 CPU, 4 thread Alder Lake processor and 500GB SSD running 13.2-RELEASE. I saw several very strange corruptions, at least one "rsync warning: some files vanished before they could be transferred". In one (the last) case, the system crashed. The 'disc' was corrupted badly enough that fsck failed and I could not boot up the system. The disk was UFS2 GPT format and EFI boot. The interface is SATA, not nVME. In this case, I was installing on a new system and copying the majority of the file system from my old system. The 'fix' is strange and probably not one many other can use. I installed a spinning rust drive of 500GB and installed FreeBSD and used rsync again and it worked. I can't say whether it was a fluke that it worked, but it really smells like some sort of race condition. Could be rsync , VFS, or device driver. Since then I have seen one crash while backing up the system disk using rsync. No corruption and doing another rsync after reboot worked fine, but it was a much smaller run as the first attempt was nearly complete when the system crashed. Maybe unrelated. I do have the core file from the crash. Stil, something weird has been going on. Same issue on two identical systems, so not likely hardware. --=20 Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 --000000000000b7b2ec0600a48f90 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:tahoma,sans-serif;font-size:small">On Sun, Jul 16, 2023 at 1:57=E2= =80=AFPM Mark Millard <<a href=3D"mailto:marklmi@yahoo.com">marklmi@yaho= o.com</a>> wrote:</div></div><div class=3D"gmail_quote"><blockquote clas= s=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid r= gb(204,204,204);padding-left:1ex">I used a sequence that looked like:<br> <br> mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /CA72optM2efi-media/ \<br> && rsync -x --delete -aAUHhh --info=3Dprogress2 /boot/efi/ /CA72opt= M2efi-media/<br> <br> that got:<br> <br> file has vanished: "/CA72optM2efi-media/BCM271~5.DTB"<br> file has vanished: "/CA72optM2efi-media/BCM271~6.DTB"<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A073.77K=C2=A0 =C2=A00%=C2=A0 =C2=A0 1.63MB= /s=C2=A0 =C2=A0 0:00:00 (xfr#7, to-chk=3D0/493)=C2=A0 =C2=A0rsync warning: = some files vanished before they could be transferred (code 24) at main.c(13= 57) [sender=3D3.2.7]<br> <br> After that, activity reported the likes of:<br> <br> rsync: [generator] delete_file: unlink(overlays/VC4-KM~8.DTB) failed: Read-= only file system (30)<br> and:<br> rsync: [receiver] mkstemp "/CA72optM2efi-media/.fixup4.dat.2Wonu9"= ; failed: Read-only file system (30)<br> <br> More than rsync was odd at that point:<br> <br> # ls -Tld /CA72optM2efi-media/*.DTB<br> ls: /CA72optM2efi-media/BCM271~5.DTB: No such file or directory<br> ls: /CA72optM2efi-media/BCM271~6.DTB: No such file or directory<br> <br> # rm /CA72optM2efi-media/*/*.DTB<br> override rwxr-xr-x root/wheel uarch for /CA72optM2efi-media/overlays/SDHOST= ~1.DTB? y<br> rm: /CA72optM2efi-media/overlays/SDHOST~1.DTB: Read-only file system<br> . . .<br> <br> But:<br> <br> # mount | grep media<br> /dev/gpt/CA72optM2efi on /CA72optM2efi-media (msdosfs, local, noatime)<br> <br> So the mount itself was not the source of the read-only status so far.<br> <br> I then tried:<br> <br> # umount /CA72optM2efi-media<br> # newfs_msdos /dev/da0p1<br> # mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /mnt<br> # cp -aRx /boot/efi/ /mnt/<br> cp: utimensat: /mnt: Invalid argument<br> <br> (which is normal).<br> <br> # umount /mnt<br> <br> No more oddities , so far after that.<br> <br> <br> For reference:<br> <br> # uname -apKU<br> FreeBSD CA72-16Gp-ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #99 main-n264171-2a= 0c0aea4209-dirty: Fri Jul 14 21:00:44 PDT 2023=C2=A0 =C2=A0 =C2=A0root@CA72= -16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/= sys/GENERIC-NODBG-CA72 arm64 aarch64 1400093 1400093<br> <br> # pkg info rsync<br> rsync-3.2.7<br> Name=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: rsync<br> Version=C2=A0 =C2=A0 =C2=A0 =C2=A0 : 3.2.7<br> Installed on=C2=A0 =C2=A0: Sat Jul 15 14:53:48 2023 PDT<br> Origin=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: net/rsync<br> Architecture=C2=A0 =C2=A0: FreeBSD:14:aarch64<br> . . .<br> Annotations=C2=A0 =C2=A0 :<br> FreeBSD_version: 1400092<br> build_timestamp: 2023-07-02T06:57:44+0000<br> built_by=C2=A0 =C2=A0 =C2=A0 =C2=A0: poudriere-git-3.3.99.20220831<br> cpe=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : cpe:2.3:a:samba:rsync:3.2.7:= ::::freebsd14:aarch64<br> port_checkout_unclean: no<br> port_git_hash=C2=A0 : f45cd5bd9d4b<br> ports_top_checkout_unclean: yes<br> ports_top_git_hash: 880f72e54deb<br> <br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_blank= ">yahoo.co</a></blockquote><div><br></div><div style=3D"font-family:tahoma,= sans-serif;font-size:small" class=3D"gmail_default">This looks a bit like a= n issue I was hitting on a 4 CPU, 4 thread Alder Lake processor and 500GB S= SD running 13.2-RELEASE.</div><div style=3D"font-family:tahoma,sans-serif;f= ont-size:small" class=3D"gmail_default"><br></div><div style=3D"font-family= :tahoma,sans-serif;font-size:small" class=3D"gmail_default">I saw several v= ery strange corruptions, at least one "rsync warning: some files vanis= hed before they could be transferred". In one (the last) case, the sys= tem crashed. The 'disc' was corrupted badly enough that fsck failed= and I could not boot up the system. The disk was UFS2 GPT format and EFI b= oot. The interface is SATA, not nVME. In this case, I was installing on a n= ew system and copying the majority of the file system from my old system. <= br></div><div style=3D"font-family:tahoma,sans-serif;font-size:small" class= =3D"gmail_default"><br></div><div style=3D"font-family:tahoma,sans-serif;fo= nt-size:small" class=3D"gmail_default">The 'fix' is strange and pro= bably not one many other can use. I installed a spinning rust drive of 500G= B and installed FreeBSD and used rsync again and it worked. I can't say= whether it was a fluke that it worked, but it really smells like some sort= of race condition. Could be rsync , VFS, or device driver. Since then I ha= ve seen one crash while backing up the system disk using rsync. No corrupti= on and doing another rsync after reboot worked fine, but it was a much smal= ler run as the first attempt was nearly complete when the system crashed. M= aybe unrelated. I do have the core file from the crash. Stil, something wei= rd has been going on. Same issue on two identical systems, so not likely ha= rdware.<br></div></div><span class=3D"gmail_signature_prefix">-- </span><br= ><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir= =3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr">Kevin Oberman, Part ti= me kid herder and retired Network Engineer<br>E-mail: <a href=3D"mailto:rko= berman@gmail.com" target=3D"_blank">rkoberman@gmail.com</a><br></div><div>P= GP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683</div></div></div><= /div></div></div></div></div></div> --000000000000b7b2ec0600a48f90--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAN6yY1vsUddvZcrUDrD_cMZL28to5MxDcNjfQNUrsxuCaUHiOw>