Date: Sun, 17 Apr 2022 01:13:02 +0300 From: Sami Halabi <sodynet1@gmail.com> To: Warner Losh <imp@bsdimp.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: recover deleted file Message-ID: <CAEW%2BogY_PeZs0AKn%2BUY4gDk8Q8XmNR%2BO5u3Yte9tncXh2fTnDQ@mail.gmail.com> In-Reply-To: <CANCZdfqNRD9G-q72mD2ZjUYPwnqwmLLzQeuopxZPdCOEyM=jxA@mail.gmail.com> References: <CAEW%2Boga0BNKZKsnAQWzEqMuUzUwAR293w0XLXM0YhQAsQiESyA@mail.gmail.com> <CANCZdfqNRD9G-q72mD2ZjUYPwnqwmLLzQeuopxZPdCOEyM=jxA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000055623b05dcccd574 Content-Type: text/plain; charset="UTF-8" Hi warner, Thanks for trying :) Actually my use case was (if you read later replies, i gave up since the downtime was too long and couldn't wait more) a VM in ESXi. so all the underlying stuff of the disks/TRIM.. is hidden an inaccessible for me (hosting provider). In my case I tried to recover MariaDB database files, and some tar.gz file of backups of the db that I accidentally deleted all together (mysql* instead of mysql/* , my bad, deleted mysql/ & mysql_backups!!). I stopped all services immediately, so maybe I would succeed if I wasn't time limited. I understand its hard to undelete since no one designed UFS/ZFS to do so.. that why I asked in later replies to see if someone would step in and implement such a "feature" and I suggested some directions/thoughts. As soren@ suggested in later reply it maybe would be easier to implement custom rm script that moves files to "Recycle bin" directory (and empty it after some period) but as a programmer I know that perfection is needed :) so It might start as a simple task and end in many what-if's (unfortunattly I did my last C programming in late 2003!). What amzes me is that this "feature" was asked too much in the last decade or two and no one ever implemented it, maybe it's not needed in daily usage, but in disasters it would be super userful, save admins many time and nerves.. For now I did some backup tools locally and used chflags to mark them undeletable so I wouldn't do that mistake again, plus I rsync them to my home storage.. so probably I would be more resilent to such mistakes in the future.. but the same problem remains.. accidently deleted file(s)/directory(s) are the nightmare of all admins in earth!! Sami On Sun, Apr 17, 2022 at 12:42 AM Warner Losh <imp@bsdimp.com> wrote: > > > On Sat, Apr 16, 2022 at 5:24 AM Sami Halabi <sodynet1@gmail.com> wrote: > >> Hi, >> is there anyway easy to restore deleted file by accident in UFS???? >> > > Do you know what the contents of the file is? At least the first, say, > ~32k? > > The problem with unrm for ufs is that the directory entry has the inode > number stored in it. > Without the inode number, you won't get very far. > With the inode number, you can get the first 12 filesystem blocks of the > file and the > first three indirect blocks. Once you have those, you can reconstruct the > file. > > But only if the inode hasn't been zero'd out (which it likely has, another > thing that makes > UFS undelete harder). But all hope isn't lost... UFS has a predictable > allocation algorithm > that lets you get much of the file back (which is why I asked if you know > how it should start: > you can find where it starts in the data blocks and maybe get lucky with > the rest if the > data spills into indirect blocks). > > However, that's only if you don't have TRIM enabled on the filesystem. If > you do, > then UFS will do a BIO_DELETE of the blocks, which means their contents are > likely gone at the drive level. I say likely because there's weasel words > in the ATA > spec that allows a drive to return the prior contents of the blocks, or > all zeros or > the drive's initialization pattern (usually all 1's) when the blocks are > later read. Same > goes for NVMe drives (with the additional constraint it must be > deterministic). So there's > may still be a chance you can read the old contents, but drives that do > that are rare > in my experience (which is admittedly quite narrow). > > But, if you want to use fsdb to try to recover this data, or write your > own tools, > then you should likely have a copy of the daemon book (The Design and > Implementation > of the FreeBSD Operating System). It explains a lot of the finer details > of UFS and > reference to it likely will catch me where my memory isn't quite right in > the above > descriptions. > > So, it's for all these reasons you can't find somebody with a unrm command > for ufs > like you can for DOS or other filesystems. I wish I had a better answer > for you. > > Warner > > >> Sami >> >> -- >> Sami Halabi >> Information Systems Engineer >> NMS Projects Expert, FreeBSD SysAdmin Expert >> Asterisk Expert >> > -- Sami Halabi Information Systems Engineer NMS Projects Expert, FreeBSD SysAdmin Expert Asterisk Expert --00000000000055623b05dcccd574 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi warner,<div>Thanks for trying :)</div><div><br></div><d= iv>Actually my use case was (if you read later replies, i gave up since the= downtime was too long and couldn't wait more) a VM in ESXi.</div><div>= so all the underlying stuff of the disks/TRIM.. is hidden an inaccessible= =C2=A0for me (hosting provider).</div><div><br></div><div>In my case I trie= d to recover MariaDB database files, and some tar.gz file of backups of the= db that I accidentally=C2=A0deleted all=C2=A0together (mysql* instead of m= ysql/*=C2=A0 , my bad, deleted mysql/ & mysql_backups!!). I stopped all= services immediately, so maybe I would succeed if I wasn't time limite= d.</div><div><br></div><div>I understand=C2=A0its hard to undelete since no= one designed UFS/ZFS to do so.. that why I asked in later replies to see i= f someone would step in and implement such a "feature" and I sugg= ested=C2=A0some directions/thoughts.</div><div><br></div><div>As soren@ sug= gested in=C2=A0later reply it maybe would be easier to implement custom rm = script that moves files to "Recycle bin" directory (and empty it = after some period) but as a programmer I know that perfection is needed :) = so It might start as a simple task and end in many what-if's (unfortuna= ttly=C2=A0I did my last C programming in late=C2=A02003!).</div><div><br></= div><div>What amzes me is that this "feature" was asked too much = in the last decade or two and no one ever implemented it, maybe it's no= t needed in daily usage, but in disasters it would be super userful, save a= dmins many time and nerves..</div><div><br></div><div>For now I did some ba= ckup tools locally and used chflags to mark them undeletable so I wouldn= 9;t do that mistake again, plus I rsync them to my home storage.. so probab= ly I would be more resilent=C2=A0to such mistakes in the future.. but the s= ame problem remains.. accidently deleted file(s)/directory(s) are the night= mare of all admins=C2=A0in earth!!</div><div><br></div><div>Sami</div></div= ><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Su= n, Apr 17, 2022 at 12:42 AM Warner Losh <<a href=3D"mailto:imp@bsdimp.co= m">imp@bsdimp.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote"= style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);p= adding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class= =3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Apr 16, 2022= at 5:24 AM Sami Halabi <<a href=3D"mailto:sodynet1@gmail.com" target=3D= "_blank">sodynet1@gmail.com</a>> wrote:<br></div><blockquote class=3D"gm= ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,= 204,204);padding-left:1ex"><div dir=3D"ltr">Hi,<div>is there anyway easy to= restore deleted file by accident in UFS????</div></div></blockquote><div><= br></div><div>Do you know what the contents of the file is? At least the fi= rst, say, ~32k?</div><div><br></div><div>The problem with unrm for ufs is t= hat the directory entry has the inode number stored in it.</div><div>Withou= t the inode number, you won't get very far.</div><div>With the inode nu= mber, you can get the first 12 filesystem blocks of the file and the</div><= div>first three indirect blocks. Once you have those, you can reconstruct t= he file.</div><div><br></div><div>But only if the inode hasn't been zer= o'd out (which it likely has, another thing that makes</div><div>UFS un= delete harder). But all hope isn't lost...=C2=A0 UFS has a predictable = allocation algorithm</div><div>that lets you get much of the file back (whi= ch is why I asked if you know how it should start:</div><div>you can find w= here it starts in the data blocks and maybe get lucky with the rest if the<= /div><div>data spills into indirect blocks).</div><div><br></div><div>Howev= er, that's only if you don't have TRIM enabled on the filesystem. I= f you do,</div><div>then UFS will do a BIO_DELETE of the blocks, which mean= s their contents are</div><div>likely gone at the drive level. I say likely= because there's weasel words in the ATA</div><div>spec that allows a d= rive to return the prior contents of the blocks, or all zeros or</div><div>= the drive's initialization pattern (usually all 1's) when the block= s are later read. Same</div><div>goes for NVMe drives (with the additional = constraint it must be deterministic). So there's</div><div>may still be= a chance you can read the old contents, but drives that do that are rare</= div><div>in my experience (which is admittedly quite narrow).</div><div><br= ></div><div>But, if you want to use fsdb to try to recover this data, or wr= ite your own tools,</div><div>then you should likely have a copy of the dae= mon book (The Design and Implementation</div><div>of the FreeBSD Operating = System). It explains a lot of the finer details of UFS and</div><div>refere= nce to it likely will catch me where my memory isn't quite right in the= above</div><div>descriptions.</div><div><br></div><div>So, it's for al= l these reasons you can't find somebody with a unrm command for ufs</di= v><div>like you can for DOS or other filesystems. I wish I had a better ans= wer for you.</div><div><br></div><div>Warner</div><div>=C2=A0</div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Sami<br cle= ar=3D"all"><div><br></div>-- <br><div dir=3D"ltr"><div dir=3D"ltr"><div><di= v dir=3D"ltr">Sami Halabi<div>Information Systems Engineer</div><div>NMS Pr= ojects Expert,=C2=A0<span style=3D"font-size:12.8px">FreeBSD SysAdmin Exper= t</span></div><div>Asterisk Expert</div></div></div></div></div></div></div= > </blockquote></div></div> </blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"= class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr">Sami Hala= bi<div>Information Systems Engineer</div><div>NMS Projects Expert,=C2=A0<sp= an style=3D"font-size:12.8px">FreeBSD SysAdmin Expert</span></div><div>Aste= risk Expert</div></div></div></div></div> --00000000000055623b05dcccd574--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAEW%2BogY_PeZs0AKn%2BUY4gDk8Q8XmNR%2BO5u3Yte9tncXh2fTnDQ>