Date: Wed, 14 Dec 2016 18:15:40 +0100 From: Dirk-Willem van Gulik <dirkx@webweaving.org> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD Hackers <hackers@freebsd.org> Subject: Re: ZFS - directory entry Message-ID: <C3EE2E11-92FA-4CA0-A193-F6CECB3E232F@webweaving.org> In-Reply-To: <CAOtMX2i5_tL=jXcq2T5F2CkiQUVK9DoYofUsktNGXOHkmLaYRg@mail.gmail.com> References: <BEAC6EE9-C50F-4FB9-B215-D5A6691E2DD9@webweaving.org> <CAOtMX2i5_tL=jXcq2T5F2CkiQUVK9DoYofUsktNGXOHkmLaYRg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 14 Dec 2016, at 17:14, Alan Somers <asomers@freebsd.org> wrote: >=20 > On Wed, Dec 14, 2016 at 8:27 AM, Dirk-Willem van Gulik > <dirkx@webweaving.org> wrote: >> A rather odd directory entry (in /root, the home dir of root/toor) = appeared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine = under ZFS during/post a backup: >>=20 >> $ ls -la /root | tail -q >> ---------- 1 root wheel 9223372036854775807 Jan 1 1970 = ?%+?kD?H???x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux?= ???<T]???R??Qv?g???]??%?R? >>=20 >> OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART=E2= =80=99 nor smartd have reported anything. nothing in dmesg, syslog of = boot log. Any suggestions as how to debug or get to the root of this ? >>=20 >> And in particular - what is a risk of a reboot (to get a kernel with = debug, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence = stopping the forensic ? >>=20 >> Dw. >>=20 >> sudo zpool list -v >> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT >> tank 25.2T 9.27T 16.0T - 17% 36% 1.53x = ONLINE - >> raidz3 25.2T 9.27T 16.0T - 17% 36% >> ada0p3 - - - - - - >> ada1p3 - - - - - - >> ada2p3 - - - - - - >> ada3p3 - - - - - - >> ada4p3 - - - - - - >> ada5p3 - - - - - - >> ada6p3 - - - - - - Most wonderful - I did not know about the inode/zdb magic. Thanks! > Two things to try: > 1) zpool scrub. This will reveal any corrupt metadata objects Ok - some 300 hours to go :) So am now trying figure out why it is = running at just 8M/s (prefetch_disable=3D1, vfs.zfs.scrub_delay=3D0). > 2) Maybe the filename is created in an encoding not supported by your > current terminal. Try "LANG=3Den_US.UTF-8 ls -l=E2=80=9D No cookie (and not overly likely - barebone install which is not visible = to anything =E2=80=98modern=E2=80=99 but ssh et.al.). > 3) Use zdb to examine the file. First, do "ls -li /root" to get the > object id. It's the same as the inode number. Then, assuming /root > is in the tank/root filesystem, do "zdb -ddddd tank/root <object id>". > That might reveal some clues. A: zdb -ddddd tank/root 7426414 gives; after a 50 second pause (pre/during =E2=80=98zpool scrub=E2=80=99)= : Dataset tank/root [ZPL], ID 40, cr_txg 6, 902M, 14669 objects, = rootbp DVA[0]=3D<0:4c000:4000> DVA[1]=3D<0:4c00004c000:4000> [L0 DMU = objset] fletcher4 uncompressed LE contiguous unique double = size=3D800L/800P birth=3D225L/225P fill=3D14669 = cksum=3D9c7252c3b:ad096bfa68f:7b6298f1d2648:4235b444c02eba0 Object lvl iblk dblk dsize lsize %full type zdb: dmu_bonus_hold(7426414) failed, errno 2 So I guess I should wait for the scrub to complete. Cannot recall scrub = to be this slow. Dw.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C3EE2E11-92FA-4CA0-A193-F6CECB3E232F>