From owner-freebsd-stable@freebsd.org Sat Dec 17 21:09:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D256EC85B9C for ; Sat, 17 Dec 2016 21:09:21 +0000 (UTC) (envelope-from david.marec@davenulle.org) Received: from smtp.smtpout.orange.fr (smtp02.smtpout.orange.fr [80.12.242.124]) by mx1.freebsd.org (Postfix) with ESMTP id 7793B188B for ; Sat, 17 Dec 2016 21:09:20 +0000 (UTC) (envelope-from david.marec@davenulle.org) Received: from dmarec.local ([109.214.21.61]) by mwinf5d04 with ME id M91a1u00C1K4e8l0391a9q; Sat, 17 Dec 2016 22:01:40 +0100 X-ME-Helo: dmarec.local X-ME-Date: Sat, 17 Dec 2016 22:01:40 +0100 X-ME-IP: 109.214.21.61 To: freebsd-stable From: David Marec Subject: [ZFS] files in a weird situtation Message-ID: <914db6e5-2d8d-15b4-ee96-bc267ce1dbda@davenulle.org> Date: Sat, 17 Dec 2016 22:01:34 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 21:09:21 -0000 [I had first posted onto the Forum about this issue] Two months ago, - next to a call to |`||delete-old-libs`| or `install world`, I don't really know - my box that is following FreeBSD-11 Stable ran into a weird situation. A set of files, especially `/lib/libjail.so.1` are in both states `existing` and `not existing`: I means: david:~>cp ~david/libjail.so.1 /lib cp: /lib/libjail.so.1: File exists But: david:~>ls /lib/libjail.so.1 ls: /lib/libjail.so.1: No such file or directory david:~>find /lib -name "libjail.so.1" -print /lib/libjail.so.1 david:~>find /lib -name "libjail.so.1" -ls find: /lib/libjail.so.1: No such file or directory With deeper investigation, the file is in fact mapped to an `inode`: root@dmarec:~ # ls -di /lib 13 /lib root@dmarec:~ # zdb -dddd zroot/ 13 | grep libjail.so.1 libjail.so.1 = 10552574 (type: Regular File) Which fails with `zdb` on: root@dmarec:~ # zdb -dddd zroot/ 10552574 Dataset zroot [ZPL], ID 21, cr_txg 1, 114G, 2570002 objects, rootbp DVA[0]=<0:b97d6ea00:200> DVA[1]=<0:1c212b0400:200> [L0 DMU objset] fletcher4 lz4 LE contiguous unique double size=800L/200P birth=3852240L/3852240P fill=2570002 cksum=17b78fb7e4:7c87a526a07:16251edfaae60:2ce0c5734ccf2f Object lvl iblk dblk dsize lsize %full type zdb: dmu_bonus_hold(10552574) failed, errno 2 `stat (2)` returns ENOENT when checking for the file: david:~>truss stat -L /lib/libjail.so.1 ... stat("/lib/libjail.so.1",0x7fffffffe7e8) ERR#2 'No such file or directory' david:~>truss stat -L /lib/libjail.so.1 A pass with `zfs scrub` didn't help. Any clue is welcome. What's that `dmu_bonus_hold` stands for ? -- David Marec https://lapinbilly.eu