Date: Sun, 10 Jan 2016 17:45:18 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: panic ffs_truncate3 (maybe fuse being evil) Message-ID: <20160110154518.GU3625@kib.kiev.ua> In-Reply-To: <1696608910.154845456.1452438117036.JavaMail.zimbra@uoguelph.ca>
index | next in thread | previous in thread | raw e-mail
On Sun, Jan 10, 2016 at 10:01:57AM -0500, Rick Macklem wrote: > Hi, > > When fooling around with GlusterFS, I can get this panic intermittently. > (I had a couple yesterday.) This happens on a Dec. 5, 2015 head kernel. > > panic: ffs_truncate3 > - backtrace without the numbers (I just scribbled it off the screen) > ffs_truncate() > ufs_inactive() > VOP_INACTIVE_APV() > vinactive() > vputx() > kern_unlinkat() > > So, at a glance, it seems that either > b_dirty.bv_cnt > or b_clean.bv_cnt > is non-zero. (There is another case for the panic, but I thought it > was less likely?) > > So, I'm wondering if this might be another side effect of r291460, > since after that a new vnode isn't completely zero'd out? > > However, shouldn't bo_dirty.bv_cnt and bo_clean.bv_cnt be zero when > a vnode is recycled? > Does this make sense or do some fields of v_bufobj need to be zero'd > out by getnewvnode()? Look at the _vdrop(). When a vnode is freed to zone, it is asserted that bufobj queues are empty. I very much doubt that it is possible to leak either buffers or counters by reuse. > > GlusterFS is using fuse and I suspect that fuse isn't cleaning out > the buffers under some circumstance (I already noticed that there > isn't any code in its fuse_vnop_reclaim() and I vaguely recall that > there are conditions where VOP_INACTIVE() gets skipped, so that VOP_RECLAIM() > has to check for anything that would have been done by VOP_INACTIVE() > and do it, if it isn't already done.) But even if fuse leaves the buffers around, is it UFS which panics for you ? I would rather worry about dandling pointers and use after free in fuse, which is a known issue with it anyway. I.e. it could be that fuse operates on reclaimed and reused vnode as its own. > > Anyhow, if others have thoughts on this (or other hunches w.r.t. what > could cause this panic(), please let me know. The ffs_truncate3 was deterministically triggered by a bug in ffs_balloc(). The routine allocated buffers for indirect blocks, but if the blocks cannot be allocated, the buffers where left on queue. See r174973, this was fixed very long time ago.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160110154518.GU3625>