Date: Fri, 8 Jul 2016 21:43:12 -0400 From: David Cross <dcrosstech@gmail.com> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-stable@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Reproducable panic in FFS with softupdates and no journaling (10.3-RELEASE-pLATEST) FOUND IT, including reproduction steps Message-ID: <CAM9edeNYDPhq3E3zxm8zmoiRcfPp=87xF2veUVP_jPGj4a67fA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Ok... I found it. All of the writes go through ffs_write (including VOP_RECLAIM, so my statement that VOP_RECLAIM couldn't handle things that vinvalbuf left behind is obviously incorrect). Sometimes it worked, sometimes it paniced, I started putting more deugging into it and I noticed the following: The problem file would balloc twice as follows: attempting to balloc inode 18237205 softdep_setup_allocdirect(18237205, 1, 72834400, 0, 8192, 0, 0xfffffe00f76a6d88) balloc at 291337600, flags: 50000 attempting to balloc inode 18237205 softdep_setup_allocdirect(18237205, 0, 72834448, 0, 16384, 0, 0xfffffe00f7749970) balloc at 291337792, flags: 7f040080 panic: softdep_deallocate_dependencies: dangling deps Furthrer reading of ffs_write to figure out why it worked sometimes and not others pointed me at the IO_SYNC flag, if passed in ffs_write dispatches to bwrite.. which gives the panic, otherwise it goes to bawrite which does not. However the problem is in ufs_balloc, around line 778 (which I saw in the earlier newbuf dump); There NO call to any write method for that buffer. If we compare this to the other calls to softdep_setup_allocdirect in that function (lines: 148, 264, 708, 828) we see that each of them has some call to bwrite, bdwrite, bawrite following it (a number of the other calls do not make any direct calls to b*writes either, I do not know nearly enough to say if those are correct or incorrect; I tried adding bwrite arround those lines with a conditional on IO_SYNC and I only made it panic earlier. I just don't know what the semantics of this enough. That being said, I was finally able to isolate a set of reproduction steps that anyone can run. As it stands it relies on a set of filesystem options that are no longer standard (but were, not that long ago), but I definitely believe they could be trivially modified to work on *any* UFS1/UFS2 filesystem... To that extent I am NOT including them, I will reply individually with the exploit code an instructions to reproduce; if you want, and you have an appropriate commit history or other credentials I will forward it on. Thanks, and I eagerly look forward to the patch, or assisting where I can in development.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM9edeNYDPhq3E3zxm8zmoiRcfPp=87xF2veUVP_jPGj4a67fA>