Date: Thu, 15 Mar 2007 10:26:59 +0100 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: Kostik Belousov <kostikbel@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: FFS writes to read-only mount Message-ID: <20070315092659.GA14080@garage.freebsd.pl> In-Reply-To: <20070315090031.GB80993@deviant.kiev.zoral.com.ua> References: <45F776AE.8090702@nokia.com> <20070314161041.GI7847@garage.freebsd.pl> <45F8EE27.6070208@nokia.com> <20070315090031.GB80993@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
--cWoXeonUoKmBZSoM Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 15, 2007 at 11:00:31AM +0200, Kostik Belousov wrote: > On Thu, Mar 15, 2007 at 04:56:39PM +1000, David Cecil wrote: > > Hi Pawel, > >=20 > > here is what I've found. Note that this problem doesn't occur=20 > > frequently, but does happen often enough to be a problem. I am still= =20 > > debugging the live system. > >=20 > > I put breakpoints in g_print_bio and g_io_request. I then continued=20 > > until the breakpoint at g_print_bio breakpoint was hit, but each time= =20 > > g_io_request was hit, I printed the backtrace. Then, at g_print_bio I= =20 > > could identify the struct bio * and look for the corresponding=20 > > g_io_request trace with a matching struct bio *. > >=20 > > That trace looks like this: > > db> bt > > Tracing pid 47 tid 100050 td 0x860d0640 > > g_io_request(864258c4,860ec440,a0020024,98b92188,a5281bf0,...) at=20 > > g_io_request+0x107 > > g_vfs_strategy(86104ce4,98b92188,a0020024,98b92188,86104c30,...) at=20 > > g_vfs_strategy+0x63 > > ffs_geom_strategy(86104ce4,98b92188) at ffs_geom_strategy+0x129 > > bufwrite(98b92188,4000,98b92188,a5281c54,805b20c1,...) at bufwrite+0x146 > > ffs_bufwrite(98b92188) at ffs_bufwrite+0x282 > > vfs_bio_awrite(98b92188) at vfs_bio_awrite+0x221 > > vop_stdfsync(a5281cbc,a5281cbc,a5281c98,806eff3e,a5281cbc,...) at=20 > > vop_stdfsync+0x121 > It would be helpful, at this frame, to print the actual vnode, as well as > the dirty buffer header together with dump of buffer data. [...] This comes from devfs_fsync(), so I suspect it's devvp, and we are synchronizing entire file system here. > [...] It seems that > the easiest way to do that would be to obtain coredump and then use kgdb. On successfull remount to read-only we could walk (under DIAGNOSTIC) the list of vnodes and assert there are no dirty buffers, but I don't think this is the case... The thing that worries me is that it doesn't happen from what I understand just after remount to read-only. Syncer wakeups every 30 seconds by default, right? So it should generates errors on first time he synchronizes file system. Which bascially means, that something tries to write to read-only file system later. Will be good to assert it somehow earlier. One thing that comes to my mind is to assert: KASSERT(((struct g_consumer *)buf->b_bufobj.bo_private)->acw > 0); in each b*write() function. This is probably not the only way to write to a file system... I still don't feel too familiar with the code, so not sure if my comments are usable:) how-to-reproduce procedure would be the best of course... David, do you use something which can be considered non-standard, like snapshots? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFF+RFjForvXbEpPzQRAvYiAJ0ZU4xcQbTA3uh4g/DGPrWoVdiHLgCgmK1y hWJIJkQGGSwy3KHE4zlnPCY= =Wcdo -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070315092659.GA14080>