Date: Wed, 04 Jul 2012 11:07:36 +0300 From: "Pavlo" <devgs@ukr.net> To: "Pavlo" <devgs@ukr.net> Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) Message-ID: <23856.1341389256.6316487571580649472@ffe17.ukr.net> In-Reply-To: <91943.1339669820.1305529125424791552@ffe15.ukr.net> References: <91943.1339669820.1305529125424791552@ffe15.ukr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--- Original message --- From: "Pavlo" <devgs@ukr.net> To: freebsd-fs@freebsd.org Date: 14 June 2012, 13:30:20 Subject: mmap() incoherency on hi I/O load (FS is zfs) > There's a case when some parts of files that are mapped and then modified getting corrupted. By corrupted I mean some data is ok (one that was written using write()/pwrite()) but some looks like it never existed. Like it was some time in buffers, when several processes simultaneously (of course access was synchronised) used shared pages and reported it's existence. But after time pass they (processes) screamed that it is now lost. Only part of data written with pwrite() was there. Everything that was written via mmap() is zero. > > So as I said it occurs on hi I/O busyness. When in background 4+ processes do indexing of huge ammount of data. Also I want to note, it never occurred in the life of our project while we used mmap() under same I/O stress conditions when mapping was done for a whole file of just a part(header) starting from a beginning of a file. First time we used mapping of individual pages, just to save RAM, and this popped up. > > Solution for this problem is msync() before any munmap(). But man says: > > The msync() system call is usually not needed since BSD implements a coherent file system buffer cache. However, it may be used to associate dirty VM pages with file system buffers and thus cause them to be flushed to physical media sooner rather than later. > > Any thoughts? Thanks. > > So I tracked issue to the place where it occurs. When I commit data to file using mmap() and pwrite() side by side, sometimes 'newest data' is being overwritten by 'elder data'. From time to time 'elder data' can be something written with mmap() either with pwrite(). It never happens when I use exclusively mmap() either pwrite(). Also this issue reproduces on UFS as well. I think there is a problem keeping mmapep pages and FS cache synced. I will try to make test to reliably reproduces issue.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?23856.1341389256.6316487571580649472>