Date: Thu, 18 Oct 2012 11:35:37 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Tristan Verniquet <tris_vern@hotmail.com> Cc: freebsd-hackers@freebsd.org Subject: Re: syncing large mmaped files Message-ID: <20121018083537.GQ35915@deviant.kiev.zoral.com.ua> In-Reply-To: <SNT124-W20F26CF7B468F7F09B9B4983760@phx.gbl> References: <SNT124-W20F26CF7B468F7F09B9B4983760@phx.gbl>
next in thread | previous in thread | raw e-mail | index | archive | help
--XA7quakUSnawneuz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 18, 2012 at 10:08:22AM +1000, Tristan Verniquet wrote: >=20 > I want to work with large (1-10G) files in memory but eventually sync > them back out to disk. The problem is that the sync process appears to > lock the file in kernel for the duration of the sync, which can run > into minutes. This prevents other processes from reading from the file > (unless they already have it mapped) for this whole time. Is there > any way to prevent this? I think I read in a post somewhere about > openbsd implementing partial-writes when it hits a file with lots of > dirty pages in order to prevent this. Is there anything available for > FreeBSD or is there another way around it? > No, currently the vnode lock is held exclusive for the whole duration of the msync(2) syscall or its analog from the syncer. Making a change to periodically drop the vnode lock in vm_object_page_clean() might be possible, but requires the benchmarking to make sure that we do not pessimize the common case. Also, this opens a possibility for the vnode reclamation meantime. Anyway, note that you cannot 'work with large files in memory', even if you have enough RAM and no pressure to hold all the file pages resident. The syncer will do a writeback periodically regardless of the application calling msync(2) or not, with the interval of approximately 30 seconds. --XA7quakUSnawneuz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlB/v1kACgkQC3+MBN1Mb4joagCgj1oYiDMQjM9s9kK7HniP4JiL RVEAn1294Rq3lIUMnPdt2G2ue1z3Jppa =Z1TH -----END PGP SIGNATURE----- --XA7quakUSnawneuz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121018083537.GQ35915>