Date: Wed, 22 Jul 2015 15:47:12 +0300 From: Dmitry Sivachenko <trtrmitya@gmail.com> To: hackers@freebsd.org Subject: Re: Strange memory management with mmap() Message-ID: <15DE3B94-3C09-4855-A274-D5655B049403@gmail.com> In-Reply-To: <3434ED75-7994-4E9E-9B06-FACCD7DC90FF@gmail.com> References: <FDB6E0F9-A3FF-4194-83C1-A3121CBAE407@gmail.com> <3434ED75-7994-4E9E-9B06-FACCD7DC90FF@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 16 =D0=B8=D1=8E=D0=BB=D1=8F 2015 =D0=B3., at 21:19, Dmitry = Sivachenko <trtrmitya@gmail.com> wrote: >=20 >>=20 >> On 16 =D0=B8=D1=8E=D0=BB=D1=8F 2015 =D0=B3., at 18:42, Dmitry = Sivachenko <trtrmitya@gmail.com> wrote: >>=20 >> Hello! >>=20 >> I am using FreeBSD-10-stable and writing a program that uses large = data file via mmap() in read only mode. >> To be specific, I have 256GB RAM machine and typical size of data = file is ~160GB (more than 1/2 of RAM and less that the whole RAM). >> There is no other programs running during the test. >>=20 >> Consider the following use case: I have two files on disk. I mmap() = the first one and prefetch data to RAM (touch every page of the file). >> After that I expect all data to be cached in RAM and subsequent = access will be fast. >>=20 >> Next I do munmap() on the first file, mmap() the second one and do = the same test: prefetch data and expect it to be cached in RAM (and some = of the pages belonging to the first file to be purged out, because = size_of(file1)+size_of(file2) > size_of(RAM). >>=20 >> Please find my test program attached. >>=20 >> I run the program with 2 files provided via command line (both about = 160GB). >> What I observe in real is: >> -- before I run the program all RAM is in FREE state as reported by = top(1). >> -- after first prefetch() of the first file, all it's data goes to = "Cache" state, RES column of the process remains the same (small) >> -- second prefetch() works fast as expected, memory goes from Cache = to Active state, RES column of the process grows up to match file size = (SIZE=3D=3DRES now) >> -- now first prefetch() for second file starts: the remaining Free = memory goes to Cache state, Active size still equals to first file size. >> -- second prefetch() for second file works as slow as first one, like = if nothing was cached in memory during the first prefetch() run, RES = column does not change. >>=20 >>=20 >> Here is the output: >> % /tmp/a.out file1.dat file2.dat >> file1.dat... First prefault time: 1235.747351 seconds >> Second prefault time: 74.893323 seconds >> Ok. >> file2.dat... First prefault time: 1316.405527 seconds >> Second prefault time: 1311.491842 seconds >> Ok. >>=20 >=20 >=20 I tried the same test program on Linux machine with similar hardware. = It behaves like expected (second prefetch works very fast): file1.dat... First prefault time: 2664.621088 seconds Second prefault time: 1.969283 seconds Ok. file2.dat... First prefault time: 2917.009003 seconds Second prefault time: 34.128762 seconds Ok.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15DE3B94-3C09-4855-A274-D5655B049403>