Date: Sun, 15 Feb 2015 14:33:24 +0300 From: Dmitry Sivachenko <trtrmitya@gmail.com> To: "hackers@freebsd.org" <hackers@freebsd.org> Cc: Konstantin Belousov <kib@FreeBSD.org> Subject: Re: mmap() question Message-ID: <E84D65C4-6E90-4C65-925C-F5239A612C4B@gmail.com> In-Reply-To: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 9 =D0=BE=D0=BA=D1=82. 2013 =D0=B3., at 15:42, Dmitry Sivachenko =
<trtrmitya@gmail.com> wrote:
>=20
> Hello!
>=20
> I have a program which mmap()s a lot of large files (total size more =
that RAM and I have no swap), but it needs only small parts of that =
files at a time.
>=20
> My understanding is that when using mmap when I access some memory =
region OS reads the relevant portion of that file from disk and caches =
the result in memory. If there is no free memory, OS will purge =
previously read part of mmap'ed file to free memory for the new chunk.
>=20
> But this is not the case. I use the following simple program which =
gets list of files as command line arguments, mmap()s them all and then =
selects random file and random 1K parts of that file and computes a XOR =
of bytes from that region.
> After some time the program dies:
> pid 63251 (a.out), uid 1232, was killed: out of swap space
There is another way to observe incorrect "out of swap space" process =
kills I encountered yesterday (fresh 10/stable).
I have a program which does mmap()+mlock() on ~200GB data file (total =
RAM is 256GB).
The goal was to update this data file to new version. I copied new file =
while the program was running with old one, stopped program and run it =
again with new data file
(old file still on disk).
On the first invocation, program crashed with "out of swap space" error. =
I run it again, it crash again.
On the 3rd try it started successfully.
I suspect the reason is the same: there were a lot of "Active" memory =
left after I stopped program and when program started to mlock new file, =
memory pages corresponding to old mmap were still using RAM.
If an old data file was deleted before I starting the program with new =
one, these "Active" memory pages would become "Free" and crash would not =
occur.
>=20
> It seems I incorrectly understand how mmap() works, can you please =
clarify what's going wrong?
>=20
> I expect that program to run indefinitely, purging some regions out of =
RAM and reading the relevant parts of files.
>=20
> Thanks!
>=20
> #include <err.h>
> #include <fcntl.h>
> #include <math.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/mman.h>
> #include <sys/stat.h>
> #include <sys/types.h>
> #include <unistd.h>
>=20
> struct f_data {
> char *beg;
> off_t size;
> };
>=20
> int
> main(int argc, char* argv[]) {
> if (argc < 2) {
> fprintf(stderr, "Usage: %s <file> ...\n", argv[0]);
> exit(0);
> }
> int i, j, fd;
> struct stat st;
> struct f_data FILES[500];
> int NUM_FILES;
> void *p;
> NUM_FILES =3D argc - 1;
> for (i=3D1; i < argc; i++) {
> printf("%s... ", argv[i]);
> if ((fd =3D open(argv[i], O_RDONLY)) < 0)
> errx(1, "open");
> if (fstat(fd, &st) !=3D 0)
> errx(1, "fstat");
> if ((p =3D mmap(NULL, st.st_size, PROT_READ, MAP_NOCORE, fd, =
0)) =3D=3D MAP_FAILED)
> errx(1, "mmap");
> FILES[i-1].beg =3D (char*)p;
> FILES[i-1].size =3D st.st_size;
> if (msync(p, st.st_size, MS_INVALIDATE) !=3D 0)
> errx(1, "msync");
> printf("Ok.\n");
> }
> char chk =3D 0;
> while(1) {
> int rf =3D floor((double)random() / 2147483647 * NUM_FILES);
> off_t offs =3D floor((double)random() / 2147483647 * =
(FILES[rf].size - 1024));
> for (j=3D0; j<1024; j++)
> chk ^=3D *(FILES[rf].beg + offs + j);
> }
> return 0;
> }
>=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E84D65C4-6E90-4C65-925C-F5239A612C4B>