Date: Wed, 08 Oct 2014 12:03:21 -0400 From: Richard Yao <ryao@gentoo.org> To: Dmitry Sivachenko <trtrmitya@gmail.com>, "hackers@freebsd.org" <hackers@freebsd.org> Subject: Re: mmap() question Message-ID: <54356049.9040009@gentoo.org> In-Reply-To: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qbfRIPaAgtlsTVMQ2WRRaraT7hWhDM8am Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable It might be worthwhile to use madvise() in your code: http://man7.org/linux/man-pages/man2/madvise.2.html On 10/09/2013 07:42 AM, Dmitry Sivachenko wrote: > Hello! >=20 > I have a program which mmap()s a lot of large files (total size more th= at RAM and I have no swap), but it needs only small parts of that files a= t a time. >=20 > My understanding is that when using mmap when I access some memory regi= on OS reads the relevant portion of that file from disk and caches the re= sult in memory. If there is no free memory, OS will purge previously rea= d part of mmap'ed file to free memory for the new chunk. >=20 > But this is not the case. I use the following simple program which get= s list of files as command line arguments, mmap()s them all and then sele= cts random file and random 1K parts of that file and computes a XOR of by= tes from that region. > After some time the program dies: > pid 63251 (a.out), uid 1232, was killed: out of swap space >=20 > It seems I incorrectly understand how mmap() works, can you please clar= ify what's going wrong? >=20 > I expect that program to run indefinitely, purging some regions out of = RAM and reading the relevant parts of files. >=20 > Thanks! >=20 > #include <err.h> > #include <fcntl.h> > #include <math.h> > #include <stdio.h> > #include <stdlib.h> > #include <sys/mman.h> > #include <sys/stat.h> > #include <sys/types.h> > #include <unistd.h> >=20 > struct f_data { > char *beg; > off_t size; > }; >=20 > int > main(int argc, char* argv[]) { > if (argc < 2) { > fprintf(stderr, "Usage: %s <file> ...\n", argv[0]); > exit(0); > } > int i, j, fd; > struct stat st; > struct f_data FILES[500]; > int NUM_FILES; > void *p; > NUM_FILES =3D argc - 1; > for (i=3D1; i < argc; i++) { > printf("%s... ", argv[i]); > if ((fd =3D open(argv[i], O_RDONLY)) < 0) > errx(1, "open"); > if (fstat(fd, &st) !=3D 0) > errx(1, "fstat"); > if ((p =3D mmap(NULL, st.st_size, PROT_READ, MAP_NOCORE, fd, 0)= ) =3D=3D MAP_FAILED) > errx(1, "mmap"); > FILES[i-1].beg =3D (char*)p; > FILES[i-1].size =3D st.st_size; > if (msync(p, st.st_size, MS_INVALIDATE) !=3D 0) > errx(1, "msync"); > printf("Ok.\n"); > } > char chk =3D 0; > while(1) { > int rf =3D floor((double)random() / 2147483647 * NUM_FILES); > off_t offs =3D floor((double)random() / 2147483647 * (FILES[rf]= =2Esize - 1024)); > for (j=3D0; j<1024; j++) > chk ^=3D *(FILES[rf].beg + offs + j); > } > return 0; > } >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 --qbfRIPaAgtlsTVMQ2WRRaraT7hWhDM8am Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJUNWBOAAoJECDuEZm+6ExkNX0P/iyUJNHnAXcztp3DoMxlDkKX tatK+LBM+QL/+hFcIE4t4UDdSZ3g5+JN4lCCShuFZHH8UtJcU8BwLmAJhlsBlE2q d3o40SO96eNkelw1ymgBNdzFhb44/PjAy9DkWDMV1tlWX0oi2BTqmeEyf908rASJ odWh42+YwiLRPo+iTAqebIxix4jP9P9mXWEXjbHsNZhrpI6Oqu/povAUaxWMBpy+ EoXagg7trzJ3JCLU3rhamo/bOLivLmZsWZlAAYkAxiy6JArUs3wHtL43A0xsn6z8 xuY3dBcCyGXawWkz9nR8+Z1ydy7xoVH/S1UPsiZiHd8z/exGwQn6S7YGBuYTmf6u CJscCFEo9WV+U7f51FI71zeNEnNBxRoZBg+ePVDurhmdUge4/2k3gp2GKJ0SkHWf sMgQ2UNJ+fYfTDSDboIeMlwiuhuXaS3iYZkAu2SlBNdhczjSCj7IfhZHbPfaZt3C pr1uiBGF/txKWLC6YT6qNHv5oG+TtamJcs6DHPExXvCAx3vQN0C/yA9uVg75ZxCP Xo0HgJALAmzqzjJ4wsB+zWMLvU1dC3lfIf7uxEQvWamX0/wpQGH+DuTM5GLBq0Lg wUeXD4MxZT05EOyJMfFK/NR3lbaTTxmil9CQmu3VrkxNw+jEJboBdmfdigcLNC9B 6XG46PcYQsSWHn2cWWvA =cJ1L -----END PGP SIGNATURE----- --qbfRIPaAgtlsTVMQ2WRRaraT7hWhDM8am--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54356049.9040009>