From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 15 11:33:31 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB34C15D; Sun, 15 Feb 2015 11:33:31 +0000 (UTC) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44B1ACDA; Sun, 15 Feb 2015 11:33:31 +0000 (UTC) Received: by mail-lb0-f169.google.com with SMTP id p9so22689794lbv.0; Sun, 15 Feb 2015 03:33:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/xA7Wyi3rxM3xLz3MrYJsIX92LBhhk75SgfvfyCFLz4=; b=a3J6KwESWG3wq3kKAz/HSC9cG7lTNZXy5lnv1n3V4RjafoZCL84kmHkRxswVgymkdH a7F/ugVLIA0QUy1pFNWOpvx4A4lpNI2ZqeMnx4E5Xd3howdrzgMZU4nUkddZUGuMBqxU /QLARQNz2vcvu2sNhfSQDRUr6KmaLHDBdu0mmpnG+Xf4gjol2kwG+hFqFb2/lDPR2t9K GnXzmaJ7VRzAXA11umlxgB65RJz1w5V9B/9PPwnlkbXknsX2aWxPOSl01mUB35NQFAQQ I3IREANEGS7Srng05cA/N+x6RTWYihkTubr+db9uz35CS5C77Vadxl7IYyrwmS09X8eM iKAA== X-Received: by 10.112.44.171 with SMTP id f11mr16971461lbm.65.1424000008752; Sun, 15 Feb 2015 03:33:28 -0800 (PST) Received: from [10.0.1.7] (broadband-5-228-253-252.nationalcablenetworks.ru. [5.228.253.252]) by mx.google.com with ESMTPSA id m8sm2365491laj.29.2015.02.15.03.33.26 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 15 Feb 2015 03:33:27 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: mmap() question From: Dmitry Sivachenko In-Reply-To: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> Date: Sun, 15 Feb 2015 14:33:24 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> To: "hackers@freebsd.org" X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Feb 2015 11:33:31 -0000 > On 9 =D0=BE=D0=BA=D1=82. 2013 =D0=B3., at 15:42, Dmitry Sivachenko = wrote: >=20 > Hello! >=20 > I have a program which mmap()s a lot of large files (total size more = that RAM and I have no swap), but it needs only small parts of that = files at a time. >=20 > My understanding is that when using mmap when I access some memory = region OS reads the relevant portion of that file from disk and caches = the result in memory. If there is no free memory, OS will purge = previously read part of mmap'ed file to free memory for the new chunk. >=20 > But this is not the case. I use the following simple program which = gets list of files as command line arguments, mmap()s them all and then = selects random file and random 1K parts of that file and computes a XOR = of bytes from that region. > After some time the program dies: > pid 63251 (a.out), uid 1232, was killed: out of swap space There is another way to observe incorrect "out of swap space" process = kills I encountered yesterday (fresh 10/stable). I have a program which does mmap()+mlock() on ~200GB data file (total = RAM is 256GB). The goal was to update this data file to new version. I copied new file = while the program was running with old one, stopped program and run it = again with new data file (old file still on disk). On the first invocation, program crashed with "out of swap space" error. = I run it again, it crash again. On the 3rd try it started successfully. I suspect the reason is the same: there were a lot of "Active" memory = left after I stopped program and when program started to mlock new file, = memory pages corresponding to old mmap were still using RAM. If an old data file was deleted before I starting the program with new = one, these "Active" memory pages would become "Free" and crash would not = occur. >=20 > It seems I incorrectly understand how mmap() works, can you please = clarify what's going wrong? >=20 > I expect that program to run indefinitely, purging some regions out of = RAM and reading the relevant parts of files. >=20 > Thanks! >=20 > #include > #include > #include > #include > #include > #include > #include > #include > #include >=20 > struct f_data { > char *beg; > off_t size; > }; >=20 > int > main(int argc, char* argv[]) { > if (argc < 2) { > fprintf(stderr, "Usage: %s ...\n", argv[0]); > exit(0); > } > int i, j, fd; > struct stat st; > struct f_data FILES[500]; > int NUM_FILES; > void *p; > NUM_FILES =3D argc - 1; > for (i=3D1; i < argc; i++) { > printf("%s... ", argv[i]); > if ((fd =3D open(argv[i], O_RDONLY)) < 0) > errx(1, "open"); > if (fstat(fd, &st) !=3D 0) > errx(1, "fstat"); > if ((p =3D mmap(NULL, st.st_size, PROT_READ, MAP_NOCORE, fd, = 0)) =3D=3D MAP_FAILED) > errx(1, "mmap"); > FILES[i-1].beg =3D (char*)p; > FILES[i-1].size =3D st.st_size; > if (msync(p, st.st_size, MS_INVALIDATE) !=3D 0) > errx(1, "msync"); > printf("Ok.\n"); > } > char chk =3D 0; > while(1) { > int rf =3D floor((double)random() / 2147483647 * NUM_FILES); > off_t offs =3D floor((double)random() / 2147483647 * = (FILES[rf].size - 1024)); > for (j=3D0; j<1024; j++) > chk ^=3D *(FILES[rf].beg + offs + j); > } > return 0; > } >=20