From owner-freebsd-arm@freebsd.org Sat Aug 4 16:09:01 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45D9B106FCC8 for ; Sat, 4 Aug 2018 16:09:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-34.consmr.mail.ne1.yahoo.com (sonic317-34.consmr.mail.ne1.yahoo.com [66.163.184.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BF59C7E0BF for ; Sat, 4 Aug 2018 16:09:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: BnOy83IVM1mU.ClszuG6XgfYiRN0iQWMDNlnfcnccchJawk.dj1lw4SvlFYBOV5 y6a7ymeizrbciLsOcszTFE62s4152oR6f7RWcAb7I2hlH50rzEwRDTNCQvIEHdEROpFuPu3Zb_RM aIohwUfbT3I3SwylhlX9dypsCoaNAJ2XM3yddWozt1MatCsgaqm66F11a50io0FNPcM.pUXiwJ7Y HvjThH7ev_Proob_e7vbm7IMthgyzEIJ0Vrs1xdYsl8YErkvarXeHDqyMEf5JTO3qMZhO2gGO.VE twWtBW7F1fj.t.D7zBMTTsEg8.RgsYUisYlKewUydKIZQ2ISO3WKfRfMbp_Rxs4W1VlKsFQhFOnm mEfPG4Z6DgTMpqOOzM8ltCNJhzgFxEFnVr9rkUnJGXcVm50_q4yDLwFQ5tFR7MRzUuMrejvOvG9D dW0cyjeo9rULFGjvclRvtl8H7uk7YjqgZHIufnX_o53Z4gaECOLF_aUxhA5yJUqs70mb2Op.u750 pEETF4_q.ckhKoEjTB4777Cg2X3hxIVOLg39BmILhdRy.5ocW5XMvCzUpF9VIvjYgtJBj7l6wYOG eUDD1jOKAFKiB2fG0JtK_ldPe0wcGG37vbTkhimXebvp7CtiIPA9SnGB8YMHfCV6gKFboMj9y8Xk hZCnVH2ei6TERKR8Pk4bmWc_RIcGa6DFrTF5fITtswd9B75b15Cj0mPiwJ922Eghlg6g9VWmHEUk VZ3BK13g.rjlpzw15gx56UqA7gv_LFH2vbM7KHQqplz6cSKLJpShMlj1ps76IaIQ5sljqPx4Ohr3 AY3qG64BTz4DrXaZueHOytQwOFsE7awzXrrLWicM_mXgUfVQDrGuIZ3F62xEYByg82OCVFwu15eL ke.yfNvw9IoNFdjHPB84S93.0rppph491YTMPQj8zCuhLFjKesW7Cy.QNEKctLEq9T4jINpPI9lG mpqOTfeM026qdc7XuUEzHC6y3f4tUo8mNJjwL5naH7tbTh_ZBtBD4iHPwANQaWHpgnEoI4uUbUev dqV55m4lEllse Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Sat, 4 Aug 2018 16:08:54 +0000 Received: from ip70-189-131-151.lv.lv.cox.net (EHLO [192.168.0.105]) ([70.189.131.151]) by smtp412.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 272678282d01dead00e1abdbd4f10ee2; Sat, 04 Aug 2018 16:08:50 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: RPI3 swap experiments ["was killed: out of swap space" with: "v_free_count: 5439, v_inactive_count: 1"] From: Mark Millard In-Reply-To: <20180804140816.GJ2884@funkthat.com> Date: Sat, 4 Aug 2018 09:08:48 -0700 Cc: Jamie Landeg-Jones , bob prohaska , freebsd-arm , markj@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <16ABD9F0-C908-479C-960D-0C1AEDE89053@yahoo.com> References: <20180801034511.GA96616@www.zefox.net> <201808010405.w7145RS6086730@donotpassgo.dyslexicfish.net> <6BFE7B77-A0E2-4FAF-9C68-81951D2F6627@yahoo.com> <20180802002841.GB99523@www.zefox.net> <20180802015135.GC99523@www.zefox.net> <201808030034.w730YURL034270@donotpassgo.dyslexicfish.net> <201808040355.w743tPsF039729@donotpassgo.dyslexicfish.net> <8CC5DF53-F950-495C-9DC8-56FCA0087259@yahoo.com> <20180804140816.GJ2884@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 16:09:01 -0000 On 2018-Aug-4, at 7:08 AM, John-Mark Gurney wrote: > Mark Millard via freebsd-arm wrote this message on Sat, Aug 04, 2018 = at 00:14 -0700: >> On 2018-Aug-3, at 8:55 PM, Jamie Landeg-Jones = wrote: >>=20 >>> Mark Millard wrote: >>>=20 >>>> If Inact+Laundry+Buf(?)+Free was not enough to provide sufficient >>>> additional RAM, I'd would have guessed that some Active Real Memory >>>> should then have been paged/swapped out and so RAM would be made >>>> available. (This requires the system to have left itself sufficient >>>> room in RAM for that guessed activity.) >>>>=20 >>>> But I'm no expert at the intent or actual operation. >>>>=20 >>>> Bob P.'s reports (for having sufficient swap space) >>>> also indicate the likes of: >>>>=20 >>>> v_free_count: 5439, v_inactive_count: 1 >>>>=20 >>>>=20 >>>> So all the examples have: "v_inactive_count: 1". >>>> (So: vmd->vmd_pagequeues[PQ_INACTIVE].pq_cnt=3D=3D1 ) >>>=20 >>> Thanks for the feedback. I'll do a few more runs and other stress = tests >>> to see if that result is consistent. I'm open to any other idea too! >>>=20 >>=20 >> The book "The Design and Implementation of the FreeBSD Operating = System" >> (2nd edition, 2014) states (page labeled 296): >>=20 >> QUOTE: >> The FreeBSD swap-out daemon will not select a runnable processes to = swap >> out. So, if the set of runnable processes do not fit in memory, the >> machine will effectively deadlock. Current machines have enough = memory >> that this condition usually does not arise. If it does, FreeBSD = avoids >> deadlock by killing the largest process. If the condition begins to = arise >> in normal operation, the 4.4BSD algorithm will need to be restored. >> END QUOTE. >>=20 >> As near as I can tell, for the likes of rpi3's and rpi2's, the = condition >> is occurring during buildworld "normal operation" that tries to use = the >> available cores to advantage. (Your context does not have the I/O >> problems that Bob P.'s have had in at least some of your OOM process >> kill examples, if I understand right.) >>=20 >> (4.4BSD used to swap out the runnable process that had been resident >> the longest, followed by the processes taking turns being swapped = out. >> I'll not quote the exact text about such.) >>=20 >> So I guess the question becomes, is there a reasonable way to enable >> the 4.4BSD style of "Swapping" for "small" memory machines in order = to >> avoid having to figure out how to not end up with OOM process kills >> while also not just wasting cores by using -j1 for buildworld? >>=20 >> In other words: enable swapping out active RAM when it eats nearly >> all the non-wired RAM. >>=20 >> But it might be discovered that the performance is not better than >> using fewer cores during buildworld. (Experiments needed and >> possibly environment specific for the tradeoffs.) Avoiding having >> to figure out the maximum -j? that avoids OOM process kills but >> avoids just sticking to -j1 seems and advantage for some rpi3 and >> rpi2 folks. >=20 > Interesting observation, maybe playing w/: > vm.swap_idle_threshold2: Time before a process will be swapped out > vm.swap_idle_threshold1: Guaranteed swapped in time for a process >=20 > will help thing... lowering 2 will likely make the processes = available > for swap sooner... Looking up related information: https://www.freebsd.org/doc/handbook/configtuning-disk.html says vm.swap_idle_enabled is also involved with those two. In fact it indicates the two are not even used until vm.swap_idle_enabled=3D1 . QUOTE 11.10.1.4. vm.swap_idle_enabled The vm.swap_idle_enabled sysctl(8) variable is useful in large = multi-user systems with many active login users and lots of idle = processes. Such systems tend to generate continuous pressure on free = memory reserves. Turning this feature on and tweaking the swapout = hysteresis (in idle seconds) via vm.swap_idle_threshold1 and = vm.swap_idle_threshold2 depresses the priority of memory pages = associated with idle processes more quickly then the normal pageout = algorithm. This gives a helping hand to the pageout daemon. Only turn = this option on if needed, because the tradeoff is essentially pre-page = memory sooner rather than later which eats more swap and disk bandwidth. = In a small system this option will have a determinable effect, but in a = large system that is already doing moderate paging, this option allows = the VM system to stage whole processes into and out of memory easily. END QUOTE The defaults seem to be: # sysctl vm.swap_idle_enabled vm.swap_idle_threshold1 = vm.swap_idle_threshold2 vm.swap_idle_enabled: 0 vm.swap_idle_threshold1: 2 vm.swap_idle_threshold2: 10 Quoting the book again: QUOTE If the swapping of idle processes is enabled and the pageout daemon can = find any processes that have been sleeping for more than 10 seconds = (swap_idle_threshold2, the cutoff for considering the time sleeping to be "a long time"), it = will swap them all out. [. . .] if none of these processes are available, the = pageout daemon will swap out all processes that has been sleeping for as briefly = as 2 seconds (swap_idle_threshold1). END QUOTE. I'd not normally expect a compile or link to sleep for such long periods (unless I/O has long delays). Having, say, 4 such processes active at = the same time may be unlikely to have any of them swap out on the default = scale. (Clang is less I/O bound and more memory bound than GCC as I remember = what I've observed. That statement ignores paging/swapping by the system.) Such would likely be true on the scale of any positive integer seconds figures? =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)