Date: Sat, 24 Nov 2018 09:47:04 -0800 From: Cy Schubert <Cy.Schubert@cschubert.com> To: Ian Lepore <ian@freebsd.org> Cc: Stefan Blachmann <sblachmann@gmail.com>, Cy Schubert <Cy.Schubert@cschubert.com>, Wojciech Puchar <wojtek@puchar.net>, freebsd-hackers Hackers <freebsd-hackers@freebsd.org>, Rebecca Cran <rebecca@bluestop.org>, Mark Johnston <markj@freebsd.org> Subject: Re: 13-CURRENT: several GB swap being used despite plenty of free RAM Message-ID: <201811241747.wAOHl4Kx079905@slippy.cwsent.com> In-Reply-To: Message from Ian Lepore <ian@freebsd.org> of "Sat, 24 Nov 2018 10:15:47 -0700." <1543079747.36737.6.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <1543079747.36737.6.camel@freebsd.org>, Ian Lepore writes: > On Sun, 2018-11-18 at 13:11 +0100, Stefan Blachmann wrote: > > The inconveniences that the new swapping strategy causes are a regular > > topic in the FreeBSD forums. > > > > Desktop users complain about lagginess, server users complain of long > > delays because server processes intended to be kept in memory for > > quick response times got swapped out and need to be swapped in again, > > resulting in outrageously poor server performance in spite of plenty > > of unused memory. > > > > Turning off swap completely, as Cy Schubert suggests, is strongly > > discouraged in the forums, as it can lead to kernel panicking because > > of being unable to swap out in critical kernel memory shortage > > situations, leading to the risk of very serious filesystem > > corruption. > > > > However, Cy Schubert is probably right when stating that the new > > swapping strategy resembles the 1960s-1980s industry's main swapping > > strategy. > > The bad thing is now, that nowadays memory is no longer scarce and > > people can dimension their memory such that under normal circumstances > > there will never be any need to swap. > > > > So I guess the unwillingness of the developer team to add an option > > like "NoPreemptiveSwapping", which disables swapping out as long as > > there is free physical memory available, is of psychological nature. > > > > Lacking such an option, there is still the possibility to use rctl to > > disable swapping for particular users, processes, jails etc to > > mitigate the problems caused by the new swapping strategy to some > > degree. > > > > Well that was a nice little rant, I hope you feel better. Perhaps it > has cleared your mind enough to eventually realize that I gave you the > command to do exactly what you sarcastically suggest "the developer > team" is unwilling to implement. > > It's still there, in the quoted text below, not quite completely > obscured by the typical word soup of irrelevancies from the usual > source which sidetracked this thread into a discussion about swapping, > which isn't involved in any way (and served with a side of mixed top > and bottom posting to make it almost impossible to follow the > conversation). > > -- Ian There was some question about swapping, which I had answered. My point was: this technology has been around for decades and that almost all operating systems manage memory in a similar fashion -- I believe my answer was a little more subtle and wordy, put bluntly: this is CS 101 and everyone here should understand it. It doesn't pay to be subtle. Bluntly: it's virtually done the same everywhere. So what is the problem? -- Cheers, Cy Schubert <Cy.Schubert@cschubert.com> FreeBSD UNIX: <cy@FreeBSD.org> Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. > > > On 11/18/18, Cy Schubert <Cy.Schubert@cschubert.com> wrote: > > > > > > In message <F5ACF6D0-DBD7-416F-9AAC-7709771FE545@yahoo.com>, Mark > > > Millard via f > > > reebsd-hackers writes: > > > > > > > > On 2018-Nov-17, at 16:13, Mark Johnston <markj at freebsd.org> > > > > wrote: > > > > > > > > > > > > > > > On Sat, Nov 17, 2018 at 04:59:48PM -0700, Ian Lepore wrote: > > > > > > > > > > > > On Sat, 2018-11-17 at 22:52 +0100, Wojciech Puchar wrote: > > > > > > > > > > > > > > freebsd will not swap with that lots of free ram. > > > > > > > but it's 90GB free NOW, how about before? > > > > > > > > > > > > > Your information is outdated. For at least a couple years now > > > > > > (since > > > > > > approximately the 10.1 - 10.2 timeframe is my vague > > > > > > estimate), freebsd > > > > > > will page out application memory that hasn't been referenced > > > > > > for some > > > > > > time, even when the system has no shortage of free memory at > > > > > > all. > > > > > No, FreeBSD will only ever swap when there is a free page > > > > > shortage. > > > > > The > > > > > difference is that we now slowly age unreferenced pages into > > > > > the > > > > > inactive queue, which makes them candidates for pageout and > > > > > subsequent > > > > > eviction. With pageout_update_period=0, anonymous memory won't > > > > > get > > > > > paged out unless there's a shortage of inactive pages, or an > > > > > application > > > > > calls madvise(MADV_DONTNEED) on a range of memory (which moves > > > > > any > > > > > backing pages to the inactive queue). > > > > Swapping is built on top of paging as I understand. The system > > > > can page without swapping but can not swap without (effectively) > > > > paging to implement the swapping, if I understand right. If I > > > > understand right, swapped-out means that kernel stacks have > > > > been written out and have to be loaded back in RAM before the > > > > process/threads can even run. (I might not understand.) > > > > > > > > If I've got that right, are there distinctions here for > > > > paging that is not part of swapping vs. actual swapping > > > > (and its use of paging)? Saying that something does not > > > > swap does not necessarily imply that it does not page: > > > > it still could have paging activity that does not include > > > > moving the kernel stacks for the process to backing media? > > > This is generally the old-school definition, IIRC third year comp > > > sci, > > > original BSD definition, which was also the way IBM defined it for > > > MVS. > > > (IBM also virtually swapped out address spaces, not written to > > > DASD, as > > > a means to deny CPU cycles through the scheduler not given the > > > chance > > > to consider the tasks (threads) in the address space). > > > > > > You can disable swapping by setting vm.swap_enabled=0. > > > > > > > > > > > > > > > At times I have trouble interpreting when wording goes back > > > > and forth between swapping and paging, both for the intended > > > > meaning and for the technical implications. > > > > > > > > > > > > > > > > > > > > > The advice I was recently given to revert to the old behavior > > > > > > is: > > > > > > > > > > > > sysctl vm.pageout_update_period=0 > > > > > > > > > > > > I've been using it on a couple systems here for a few days > > > > > > now, and so > > > > > > far results are promising, I am no longer seeing gratuitous > > > > > > swapfile > > > > > > usage on systems that have so much free physical ram that > > > > > > they should > > > > > > never need to page anything out. I haven't yet pushed one of > > > > > > those > > > > > > systems hard enough to check what happens when they do need > > > > > > to start > > > > > > proactively paging out inactive memory due to shortages -- it > > > > > > could be > > > > > > that turning off the new behavior has downsides for some > > > > > > workloads. > > > > > > > > > > > > > > > > === > > > > Mark Millard > > > > marklmi at yahoo.com > > > > ( dsl-only.net went > > > > away in early 2018-Mar) > > > > > > > > _______________________________________________ > > > > freebsd-hackers@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > > To unsubscribe, send any mail to > > > > "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > -- > > > Cheers, > > > Cy Schubert <Cy.Schubert@cschubert.com> > > > FreeBSD UNIX: <cy@FreeBSD.org> Web: http://www.FreeBSD.org > > > > > > The need of the many outweighs the greed of the few. > > > > > > > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freeb > > > sd.org" > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd > > .org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201811241747.wAOHl4Kx079905>