From owner-freebsd-hackers@freebsd.org Sun Nov 18 22:05:20 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB99D1125720 for ; Sun, 18 Nov 2018 22:05:20 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 92CE3731E3; Sun, 18 Nov 2018 22:05:19 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id OVBkg5bbMP7x2OVBlgu86r; Sun, 18 Nov 2018 15:05:11 -0700 X-Authority-Analysis: v=2.3 cv=X9GD11be c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=JHtHm7312UAA:10 a=xfDLHkLGAAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=J_VOeTAUeCDFlHUBMgUA:9 a=CjuIK1q_8ugA:10 a=IfaqVvZgccqrtc8gcwf2:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 7D8FF35A9; Sun, 18 Nov 2018 14:05:07 -0800 (PST) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id wAIM56q7036244; Sun, 18 Nov 2018 14:05:06 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id wAIM543W036241; Sun, 18 Nov 2018 14:05:04 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201811182205.wAIM543W036241@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Stefan Blachmann cc: Cy Schubert , Mark Millard , Rebecca Cran , freebsd-hackers Hackers , Mark Johnston , Ian Lepore , Wojciech Puchar Subject: Re: 13-CURRENT: several GB swap being used despite plenty of free RAM In-Reply-To: Message from Stefan Blachmann of "Sun, 18 Nov 2018 13:11:49 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 18 Nov 2018 14:05:04 -0800 X-CMAE-Envelope: MS4wfESIzqdUbe6vDrEggYLVoj/H9qpzCTuG+2TzbC09RZG9OBTt+m8KwwWc1Ku8JdhJmVq3tGB67zvOChDKqV1LyB+/99oo7nSvqUbAJJurvQ9GypKhh+dJ OeW4zVIWCnZaN3OCR2a8t/EjLkqF9+GCswA72L3ou1+3e2Ub0myrYM1oWNLEDhMDIekbnetinGP+cN90OknsvCXuxTIYShf5v8cxA9fTan90a9WodasDYzG6 41feLkKeVjwgkkKOKZaIyk3U6RTfp6IsRpsQ6KpaztfpzNwZBF4myNDbnip/0+ej69PbjzBG5PiC5RnTYik4zfEfqYsNlzRYs+FZmar8R5QyRYSY9Q7M0F4j XvIdfX0x X-Rspamd-Queue-Id: 92CE3731E3 X-Spamd-Result: default: False [-2.54 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.zen.spamhaus.org : 127.0.0.11]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: spqr.komquats.com]; NEURAL_HAM_SHORT(-0.94)[-0.945,0]; RCPT_COUNT_SEVEN(0.00)[8]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[13.134.59.64.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; IP_SCORE(-0.89)[ipnet: 64.59.128.0/20(-2.41), asn: 6327(-1.93), country: CA(-0.10)]; FROM_EQ_ENVFROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Nov 2018 22:05:20 -0000 In message , Stefan Blachmann writes: > The inconveniences that the new swapping strategy causes are a regular > topic in the FreeBSD forums. > > Desktop users complain about lagginess, server users complain of long > delays because server processes intended to be kept in memory for > quick response times got swapped out and need to be swapped in again, > resulting in outrageously poor server performance in spite of plenty > of unused memory. There are many factors for this. One of the biggest mistakes people make is use UFS and ZFS on the same system. Limiting ZFS ARC max would be a good place to start. Unfortunately like many operating systems these days, FreeBSD's tuning parameters are generally baked in. And, fencing (an IBM term to limit address space memory use or to establish a minimum) isn't available. So, tuning for a site specific workload is more difficult. Making sure there is plenty of RAM installed and tuning down ZFS ARC can go a long way. Having said that, it's been a while since I've had to do this. The updates made to ZFS ARC and NUMA have allowed me to rely on the algorithms baked into FreeBSD. My 8 GB systems have performed rather well. > > Turning off swap completely, as Cy Schubert suggests, is strongly > discouraged in the forums, as it can lead to kernel panicking because > of being unable to swap out in critical kernel memory shortage > situations, leading to the risk of very serious filesystem > corruption. Actually, I didn't suggest it. I simply pointed out the option. Furthermore, it was a mistake for Linux to remove swapping entirely from their kernel. For those who want to try it, however, the option is there. > > However, Cy Schubert is probably right when stating that the new > swapping strategy resembles the 1960s-1980s industry's main swapping > strategy. > The bad thing is now, that nowadays memory is no longer scarce and > people can dimension their memory such that under normal circumstances > there will never be any need to swap. Adding more RAM has been the general thinking over the last 15-20 years. > > So I guess the unwillingness of the developer team to add an option > like "NoPreemptiveSwapping", which disables swapping out as long as > there is free physical memory available, is of psychological nature. How do you know processes are indeed swapped out rather than (many) pages paged out. You can't tell and using top's w mode, sorted by swap doesn't tell us anything because IMO it's bogus, incorrect numbers. > > Lacking such an option, there is still the possibility to use rctl to > disable swapping for particular users, processes, jails etc to > mitigate the problems caused by the new swapping strategy to some > degree. Our options are limited. I'd prefer something similar to what we had on MVS, when I worked on it, however even IBM has dumbed down SRM (System Resource Manager) such that knobs that were once there no longer are. Tuning based upon analysis of site specific stats are a thing of the past. It's 2018 and sadly one size fits all. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.