From owner-freebsd-stable@freebsd.org Tue Jun 19 19:29:46 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB5301009BB7 for ; Tue, 19 Jun 2018 19:29:45 +0000 (UTC) (envelope-from paulz@vanderzwan.org) Received: from mail.paztec.nl (mail.paztec.nl [81.171.19.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.paztec.nl", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 27E6072609 for ; Tue, 19 Jun 2018 19:29:43 +0000 (UTC) (envelope-from paulz@vanderzwan.org) Received: from gaspode.vanderzwan.org (static-145.133.145.71.ip.telfort.nl [145.133.145.71]) (authenticated bits=0) by mail.paztec.nl (8.15.2/8.15.2) with ESMTPSA id w5JJGBc5088307 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 19 Jun 2018 21:16:12 +0200 (CEST) (envelope-from paulz@vanderzwan.org) X-Authentication-Warning: vps3.vanderzwan.org: Host static-145.133.145.71.ip.telfort.nl [145.133.145.71] claimed to be gaspode.vanderzwan.org From: Paul van der Zwan Message-Id: <822A9582-3C11-46AE-B492-F663E35CD10D@vanderzwan.org> Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) Subject: Re: lightly loaded system eats swap space Date: Tue, 19 Jun 2018 21:16:10 +0200 In-Reply-To: Cc: Jeremy Chadwick , freebsd-stable@freebsd.org To: Cassiano Peixoto References: <20180619172936.GA24967@icarus.home.lan> X-Mailer: Apple Mail (2.3445.8.2) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.paztec.nl [81.171.19.13]); Tue, 19 Jun 2018 21:16:12 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jun 2018 19:29:46 -0000 Hi I had something similar on FreeNAS 11.1 ( based on FreeBSD 11.1). Swap was allocated and never released until it ran out. Some time ago I set the following sysctl: vm.disable_swapspace_pageouts: 1 That completely stopped swap allocation and I have not rebooted since, = except for OS patching. =46rom what I found using google this sysctl may have some nasty side = effects when system runs out of memory, but that has not happened on my system. Paul > On 19 Jun 2018, at 19:57, Cassiano Peixoto = wrote: >=20 > Hi, >=20 > I have the very same issue on many servers. Mainly on mail servers = running > qmail+spamassassin+clamav. I've tuned some variables on loader.conf: >=20 > vfs.zfs.vdev.cache.size=3D"2G" > vfs.zfs.arc_min=3D"614400000" > vfs.zfs.arc_max=3D"4915200000" >=20 > But after some days, it begins eating swap and the system comes very = very > slow then I need to reboot it. >=20 > My system config is: >=20 > FreeBSD 11.1-STABLE #5 r321625M: Thu Sep 21 16:01:56 -03 2017 > root@mail.com:/usr/obj/usr/src/sys/MAIL amd64 > FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on = LLVM > 4.0.0) > VT(vga): resolution 640x480 > CPU: Intel(R) Atom(TM) CPU C2518 @ 1.74GHz (1750.04-MHz K8-class = CPU) > Origin=3D"GenuineIntel" Id=3D0x406d8 Family=3D0x6 Model=3D0x4d = Stepping=3D8 >=20 > = Features=3D0xbfebfbff >=20 > = Features2=3D0x43d8e3bf > AMD Features=3D0x28100800 > AMD Features2=3D0x101 > Structured Extended Features=3D0x2282 > VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > TSC: P-state invariant, performance statistics > real memory =3D 8589934592 (8192 MB) > avail memory =3D 8213245952 (7832 MB) >=20 > It's configured with 4GB of swap. Let me know if I can help with any = other > information. >=20 > Thanks. >=20 >=20 > On Tue, Jun 19, 2018 at 2:29 PM, Jeremy Chadwick = wrote: >=20 >> (I am not subscribed to -stable, so please CC me, though I doubt I = can >> help in any way/shape/form past this Email) >>=20 >> Not the first time this has come up -- and every time it has, all = that's >> heard is crickets in the threads. Recent proof: >>=20 >> = https://lists.freebsd.org/pipermail/freebsd-stable/2018-April/088727.html >> = https://lists.freebsd.org/pipermail/freebsd-stable/2018-April/088728.html >> = https://lists.freebsd.org/pipermail/freebsd-stable/2018-June/089094.html >>=20 >> I sent private mail to Peter Jeremy about his issue. I will not >> disclose that Email here. However, I will disclose the commits I >> included in said Email that have touched ZFS ARC-related code: >>=20 >> http://www.freshbsd.org/commit/freebsd/r332785 >> http://www.freshbsd.org/commit/freebsd/r332552 >> http://www.freshbsd.org/commit/freebsd/r332540 (may help give = insights) >> http://www.freshbsd.org/commit/freebsd/r330061 >> http://www.freshbsd.org/commit/freebsd/r328235 >> http://www.freshbsd.org/commit/freebsd/r327491 >> http://www.freshbsd.org/commit/freebsd/r326619 >> http://www.freshbsd.org/commit/freebsd/r326427 (quota-related, maybe >> irrelevant) >> http://www.freshbsd.org/commit/freebsd/r323667 >>=20 >> In short (and nebulous as hell; sorry, I cannot be more specific = given >> the nature of the problem): there have been changes about ZFS's = memory >> allocation/releasing decision-making scheme compared to ZFS on = "older" >> FreeBSD (i.e. earlier 11.x, and definitely 10.x and 9.x). >>=20 >> Recommendations like "limit your ARC" are nothing new in FreeBSD, but >> are still ridiculous kludges: tech-lists' system clearly has 105GB = MRU >> (MRU =3D most recently used) in ARC, meaning there is memory that can = be >> released back to the rest of the OS for general use (re: memory >> contention/pressure situation), but the OS is choosing to use swap >> instead, eventually exhausting it. That logic sounds broken, IMO. = (And >> yes I did notice the size of bhyve process) >>=20 >> ZFS-related kernel folks need to be involved in this conversation. = For >> whatever reason, in the past several years, related committers are no >> longer participating in these type of discussions. The opposite was >> true back in the 7.x to 9.x days. The answers have to come from = them. >> I don't know, today, a) how they prefer these problems get reported = to >> them, or b) what exact information they want that can help narrow it >> down (tech-lists' provided data is, IMO, good and par for the = course). >>=20 >> -- >> | Jeremy Chadwick jdc@koitsu.org | >> | UNIX Systems Administrator http://jdc.koitsu.org/ | >> | Making life hard for others since 1977. PGP 4BD6C0CB | >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >=20