From owner-freebsd-stable@freebsd.org Mon Jun 29 13:17:35 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0AF9E34998B for ; Mon, 29 Jun 2020 13:17:35 +0000 (UTC) (envelope-from dwilde1@gmail.com) Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49wShZ2f24z3XmS; Mon, 29 Jun 2020 13:17:34 +0000 (UTC) (envelope-from dwilde1@gmail.com) Received: by mail-lj1-x244.google.com with SMTP id e4so17996532ljn.4; Mon, 29 Jun 2020 06:17:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/5EaNkCuTD4hc8Ikfcg2RSa7qcHEAvHH8KNsoui8iao=; b=KIWuuqIzH05Kimyp0HyEjm5f2EUUY7eQaNgF8cfOx3BkYFtt04xCo31sj9pLVU6fW6 nfvuB4nnv4LyWeJPl+unqlQZ9Xvaeg5F0/4wN7paIJ1JjLmvtoEjl78P804gK1zqIUwb oHWIyKjqgbjumFhTbgW64RzGYIOVAIWI7VWPmYw12c4+JCBpv+S1OQ43JtcUSI9XdGun WKeEhdsY5+dZyhbwvRx1Tl8xGg8HigwoT0xUmZObBq/fogs3AkNG+V0o5/aflwayNsl/ js/PMOmZZPQB/56W+wYkhBX/0UBDUCA9DKpphZVkFTuqazglPpw4Ie+aZ7tJQ2j4hqxS qcTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=/5EaNkCuTD4hc8Ikfcg2RSa7qcHEAvHH8KNsoui8iao=; b=Dg8FL7h/gvT417+wnsm+Bw1Ph7hYpLTQ6BkajroDbXQrRXZTfJmYjRZm20rDLEyAJB 2yRAQQ8qe6/XF9aSvkNT7IoEaIHxRkkt1Z2uwzasZK7s+b82yI/DZ5wHfrW/ysMQw/A3 SCgJNH9yPsiVb/0l/H35bBxpmgf7+mhwtkm+d/ilLbfu738bLbTjZ0Rv/eWLi/NTU1F2 vbMSHj0W7sisbz+OcLM1uuI0HWqDQbGBXqKZY7IryGehCSeIIybhtkL4cboqst8cYgCm 9a68mYoAtEe6WgMGsLQ7QoPgJCl+y0aHIQzQVDJM+mGSBcyACKcoRLMgUJsv2iwtGR0M cqYQ== X-Gm-Message-State: AOAM533KhGfr5IHFSVl0cVrsaLJ1RFnnqtVEj1Laqf9+Ph45kzEaSsO/ s11axgInp9h9aN57W6zErT9I6jJKkwo0M+gpetc= X-Google-Smtp-Source: ABdhPJyQq+DwEZsibmuzW4qOB6VXCA+OvOdtzmbwwPM6s0PWZStP3mQe2nI6Di5RfPD2or5qRVg9X2wM/VnAwXg75Rs= X-Received: by 2002:a2e:590:: with SMTP id 138mr7946783ljf.85.1593436652656; Mon, 29 Jun 2020 06:17:32 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ab3:4703:0:0:0:0:0 with HTTP; Mon, 29 Jun 2020 06:17:31 -0700 (PDT) Reply-To: dwilde1@gmail.com In-Reply-To: References: From: Donald Wilde Date: Mon, 29 Jun 2020 06:17:31 -0700 Message-ID: Subject: Re: swap space issues To: Mark Millard Cc: freebsd-stable@freebsd.org, ericbsd , bdrewery@freebsd.org Content-Type: multipart/mixed; boundary="000000000000a789cb05a938e1c6" X-Rspamd-Queue-Id: 49wShZ2f24z3XmS X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=KIWuuqIz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dwilde1@gmail.com designates 2a00:1450:4864:20::244 as permitted sender) smtp.mailfrom=dwilde1@gmail.com X-Spamd-Result: default: False [-3.43 / 15.00]; HAS_REPLYTO(0.00)[dwilde1@gmail.com]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.36)[-0.355]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.041]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.03)[-1.033]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::244:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2020 13:17:35 -0000 --000000000000a789cb05a938e1c6 Content-Type: text/plain; charset="UTF-8" [adding maintainers of synth and ccache] On 6/29/20, Mark Millard wrote: > Based on "small arm system" context experiments > mostly . . . > > If your console messasges do not include > messages about "swap_pager_getswapspace(...): failed", > then it is unlikely that being out of swap space > is the actual issue even when it reports: "was killed: > out of swap space" messages. For such contexts, making > the swap area bigger does not help. > It did not show those getswapspace messages. > In other words, "was killed: out of swap space" > is frequently a misnomer and not to be believed > for "why" the kill happened or what should be > done about it --without other evidence also being > present anyway. > > Other causes include: > > Sustained low free RAM (via stays-runnable processes). > A sufficiently delayed pageout. > The swap blk uma zone was exhausted. > The swap pctrie uma zone was exhausted. > > (stays-runnable processes are not swapped out > [kernel stacks are not swapped out] but do actively > compete for RAM via paging activity. In such a > context, free RAM can stay low.) > > The below material does not deal with the > the "exhausted" causes but does deal with > the other 2. > > Presuming that you are getting "was killed: out > of swap space" notices but are not getting > "swap_pager_getswapspace failed" notices and > that kern.maxswzone vs. system load has not > been adjusted in a way that leads to bad > memory tradeoffs . . . > > I recommend attempting use of, say, (from > my /etc/sysctl.conf ): > Attached is what I tried, but when I ran synth again, I got a corrupted HDD that fsck refuses to fix, whether in 1U mode or with fs mounted. It just will not SALVAGE even when I add the -y flag. What got corrupted was one of the /usr/.ccache directories, but 'ccache -C' doesn't clear it. I restored the original /etc/sysctl.conf, but I can't add packages or ports any more, so I'm afraid I'm going to have to dd if=/dev/zero the disk and reload from 12.1R and start over again. I can't even 'rm -Rf /usr/.ccache'. It says 'Directory not empty'. I don't need this system up and running, so I'm not going to make any more changes until I see if any of you have suggestions to try first. -- Don Wilde **************************************************** * What is the Internet of Things but a system * * of systems including humans? * **************************************************** --000000000000a789cb05a938e1c6 Content-Type: application/octet-stream; name="sysctl.conf.new" Content-Disposition: attachment; filename="sysctl.conf.new" Content-Transfer-Encoding: base64 X-Attachment-Id: file0 IyAkRnJlZUJTRDogc3RhYmxlLzEyL3NiaW4vc3lzY3RsL3N5c2N0bC5jb25mIDMzNzYyNCAyMDE4 LTA4LTExIDEzOjI4OjAzWiBicmQgJAojCiMgIFRoaXMgZmlsZSBpcyByZWFkIHdoZW4gZ29pbmcg dG8gbXVsdGktdXNlciBhbmQgaXRzIGNvbnRlbnRzIHBpcGVkIHRocnUKIyAgYGBzeXNjdGwnJyB0 byBhZGp1c3Qga2VybmVsIHZhbHVlcy4gIGBgbWFuIDUgc3lzY3RsLmNvbmYnJyBmb3IgZGV0YWls cy4KIwoKIyBVbmNvbW1lbnQgdGhpcyB0byBwcmV2ZW50IHVzZXJzIGZyb20gc2VlaW5nIGluZm9y bWF0aW9uIGFib3V0IHByb2Nlc3NlcyB0aGF0CiMgYXJlIGJlaW5nIHJ1biB1bmRlciBhbm90aGVy IFVJRC4KI3NlY3VyaXR5LmJzZC5zZWVfb3RoZXJfdWlkcz0wCnNlY3VyaXR5LmJzZC5zZWVfb3Ro ZXJfdWlkcz0wCnNlY3VyaXR5LmJzZC5zZWVfb3RoZXJfZ2lkcz0wCnNlY3VyaXR5LmJzZC5zZWVf amFpbF9wcm9jPTAKCiMKIyBEZWxheSB3aGVuIHBlcnNpc3RlbnQgbG93IGZyZWUgUkFNIGxlYWRz IHRvCiMgT3V0IE9mIE1lbW9yeSBraWxsaW5nIG9mIHByb2Nlc3Nlcy4gVGhlCiMgZGVsYXkgaXMg YSBjb3VudCBvZiBrZXJuZWwtYXR0ZW1wdHMgdG8gZ2FpbgojIGZyZWUgUkFNIChzbyBub3QgdGlt ZSB1bml0cykuCiN2bS5wYWdlb3V0X29vbV9zZXE9MTIgIyBkZWZhdWx0CiN2bS5wYWdlb3V0X29v bV9zZXE9LTEgIyBpbmZpbml0ZSAobm90IHN1cmUgZXhpc3RzID8/PykKdm0ucGFnZW91dF9vb21f c2VxPTEyMAojCiMgRm9yIHBsdW50eSBvZiBzd2FwL3BhZ2luZyBzcGFjZSAod2lsbCBub3QKIyBy dW4gb3V0KSwgYXZvaWQgcGFnZW91dCBkZWxheXMgbGVhZGluZyB0bwojIE91dCBPZiBNZW1vcnkg a2lsbGluZyBvZiBwcm9jZXNzZXM6CiN2bS5wZmF1bHRfb29tX2F0dGVtcHRzPS0xICMgaW5maW5p dGUKIwojIEZvciBwb3NzaWJseSBpbnN1ZmZpY2llbnQgc3dhcC9wYWdpbmcgc3BhY2UKIyAobWln aHQgcnVuIG91dCksIGluY3JlYXNlIHRoZSBwYWdlb3V0IGRlbGF5CiMgdGhhdCBsZWFkcyB0byBP dXQgT2YgTWVtb3J5IGtpbGxpbmcgb2YKIyBwcm9jZXNzZXM6CnZtLnBmYXVsdF9vb21fYXR0ZW1w dHM9IDEwCnZtLnBmYXVsdF9vb21fd2FpdD0gMQojIChUaGUgbXVsdGlwbGljYXRpb24gaXMgdGhl IHRvdGFsIGJ1dCB0aGVyZQojIGFyZSBvdGhlciBwb3RlbnRpYWwgdHJhZG9mZnMgaW4gdGhlIGZh Y3RvcnMKIyBtdWx0aXBsaWVkIGZvciB0aGUgc2FtZSB0b3RhbC4pCgojTm90ZTogQXMgb2Ygc3Rh YmxlLzEyIC1yMzUxNzc2ICwgc3RhYmxlIGdvdAojc3VwcG9ydCBmb3Igdm0ucGZhdWx0X29vbV9h dHRlbXB0cyBhbmQKI3ZtLnZtLnBmYXVsdF9vb21fd2FpdCB2aWEgYW4gTUZDLiBTdGFibGUgaGFz CiNoYWQgc3VwcG9ydCBmb3Igdm0ucGFnZW91dF9vb21fc2VxIGZvcgojbXVjaCBsb25nZXIgdGhh biB0aGF0LgojCiNOb3RlOiBMYXJnZXIgdmFsdWVzIG9mIHZtLnBhZ2VvdXRfb29tX3NlcQojd2ls bCB3YWl0IGV2ZW4gbG9uZ2VyLiBUaGUgZGVmYXVsdCB2YWx1ZSBpcwojMTIgKGxhc3QgSSBjaGVj a2VkKS4gTm8gZmlndXJlIHR1cm5zIG9mZgojdGhlIHNwZWNpZmljIG1lY2hhbmlzbSBhcyBmYXIg YXMgSSBrbm93LgojCg== --000000000000a789cb05a938e1c6--