From owner-freebsd-stable@freebsd.org Sat Aug 10 00:34:33 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 667C2AE347 for ; Sat, 10 Aug 2019 00:34:33 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 46535h6jdCz4Djf for ; Sat, 10 Aug 2019 00:34:32 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id E1F0AAE346; Sat, 10 Aug 2019 00:34:32 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1674AE345 for ; Sat, 10 Aug 2019 00:34:32 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46535h5gVHz4Djd; Sat, 10 Aug 2019 00:34:32 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x32f.google.com with SMTP id q20so139224012otl.0; Fri, 09 Aug 2019 17:34:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zfXxn7xJscnw4FMfrXMpVfKmKGkodne3vF6p6tiTbng=; b=buQh0xxEQa3fe26jCuJ+PENpEJDPRposPsQ7B1u+45OIatOpq7fWPvQSNXWbqvZ8EO pOcX8a3olOPLrJq9Ez5ZiKl/xMMY5kohvNt+/c4dyxqwFYo+gZge4M6wOg+7PHy9R8Lq mlNvzplxBNARaTwrQTGfEruPAVCKPCP5hDtOc7NMRoD6jXkuKjCc59UWIoO7ZTpFKyx7 zTS264Qu16iBOEYmDmUGJ5vMwmfPX0pzXy+00LZ00Oq37E4dI2auHJg89d8hGAZoJisN NDzBanZMh+6rZ101NW2a8hvh/ogO+onADIVki846pGkPDYbrj3YYoIByyt/HPLd9/Wm6 iRLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zfXxn7xJscnw4FMfrXMpVfKmKGkodne3vF6p6tiTbng=; b=bbhfkwkaD+yrMSrjJqYhj8k1yicUaRgi0F62c+y3zOW0hpjtBdQ3IMwVGWePxlASW2 1kq1D4CfMNzIOfGOE1Pfry6y2Krehb32X4lPszXSAWXxpvRl+J8TjZAZfn3ml9HBwd03 0PBlB7rrYZv+vFpt/Zdck0irurXwWPP2vlaHgsDcWCyBo7+w3GyobLZoZe29yFYdLGnd ol2/RDBdKzoREYOfv7Z0Ex1r0o43b6uYarl7AX5DKwro6vNU03c+ca32NmfL/uy3dRiS 6VLCtoDALRnjTkMwxH9U2y+Fg9aQl2W7xLbkWc42YTfd/vht+Rv4W6tvYkVbG9F5iOWY j+TA== X-Gm-Message-State: APjAAAUPZz0N0M8VhqCnyRErLSkbcwR8575cdU59aXlt+Dz4i3WPr0Es S3nj8nCI1aiVsRCUa/iCY6oN2Lmw97Qx/b8RgniFws1K1+8= X-Google-Smtp-Source: APXvYqwchFcIS/ex3O/08aSoVNkCMB8mKyKpHLh0hvJX3vH4ssr8vT/p51i5LmYPxDyu+yohGggZwLhOBEkjWqcOgRg= X-Received: by 2002:a9d:7988:: with SMTP id h8mr20346737otm.301.1565397270924; Fri, 09 Aug 2019 17:34:30 -0700 (PDT) MIME-Version: 1.0 References: <20190809183547.GE86532@raichu> <20190809211612.GF86532@raichu> In-Reply-To: <20190809211612.GF86532@raichu> From: Kevin Oberman Date: Fri, 9 Aug 2019 17:34:14 -0700 Message-ID: Subject: Re: Memory management changes after kernel update on 6-Aug To: Mark Johnston Cc: FreeBSD Stable ML X-Rspamd-Queue-Id: 46535h5gVHz4Djd X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.993,0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Aug 2019 00:34:33 -0000 On Fri, Aug 9, 2019 at 2:16 PM Mark Johnston wrote: > On Fri, Aug 09, 2019 at 01:05:50PM -0700, Kevin Oberman wrote: > > On Fri, Aug 9, 2019 at 11:35 AM Mark Johnston wrote: > > > > > On Fri, Aug 09, 2019 at 11:09:24AM -0700, Kevin Oberman wrote: > > > > Since I updated my 12.0-STABLE system on 6-Aug I have been seeing > issues > > > > resuming my Win7 VM on VirtualBox. My prior kernel was built on > 24-Jul. > > > If > > > > there is not sufficient memory available to reload the system (4 > Meg.), > > > the > > > > > > Where does this number come from? What memory usage stats do you see > in > > > top(1) when the error occurs? > > > > > > > I am monitoring memory usage with gkrellm. It appears to define "Free" as > > the sum of "Inactive" and "Free". If you are referring to size of the VM, > > was supposed to be the memory specified when I created the VM, but my > > fingers got ahead of my brain and it should have been 4G, not 4M. Hey! > > What's a few orders of magnitude? > > > > Oddly, when I watch memory space closely I note that, as the VM loads, I > > started seeing swap utilization increase as free space was exhausted at > > about 80% loaded. Loading continued to 98%. at that point loading stopped > > and swap use continued to grow for a bit. Then free space started to > > increase from about 300M to about 700M before the error window popped up. > > > > > > > > resume fails with a message that memory was exhausted. Usually I can > try > > > > resuming again and it will work. Sometimes I get the error two or > three > > > > times before the system resumes. > > > > > > What exactly is the error message? > > > > > Failed to open a session for the virtual machine Win7. > > > > Failed to load unit 'pgm' (VERR_EM_NO_MEMORY). > > > > Result Code: NS_ERROR_FAILURE (0x80004005) > > Component: ConsoleWrap > > Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed} > > > > > > > > > > > Since I have not touched VirtualBox other than to rebuild the kmod > after > > > > the kernel build, it looks like something in the OS triggered this. > Since > > > > the system frees up some memory each time so that the VM eventually > > > > resumes, it looks like the memory request is made to the OS, but VB > is > > > not > > > > waiting or not enough memory is freed to allow the VB to complete the > > > > resume. > > > > > > > > Any clue what might have changed over those 13 days? I am running > GENERIC > > > > except that I run the 4BSD scheduler. > > > > > > Possible culprits are r350374 and r350375, but I can't really see how. > > > > > > > This started after the 6-Aug build (r350664). My prior build was r350292, > > so just before these two commits. > > > > Can I try just reverting these two? Once I do, it will need to run for a > > while or do something to tie up a lot of memory before the error will > > recur. In normal use it is a matter of firefox increasing resident memory > > until there is not enough free memory to load the VM without swapping. > > (These days I often see the sum of all firefox process resident memory > > exceeding 3G after it's been up for a day or two. Still, not worse than > > chromium.) > > Those commits can simply be reverted, but I am skeptical that they will > help. You should also verify that these same conditions don't lead to > errors on your prior build, if you haven't already. > OK. Running identical kernel except for 350374-5. Yes, I am sure that it was not happening with the r350292 kernel. I hit this quite consistently when firefox has been running for a while. Firefox has rss of just under 3G on startup and will slowly grow until I don't have the resources to run the Win7 VM without the error. Right now the VM completes loading with no swapping and about 800M of memory free after it is running. I'll let you know when it gets big enough to cause a problem and whether it fails. Probably won't happen until tomorrow. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683