From owner-freebsd-hackers@freebsd.org Sun Nov 18 01:51:47 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D271112135F for ; Sun, 18 Nov 2018 01:51:47 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-it1-x142.google.com (mail-it1-x142.google.com [IPv6:2607:f8b0:4864:20::142]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAA4F8C1D4 for ; Sun, 18 Nov 2018 01:51:46 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-it1-x142.google.com with SMTP id k141-v6so3003103itk.3 for ; Sat, 17 Nov 2018 17:51:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=SPe8QtCvYnSiXpqfQG/M8ZW5Qi4n9NsYbau32vRTNIM=; b=HMMnRhbpzkLspbzZo2uacJO5Uze5EcbRDON4PYrvTJCxq3LLOmSZhD7zJD9b0miN5o eUL4UVAJ/N63Xidxe2F/xNdB0Ep3p8uezoH6MJzj9EuD+VErUakvsiKqibAUYLK9A7jx YH3CewZu5+8kIqk7w4sBpbRHeJbSk8VbRc8PS0wqFwSIbZ2AECiWmBHYpzozGjyEti2B FcxjgNkjYd1i+cmoA2Ruijk4OC3qI1RFsWzaCswvenEkhj9oNsCzTb0Jrk+Mt/tlNajr 9WEwZjuZKho+6jvewNmOF73K0BwzikP9bdJgUApnCEakEuddmF2R3jUpA+On2ZG4y+z3 awDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=SPe8QtCvYnSiXpqfQG/M8ZW5Qi4n9NsYbau32vRTNIM=; b=mp8dAaylgxJuLLjMl5gE/i2u8ASl3EYc7yPGqIUyMEVomabf9YczB9pCFUJJY4ocF/ TBWghvfUL2sTsmK82OWy3cUOeR15ITw3TfHWdBlDi/OPhDYJ44UY1sEaZeGYFcioearf Aa9NelxOzfnt/EEX0D759nFg27PR7NeM5eYrqq3pM+wiZnBX1KHbtZzCx3SFi1rHoFka wfYTnLzvsz3GYvUK7NO8pLlP1xgUF8avVdkJpYxb2usnvUl4iIoYA/VjdY+komYu+7s4 EsQwhQBt+xGor98H7SDGsQjr3al73GVGPle1o8WjOnWHrdCnsho983R+VcWhdIkyBHbU Dexw== X-Gm-Message-State: AGRZ1gLtzUHt/oy46Qb4WCxkkeqORiNMQlNwCMnTkNgBk6wTLcAi6KnZ nwLaeJSYHIpNMJRQkAIaEEk= X-Google-Smtp-Source: AJdET5fuhZX8k9+UD0kEqQ4bfNZQNRS9JC5go39Mx3kZkzitGvmThgKfJfX1Vqm/ROet3gCPK8TWQw== X-Received: by 2002:a02:4007:: with SMTP id n7-v6mr14142120jaa.89.1542505905892; Sat, 17 Nov 2018 17:51:45 -0800 (PST) Received: from raichu (toroon0560w-lp130-07-64-229-95-98.dsl.bell.ca. [64.229.95.98]) by smtp.gmail.com with ESMTPSA id g186-v6sm13268413ita.22.2018.11.17.17.51.44 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 17 Nov 2018 17:51:45 -0800 (PST) Sender: Mark Johnston Date: Sat, 17 Nov 2018 20:51:43 -0500 From: Mark Johnston To: Mark Millard Cc: freebsd-hackers Hackers Subject: Re: 13-CURRENT: several GB swap being used despite plenty of free RAM Message-ID: <20181118015143.GD2799@raichu> References: <1748688.u6MfGjpqfb@photon.int.bluestop.org> <1542499188.56571.59.camel@freebsd.org> <20181118001318.GB2799@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: AAA4F8C1D4 X-Spamd-Result: default: False [-0.74 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.84)[-0.840,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; IP_SCORE(-0.19)[ip: (3.49), ipnet: 2607:f8b0::/32(-2.61), asn: 15169(-1.76), country: US(-0.10)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Nov 2018 01:51:47 -0000 On Sat, Nov 17, 2018 at 04:41:55PM -0800, Mark Millard wrote: > On 2018-Nov-17, at 16:13, Mark Johnston wrote: > > > > On Sat, Nov 17, 2018 at 04:59:48PM -0700, Ian Lepore wrote: > >> On Sat, 2018-11-17 at 22:52 +0100, Wojciech Puchar wrote: > >>> freebsd will not swap with that lots of free ram. > >>> but it's 90GB free NOW, how about before? > >>> > >> > >> Your information is outdated. For at least a couple years now (since > >> approximately the 10.1 - 10.2 timeframe is my vague estimate), freebsd > >> will page out application memory that hasn't been referenced for some > >> time, even when the system has no shortage of free memory at all. > > > > No, FreeBSD will only ever swap when there is a free page shortage. The > > difference is that we now slowly age unreferenced pages into the > > inactive queue, which makes them candidates for pageout and subsequent > > eviction. With pageout_update_period=0, anonymous memory won't get > > paged out unless there's a shortage of inactive pages, or an application > > calls madvise(MADV_DONTNEED) on a range of memory (which moves any > > backing pages to the inactive queue). > > Swapping is built on top of paging as I understand. The system > can page without swapping but can not swap without (effectively) > paging to implement the swapping, if I understand right. Right. > If I > understand right, swapped-out means that kernel stacks have > been written out and have to be loaded back in RAM before the > process/threads can even run. (I might not understand.) When free pages are scarce, one measure that the kernel may take to address the shortage is to swap out the kernel stacks of the threads in a process, thus allowing the pages backing the stacks to be reused for some other purpose, but preventing that process from running on a CPU until the stacks are swapped back in and "locked" (wired) into memory. Most of the pages consumed by an application like firefox are not used for kernel stacks. Most of them are used for the application's heap memory, and are thus private to that process. In general, pieces of such memory are subject to being paged out to the swap device, particularly when they are not frequently referenced (read or written to) by the application, in order to replenish the pool of free pages. Such memory is often said to be swapped out. As a side note, there are some system calls that modify this behaviour. mlock(2) effectively prevents the kernel from swapping out pages backing the specified virtual addresses; this guarantees that an access of the virtual memory range will never incur the cost of an expensive page-in. madvise(MADV_FREE) tells the kernel that the specified pages may be freed without first being written to the swap device. Thus, a subsequent read of an affected page may return the page's previous contents (if the page had not yet been reclaimed to make up for a shortage), or all zeroes (if the page had been freed without saving its contents to swap). > If I've got that right, are there distinctions here for > paging that is not part of swapping vs. actual swapping > (and its use of paging)? Saying that something does not > swap does not necessarily imply that it does not page: > it still could have paging activity that does not include > moving the kernel stacks for the process to backing media? Indeed, as I tried to describe above, kernel stack swapouts usually represent only a small portion of many applications' total swap usage, to the point where I at least usually don't think much about them. > At times I have trouble interpreting when wording goes back > and forth between swapping and paging, both for the intended > meaning and for the technical implications. When discussing paging activity related to the swap pager, I typically use those terms interchangeably. Sorry for the confusion.