From owner-freebsd-current@freebsd.org Wed Apr 7 19:54:42 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7D4A75C2CA9 for ; Wed, 7 Apr 2021 19:54:42 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FFw8f25D6z4h0h for ; Wed, 7 Apr 2021 19:54:42 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 479B35C2B37; Wed, 7 Apr 2021 19:54:42 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4752A5C2A2D; Wed, 7 Apr 2021 19:54:42 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FFw8f1P1hz4gZY; Wed, 7 Apr 2021 19:54:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x829.google.com with SMTP id y2so14684429qtw.13; Wed, 07 Apr 2021 12:54:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=DrjsRwsG4g3z0LP6lHpuF3SsvXCyAgTNf+PYp+22L5s=; b=hmhjuSKacdgH4AB6qvTfaGp6sZ+a+KWY4mVKcxu4vT7d8eQefXgBlUcKR3PjUQs3rd +ekgsIHcOU3caQjUbhI05G/dB05LSp9hCTLhCvlVMKhAOSNOXZtqn9eoyc6XSoPINx50 znAaT0KC076MaBmd2mHtphVOVKSSrZOs7RAMaD5jj84VnLkZ69QfgFlUzw+QIeK2RGaR NzkAQocw4HaoaUPmqQMkZiH5/QHNrwuPGmT7h6osJjZCS4zz3eAOJD6WDldj8sTCC5Rz +Xo5XMD+AxPJSe40bNE3KWOIJm7Jm0KwBTudcZrlvMW18EwFXMng5s9FKqkb5HggtIOs Odew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=DrjsRwsG4g3z0LP6lHpuF3SsvXCyAgTNf+PYp+22L5s=; b=ptBG9ygUgSzxEvzk94tiB9tFUVagtn5go5UqFpoEqN7+xv88ZHU3iqmGycqfFIK9H5 VRbKNVgKmM0NOPqc4Oze3AUPC+Q7aNkyCSIP85D3blC9A3LZ4e76XSbyVpEZjYCMPpuf EVFekRlbQGS1LdhanWEdbJ+pFYcZ0qrPB1m2Tr/35HkWgwsXz839zdrJOLjcXxyIIQDS 7AqSafntSqZocMo95FGcCAgG/qMSH3ei7jnaSX4I8XVAJPZgwc2HcfBzZXMQERw+RSUr hOzFyJkQVMXkWyZ+ew+Np+pIZY+UgNnUpCdj8vyTV1G89Os7qUbgiggCqMH6M/YYC0CW r/6A== X-Gm-Message-State: AOAM533sAovwp/vpUkEOXqKhla1dkNxNLZBDYHG64Tw2fN2/4ZH9U0TA KGRJDg9bTyfAeHiAavflDVo3NXYLHfO19Q== X-Google-Smtp-Source: ABdhPJwd5794Nb1wdNKKs7RJhZK4NrFWjJgHDZOJvFs8wxi1jryRIel04UCJ+5hForjCg3+gHZqZyQ== X-Received: by 2002:a05:622a:134f:: with SMTP id w15mr4186698qtk.162.1617825280846; Wed, 07 Apr 2021 12:54:40 -0700 (PDT) Received: from nuc ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id r133sm19337280qke.20.2021.04.07.12.54.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 12:54:40 -0700 (PDT) Sender: Mark Johnston Date: Wed, 7 Apr 2021 15:54:41 -0400 From: Mark Johnston To: Andriy Gapon Cc: freebsd-stable List , FreeBSD Current Subject: Re: stable/13, vm page counts do not add up Message-ID: References: <0606571f-fec0-c7ad-98e8-a0b8554918e2@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0606571f-fec0-c7ad-98e8-a0b8554918e2@FreeBSD.org> X-Rspamd-Queue-Id: 4FFw8f1P1hz4gZY X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2021 19:54:42 -0000 On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote: > > I regularly see that the top's memory line does not add up (and by a lot). > That can be seen with vm.stats as well. > > For example: > $ sysctl vm.stats | fgrep count > vm.stats.vm.v_cache_count: 0 > vm.stats.vm.v_user_wire_count: 3231 > vm.stats.vm.v_laundry_count: 262058 > vm.stats.vm.v_inactive_count: 3054178 > vm.stats.vm.v_active_count: 621131 > vm.stats.vm.v_wire_count: 1871176 > vm.stats.vm.v_free_count: 187777 > vm.stats.vm.v_page_count: 8134982 > > $ bc > >>> 187777 + 1871176 + 621131 + 3054178 + 262058 > 5996320 > >>> 8134982 - 5996320 > 2138662 > > As you can see, it's not a small number of pages either. > Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on this > system. > > This is 47c00a9835926e96, 13.0-STABLE amd64. > I do not think that I saw anything like that when I used (much) older FreeBSD. One relevant change is that vm_page_wire() no longer removes pages from LRU queues, so the count of pages in the queues can include wired pages. If the page daemon runs, it will dequeue any wired pages that are encountered. This was done to reduce queue lock contention, operations like sendfile() which transiently wire pages would otherwise trigger two queue operations per page. Now that queue operations are batched this might not be as important. We could perhaps add a new flavour of vm_page_wire() which is not lazy and would be suited for e.g., the buffer cache. What is the primary source of wired pages in this case?