From owner-freebsd-fs@freebsd.org Wed Nov 21 14:19:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89992113A4AE; Wed, 21 Nov 2018 14:19:47 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BBED7825D1; Wed, 21 Nov 2018 14:19:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id wALEJcBN093894 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 21 Nov 2018 15:19:39 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: lev@FreeBSD.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id wALEJcja052043 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 21 Nov 2018 21:19:38 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Where is my memory on 'fresh' 11-STABLE? It should be used by ARC, but it is not used for it anymore. To: Lev Serebryakov , Mark Johnston References: <20181120204537.GB2630@raichu> <1999292077.20181121135618@serebryakov.spb.ru> <123493949.20181121143445@serebryakov.spb.ru> <228548325.20181121170325@serebryakov.spb.ru> Cc: FreeBSD FS , FreeBSD stable From: Eugene Grosbein Message-ID: <92e3ea2e-b7ed-ec26-c56b-2d2acbf41772@grosbein.net> Date: Wed, 21 Nov 2018 21:19:32 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <228548325.20181121170325@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: BBED7825D1 X-Spamd-Result: default: False [-5.14 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; MX_INVALID(0.50)[cached]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-2.55)[ip: (-6.51), ipnet: 2a01:4f8::/29(-3.35), asn: 24940(-2.88), country: DE(-0.01)]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Nov 2018 14:19:47 -0000 21.11.2018 21:03, Lev Serebryakov wrote: >>> And here top consumers. I don't like that most of te memory is in FREE >>> abd_chunks - 8GB! >>> >>> USED (b) FREE (b) NAME >>> 531902464 8392536064 abd_chunk >> This may be a leak. Can you monitor this value for longer run? > No problem. > > But I don't think it is "true" leak, when memory is NEVER > freed, as uptime of this box is 11 days already, and sometimes ARC is much > larger than 2GiB. For example, now it is inflated to 8GiB. > > Ok, and now it is: > > USED (b) FREE (b) NAME > 7307493376 2383060992 abd_chunk > > It is more likely that there is a bug in "priority" for memory > distribution: abd FREE memory has higher priority than ARC somehow. OTOH, if "abd_chunks" part of ZFS ARC is not released to the UMA from which it was allocated (to be reused by other parts of the kernel) but is reused by ZFS itself, then top(1) should report it as part of ZFS ARC and not as other "Wired" memory.