From owner-freebsd-fs@freebsd.org  Thu Aug 18 20:27:00 2016
Return-Path: <owner-freebsd-fs@freebsd.org>
Delivered-To: freebsd-fs@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F5ECBBEE34
 for <freebsd-fs@mailman.ysv.freebsd.org>; Thu, 18 Aug 2016 20:27:00 +0000 (UTC)
 (envelope-from slw@zxy.spb.ru)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 214F21730;
 Thu, 18 Aug 2016 20:27:00 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>)
 id 1baTty-000JwN-0I; Thu, 18 Aug 2016 23:26:58 +0300
Date: Thu, 18 Aug 2016 23:26:57 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: Andriy Gapon <avg@FreeBSD.org>
Cc: freebsd-fs@FreeBSD.org, Alexander Motin <mav@FreeBSD.org>
Subject: Re: ZFS ARC under memory pressure
Message-ID: <20160818202657.GS8192@zxy.spb.ru>
References: <20160816193416.GM8192@zxy.spb.ru>
 <8dbf2a3a-da64-f7f8-5463-bfa23462446e@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8dbf2a3a-da64-f7f8-5463-bfa23462446e@FreeBSD.org>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Aug 2016 20:27:00 -0000

On Thu, Aug 18, 2016 at 11:00:28PM +0300, Andriy Gapon wrote:

> On 16/08/2016 22:34, Slawa Olhovchenkov wrote:
> > I see issuses with ZFS ARC inder memory pressure.
> > ZFS ARC size can be dramaticaly reduced, up to arc_min.
> > 
> > As I see memory pressure event cause call arc_lowmem and set needfree:
> > 
> > arc.c:arc_lowmem
> > 
> >         needfree = btoc(arc_c >> arc_shrink_shift);
> > 
> > After this, arc_available_memory return negative vaules (PAGESIZE *
> > (-needfree)) until needfree is zero. Independent how too much memory
> > freed. needfree set to 0 in arc_reclaim_thread(), when arc_size <=
> > arc_c. Until arc_size don't drop below arc_c (arc_c deceased at every
> > loop interation).
> > 
> > arc_c droped to minimum value if arc_size fast enough droped.
> > 
> > No control current to initial memory allocation.
> > 
> > As result, I can see needless arc reclaim, from 10x to 100x times.
> > 
> > Can some one check me and comment this?
> 
> You might have found a real problem here, but I am short of time right now to
> properly analyze the issue.  I think that on illumos 'needfree' is a variable
> that's managed by the virtual memory system and it is akin to our
> vm_pageout_deficit.  But during the porting it became an artificial value and
> its handling might be sub-optimal.

As I see, totaly not optimal.
I am create some patch for sub-optimal handling and now test it.