From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 23 07:28:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ACE51065694; Mon, 23 Aug 2010 07:28:09 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 83C798FC18; Mon, 23 Aug 2010 07:28:08 +0000 (UTC) Received: by vws7 with SMTP id 7so5779518vws.13 for ; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Jgtv7TWfJwnpmhOm6b4jsBomOPI9yv3rowE3OKMkCZo=; b=pzN3epxiogxs+7ZvR6Wxm8GLOPWhd/jJ9CqqhTEL2UGvNErmfTms30kAKRJXCuX4y/ 1THI3jhqLTA1kB1Y9wD3/iBSFXPF2JmvVP3Ypz1tkQOWszqL1rEJl/P9Gc4oNkowneDn eIWevlPBz9syLluiCV8LL85jGVQs0O+eVQOjk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=II9i4RHi031WiwmDntjWMTNywCUCIWVJ+XW2EdC+kYlk/H6eEKQOW6e338ONYbEGWl ukwRvxBAhcQuajAXAVgqrmMoKmW4kfiNqjdpjGg2TEmb45PrVa4EkZ+q9h0ti9RBLPpQ GpKzR4n8lJiyaV/huaLyg127B/GgnOKDzTvwE= MIME-Version: 1.0 Received: by 10.220.124.74 with SMTP id t10mr2850507vcr.179.1282548487609; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) In-Reply-To: <4C721161.40403@freebsd.org> References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> Date: Mon, 23 Aug 2010 00:28:07 -0700 X-Google-Sender-Auth: 8X7QKvbEAmQydQw8TDx4sxgnh9c Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 07:28:09 -0000 Ah! After re-reading your first email and I think I've finally got what you're saying -- with your change ARC would only start giving up memory when pagedaemon is awake. Presumably once it's awake it will also run through inactive list pushing some of it to cache. On the other hand existing code voluntarily frees up memory even before pagedaemon wakes up and does such a good job that pagedaemon practically never wakes up and thus does not get to clean inactive list. Can anyone test the patch on a system with mix of UFS/ZFS filesystems and see if the change mitigates or solves the issue with inactive memory excessively backpressuring ARC. Overall I think that your proposed change seems to make sense to me. If we could also deal with zone fragmentation issue you've written in another thread, that should bring ZFS even closer to being usable without shaman-style (the one with lots of muttering, swearing and dancing around) tuning. Actually, it may be worth trying your test with re-enabled UMA allocator for ARC. Now that pagedaemon will be running, it would also invoke UMA's low memory handlers and those should be able to give some memory back to the system. --Artem On Sun, Aug 22, 2010 at 11:12 PM, Andriy Gapon wrote: > on 23/08/2010 02:52 Artem Belevich said the following: >> Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size >> behavior in addition to the stuff included on your graphs now? > > Yes, I do and not by a chance :-) > >> All I >> can tell from your graphs is that v_free_count+v_cache_count shifted a >> bit lower relative to v_free_target+v_cache_min. > > Don't belittle those graphs :-) > Remember that the "fuchsia" line is when pagedaemon is woken up. > >> It would be >> interesting to see what effect your patch has on ARC itself, >> especially when ARC will start giving up memory and when does it stop >> shrinking. > > In an extreme case it stops at arc_c_min as expected. =A0An extreme case = is when > userland application(s) demand a lot of memory fast. > > Now the graphs: > http://people.freebsd.org/~avg/arc1.png > http://people.freebsd.org/~avg/arc2.png > http://people.freebsd.org/~avg/pages.png > http://people.freebsd.org/~avg/arc3.png > > What do you see? =A0What do you think? > > -- > Andriy Gapon >