From owner-freebsd-fs@freebsd.org Wed Jan 17 10:35:09 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9988EBBE6F for ; Wed, 17 Jan 2018 10:35:09 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv190.fwdcdn.com (frv190.fwdcdn.com [212.42.77.190]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AD99963A61 for ; Wed, 17 Jan 2018 10:35:08 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv198.fwdcdn.com ([212.42.77.198]) by frv190.fwdcdn.com with esmtp ID 1ebkoA-0009eL-Jx for freebsd-fs@freebsd.org; Wed, 17 Jan 2018 12:19:02 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:To: Subject:From:Date:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=AiJ0OmtPqgfj2f5rBFIqhfgQsiwtFY+YGCUB97oaE/k=; b=ek7jV3EJwqB1jcMRxxiBEpY3Kh K8A/WK20GdZMiukv6qgcYBnGNScG76KQSSvl/R5O+56POcY4tfnv7i+mYh5AosU3O92kjmwe5zZY+ M7uhXC/WCp10AtCtr30l7BK0tazQwnnPQRc6UYr0ga7GcptkSxV4HpNmRseWSgnF8Zi8=; Received: from [10.10.10.33] (helo=frv33.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1ebko2-0009Lj-6A for freebsd-fs@freebsd.org; Wed, 17 Jan 2018 12:18:54 +0200 Date: Wed, 17 Jan 2018 12:18:54 +0200 From: Paul Subject: Is ARC too conservative? To: freebsd-fs@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1516183581.432902299.yk20k8gl@frv33.fwdcdn.com> Received: from devgs@ukr.net by frv33.fwdcdn.com; Wed, 17 Jan 2018 12:18:54 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 10:35:10 -0000 Hello team, On numerous occasions we have observed a strange thing. Basically, we observer it after every restart of long running server. The thing is that disk busy % (both SSD and HDD) drops dramatically (given the identical overall server load): by 40% - 50%.  Then slowly, over period of few weeks, busy percentage grows to the previous value where it stabilises. You don't even have to restart server to reproduce this thing. It is enough to drop ARC size to it's minimum value by applying synthetic memory pressure. Could it be that ARC is too conservative regarding the data that is already in cache so that it gives less opportunity for new data to be added? I strongly suspect it's the MFU that is too conservative. How long may the blocks that have a lot of hits stay there? How the score is calculated and is there a way to tweak the formula via sysctl? Thanks. Best regards, Paul