From owner-freebsd-fs@freebsd.org Mon Aug 3 15:10:11 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15A0A9AE41C for ; Mon, 3 Aug 2015 15:10:11 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B609DFC0 for ; Mon, 3 Aug 2015 15:10:10 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.14.7/8.14.7) with ESMTP id t73F0854046209 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Mon, 3 Aug 2015 16:00:08 +0100 (BST) (envelope-from matt.churchyard@userve.net) Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 3 Aug 2015 16:00:07 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 3 Aug 2015 16:00:07 +0100 From: Matt Churchyard To: Alexander Leidinger , Quartz CC: FreeBSD FS Subject: RE: ZFS: Disabling ARC? Thread-Topic: ZFS: Disabling ARC? Thread-Index: AQHQy/JC06cuMEFgTU69kuki7wDyb5328/yAgANps/A= Date: Mon, 3 Aug 2015 15:00:07 +0000 Message-ID: <45e74b89dd754991a366a46aa5101822@SERVER.ad.usd-group.com> References: <55BC14B7.9010009@sneakertech.com> <20150801133635.00002ecc@Leidinger.net> In-Reply-To: <20150801133635.00002ecc@Leidinger.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2015 15:10:11 -0000 >On Fri, 31 Jul 2015 20:37:11 -0400 >Quartz wrote: > Can someone help clear up a few ZFS basics for me? >=20 > A few recent threads about ARC issues and memory-induced panics have=20 > made me realize I'm not 100% sure I understand ARC as well as I=20 > thought I did. >=20 > Say you have a ZFS file server that houses very large single files=20 > which are very infrequently accessed. For the sake of argument, let's=20 > say you're using ZFS on a home server for your family, and it holds=20 > exclusively a whole bunch of multi-gig bluray rips or whatever=20 > (nothing else). When someone wants to watch something, they copy the=20 > file to their desktop and watch it there. Although the family will=20 > watch several videos each day, any given file will only be accessed=20 > maybe once every couple months. (I know streaming would make more=20 > sense in real life, and that this example is kinda silly in general,=20 > but ignore that for now). >No matter if you stream or copy, it's the same operation, read once in a w= hile. > If I understand ARC correctly this would be a worst case scenario,=20 > right? Besides hogging ram, would ARC cause any problems here? Would=20 > disabling ARC and devoting the ram to other things be a wise idea? Is=20 > disabling ARC ever a wise idea? >You can tune how the ARC is used: ># zfs get all space/export/Movies | grep cache >space/export/Movies primarycache metadata local >space/export/Movies secondarycache none local >"primarycache" is the ARC in RAM, "secondarycache" is a cache device / L2A= RC (SSD). >"metadata" is directory listing, file sizes, access permissions and the li= ke. >So the above example means that metadata is allowed to go to the ARC in RA= M, and nothing of the real data in this dataset shall be cached >anywhere a= t all (neither in a cache device nor in RAM). >Bye, >Alexander. >-- >http://www.Leidinger.net Alexander@Leidinger.net: PGP 0xC773696B3BAC17DC >http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0xC773696B3BAC17DC I don't know if it's changed, but even changing ARC to metadata only used t= o cause massive performance issues when reading large files in small chunks= . Reading a 128k ZFS record in 4k chunks would cause ZFS to read the same 1= 28k record from disk 32 times. There's a forum thread about it here - https://forums.freebsd.org/threads/z= fs-primarycache-all-versus-metadata.45555/ Generally I've found ARC to be one of the most important parts of ZFS. Ther= e's no case I know of where it will actually adversely affect performance (= not that I've really looked). Its only downside is that it doesn't seem to = manage memory as well as it should, and so a lot of people (including me) h= ave resorted to limiting it. While it seems easy to "tune" using the primarycache option, it's rarely a = good idea except for very specific situations. If ARC isn't getting many hits (I use sysutils/zfs-stats to view stats), an= d you really don't want it using up all your memory, just reduce arc_max. Matt