From owner-freebsd-fs@freebsd.org Wed Feb 27 19:24:57 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 124BA1502A70 for ; Wed, 27 Feb 2019 19:24:57 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-b.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C746B8BDF5 for ; Wed, 27 Feb 2019 19:24:55 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-b.userve.net (Postfix) with ESMTPS id 2608EE05 for ; Wed, 27 Feb 2019 19:24:25 +0000 (GMT) Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Wed, 27 Feb 2019 19:24:24 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Wed, 27 Feb 2019 19:24:24 +0000 From: Matt Churchyard To: "freebsd-fs@freebsd.org" Subject: ZFS memory use (arc limit) Thread-Topic: ZFS memory use (arc limit) Thread-Index: AdTOzy9S7FYs4h38TmiLAaAlftQV/A== Date: Wed, 27 Feb 2019 19:24:24 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] MIME-Version: 1.0 X-Rspamd-Queue-Id: C746B8BDF5 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.22 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.977,0]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:217.196.1.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-0.02)[country: GB(-0.09)]; NEURAL_SPAM_SHORT(0.58)[0.580,0]; MX_GOOD(-0.01)[av-smtp-in2.userve.net,av-smtp-in1.userve.net]; DMARC_POLICY_ALLOW(-0.50)[userve.net,reject]; RCVD_IN_DNSWL_NONE(0.00)[22.1.196.217.list.dnswl.org : 127.0.10.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:20652, ipnet:217.196.0.0/20, country:GB]; RCVD_TLS_LAST(0.00)[] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2019 19:24:57 -0000 Hello, What's the current best practices for handling memory use for ZFS ARC? I've been a ZFS user for over 10 years starting with v15 pretty much back w= hen it first appeared. Very quickly I adopted the attitude of immediately limiting ARC manually vi= a sysctl, as I seemed to get guaranteed swap issues otherwise. Over the yea= rs I've seen references to other people seeing the same issues, various wor= k on ZFS memory use, but I've never relied on the system to sort itself out= and always just hard limited it by default. Just this week I installed a new system (11.2-REL) with 12GB RAM (not a hug= e amount but more than most my FreeBSD systems) that does nothing else othe= r than NFS sharing. I never got round to limiting the ARC and thought it wo= uld probably be alright considering how mature the ZFS support is these day= s. I did check the sysctl and it seemed to be set around 11GB or so. Just o= ver 24 hours after booting the NFS shares went down and a check of dmesg sh= ows the familiar "x was killed; out of swap space" errors. Swap space is 4G= B, which could probably be higher, but I've never seen it above a few MB in= use. (Maybe I need to set it the same as RAM?) As such, it seems that it's still absolutely vital that the admin sets a ha= rd limit on ARC, leaving enough headroom for whatever applications are bein= g run on the system? Is this recommended, or is there something I'm not doing right...? Regards, Matt Churchyard