From owner-freebsd-fs@freebsd.org Thu Jun 14 05:43:19 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4DFC10017E1 for ; Thu, 14 Jun 2018 05:43:19 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from mx-p1.obspm.fr (mx-p1.obspm.fr [145.238.193.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "*.obspm.fr", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 589FE7446B for ; Thu, 14 Jun 2018 05:43:19 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from io.chezmoi.fr (vpn.obspm.fr [145.238.186.39]) (authenticated bits=0) by mx-p1.obspm.fr (8.14.4/8.14.4/DIO Observatoire de Paris - 15/04/10) with ESMTP id w5E5hFHY179344 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jun 2018 07:43:17 +0200 Date: Thu, 14 Jun 2018 07:43:33 +0200 From: Albert Shih To: Shane Ambler Cc: freebsd-fs@freebsd.org Subject: Re: What append when arc full Message-ID: <20180614054333.GA7053@io.chezmoi.fr> References: <20180613200700.GA3156@io.chezmoi.fr> <248ad0be-2002-89b5-0c1a-3ac73d493149@ShaneWare.Biz> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <248ad0be-2002-89b5-0c1a-3ac73d493149@ShaneWare.Biz> User-Agent: Mutt/1.10.0 (2018-05-17) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.5.11 (mx-p1.obspm.fr [145.238.193.20]); Thu, 14 Jun 2018 07:43:17 +0200 (CEST) X-Virus-Scanned: clamav-milter 0.99.3 at mx-p1.obspm.fr X-Virus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jun 2018 05:43:19 -0000 Le 14/06/2018 à 13:06:07+0930, Shane Ambler a écrit > On 14/06/2018 05:37, Albert Shih wrote: > > > > > > On one of my server I need to reserve some ram to the application. > > > > So I would like to know what's happend when the ARC are full. > > ARC is a cache and it will use any unused ram that it can. When a > process makes a memory request some of the ARC can be released to > satisfy the request. This can add a delay to a process starting, which > is not always desirable. Ok. Because in my case it's not just a delay. Here my problem (I know I should start here), I'm planing to change my current imap server (for 2000 users) to a new one running FreeBSD 11 + Cyrus Imap. So I got a server with 192Go of Ram, 2 SSD, 28 mechanicals disks for email, and 2 mechanicals disks for the host system boot. The imap server run inside a jail on the host, the imap server root FS is a zpool with two ssd, and the data are on a zpool with 4 vdev (7 disks each). The perf are good, and everything seem working fine, but I just find out I can't use zfs send/recv. When I try to send (recv are OK) a snapshot (especially with a large one) after 1-5 minutes from the moment I start the sending the host become extremely slow, the "zfs list" command hang, and my jail cannot be join. To be honest the server (both) don't crash, but I can't let the imap server hang during long time (it are in pre-production) event I think after the sending everything will be fine. I try last night, if I put vfs.zfs.arc_max=96G everything work fine (the sending). So I think that's the point. But my concern are what append if I shrink a little more the arc_max, let's say 64G knowing I got < 30To of data. Does I'm going to loose lot of performance ? > > If you want to keep some ram free for your processes, then you can set > vfs.zfs.arc_max to limit the ram used by ARC. On recent system versions > you can change it dynamically using sysctl or you can set it in "dynamically" .... I though it's only can set at boot time, at least it's what https://www.freebsd.org/doc/handbook/zfs-advanced.html say... > /boot/loader.conf to set it at startup, it accepts human units so you > can use vfs.zfs.arc_max=25G Lots of thanks for your help. Regards -- Albert SHIH Observatoire de Paris France xmpp: jas@obspm.fr Heure local/Local time: Thu Jun 14 07:29:32 CEST 2018