From owner-freebsd-arch@freebsd.org Thu Jan 23 03:11:42 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E32E22A01F for ; Thu, 23 Jan 2020 03:11:42 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4836kP07mrz3xW6; Thu, 23 Jan 2020 03:11:40 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 00N3BWhn027379; Wed, 22 Jan 2020 19:11:33 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 00N3BVju027378; Wed, 22 Jan 2020 19:11:31 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202001230311.00N3BVju027378@gndrsh.dnsmgr.net> Subject: Re: Minimum memory for ZFS (was Re: svn commit: r356758 - in head/usr.sbin/bsdinstall: . scripts) In-Reply-To: <202001230207.00N274xO042659@mail.karels.net> To: mike@karels.net Date: Wed, 22 Jan 2020 19:11:31 -0800 (PST) CC: Ben Woods , Philip Paeps , freebsd-arch@freebsd.org, Ed Maste , Conrad Meyer X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4836kP07mrz3xW6 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [2.63 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.03)[ip: (0.13), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.02), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_SPAM_MEDIUM(0.75)[0.746,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.95)[0.947,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 03:11:42 -0000 > I took the liberty of changing the subject line to make it stand out a > bit more. Thanks, good idea, I seem to have missed some of this thread. > Ben wrote: > > > On Sat, 18 Jan 2020 at 09:16, Mike Karels wrote: > > > > > On Fri, 17 Jan 2020 at 08:21, Ben Woods wrote: > > > > > > > > Perhaps we could simply include a message on that bsdinstall > > > partitioning > > > > > mode selection screen that UFS is recommended on systems with < 4 Gb > > > RAM? > > > > > > > > > > > > I have uploaded a diff for this here: https://reviews.freebsd.org/D23224 > > > > > > > Please let me know your thoughts (comments in the phabricator review > > > would > > > > be best). > > > > > > I think this needs more discussion, preferably on this list. I am not > > > convinced that systems with as little as 4 GB should use ZFS. Conventional > > > wisdom on the FreeNAS mailing list says that 8 GB is required for ZFS, > > > and FreeNAS no longer includes UFS as an option. Conrad suggested a > > > cutoff of 16 GB; I am happier with 16 GB than 4 GB as a cutoff. Also, > > > there was mention of auto-tuning for smaller systems; I don't think that > > > has materialized yet. I'm not sure how plausible that is without knowing > > > the workload. I use ZFS on a workstation/server with 64 GB that runs 4 > > > bhyve guests that do things like buildworld. ZFS wants 63 GB for arc_max; > > > needless to say, I have a tunable set to a much lower value. If tuning > > > is required, it is unclear that ZFS is a good default. > > > > > > Mike > > > > > > > Hi everyone, > > > Before I commit phabricator review D23224, is there any final comments? > > > Particularly on these 2 lines of help-text: > > msg_partitioning_zfs_help="ZFS is recommended if you have at least 4GB RAM" > > msg_partitioning_ufs_help="UFS is recommended if you have less than 4GB of > > RAM" > > > There is some disagree about what these 2 recommendations should be. ZFS is recommended if you have read the ZFS memory tuning guide :-) I have run it in some pretty tiny memory configurations, with proper tunning including 32bit nodes with 512MB of memory and no other memory intense processes. > > 4GB was recommended by: imp, emaste, philip, eugen, dteske > > 8GB was recommended by: mike > > 16GB was recommended by: cem > > > The 4GB limit seems to have the best consensus, however there was some > > debate about whether ZFS is recommended on a system with 4GB, or only > > systems with MORE THAN 4GB. > > I don't remember what everyone else wrote, but IIRC, Devin said that if > you use ZFS with 4 GB, you will soon end up with a dozen tunables set. > That doesn't sound like a recommendation for 4 GB. >From my perspective ZFS is a great thing, BUTT in no case where I have it installed is it running with out of the box settings. In all cases I minimally tweak vfs.zfs.arc_max as the default for this leaves systems with a configuratoin that invariable runs into OOM issues. Until that issue is fixed for me it is a mistake to make ZFS default on any system, no matter how much memory it has. > > > As for the ZFS auto-tuning, I see that as being a separate discussion > > (which could ultimately change this recommendation, but shouldn't prevent > > us from committing this help text now). > > Agreed, but the lack of tuning should factor into the current recommendation. And I agree with Mike here, these tuning issues defanitly should affect our choices of default install recomendations. > Mike > > > Regards, > > Ben -- Rod Grimes rgrimes@freebsd.org