From owner-freebsd-fs@freebsd.org Tue Aug 25 02:39:06 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 946E83B696B; Tue, 25 Aug 2020 02:39:06 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BbCqZ3NXzz4cmj; Tue, 25 Aug 2020 02:39:06 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id 5190E10DDD; Tue, 25 Aug 2020 02:39:06 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lj1-f173.google.com with SMTP id g6so11974871ljn.11; Mon, 24 Aug 2020 19:39:06 -0700 (PDT) X-Gm-Message-State: AOAM530dii3UhZxdN+hSz98L4rnEp2kuUf2YQauuT1Uz4+cFgN5BKGOk oL/yQjAhE/MWGgy77jQharckOSEy6mXOK7D+fGc= X-Google-Smtp-Source: ABdhPJy56UhQae7VpPfjbSj4RzcmpOd3+g8GZ+lL91rP9ThNDX1Hgdd48bzYGyfS37qdEen/iaM1FuDsjrwVpdD+Sh8= X-Received: by 2002:a2e:b00c:: with SMTP id y12mr3944444ljk.18.1598323144939; Mon, 24 Aug 2020 19:39:04 -0700 (PDT) MIME-Version: 1.0 From: Matthew Macy Date: Mon, 24 Aug 2020 19:38:53 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: OpenZFS support merged To: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 02:39:06 -0000 r364746 merged OpenZFS support in to HEAD. The change should be transparent unless you want to use new features. I caution against 'zpool upgrade' for the next few weeks. https://svnweb.freebsd.org/base?view=revision&revision=364746 If you encounter problems please report them to me, Ryan Moeller, and -current. From owner-freebsd-fs@freebsd.org Tue Aug 25 19:30:06 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 960373B7E5D; Tue, 25 Aug 2020 19:30:06 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BbfG61VXnz3ytW; Tue, 25 Aug 2020 19:30:06 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.15.2/8.15.2) with ESMTPS id 07PJShVW055855 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 25 Aug 2020 12:28:43 -0700 (PDT) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.15.2/8.15.2/Submit) id 07PJShYb055854; Tue, 25 Aug 2020 12:28:43 -0700 (PDT) (envelope-from warlock) Date: Tue, 25 Aug 2020 12:28:43 -0700 From: John Kennedy To: Matthew Macy Cc: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org Subject: Re: OpenZFS support merged Message-ID: <20200825192843.GA10816@phouka1.phouka.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4BbfG61VXnz3ytW X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 19:30:06 -0000 On Mon, Aug 24, 2020 at 07:38:53PM -0700, Matthew Macy wrote: > r364746 merged OpenZFS support in to HEAD. > > The change should be transparent unless you want to use new features. > I caution against 'zpool upgrade' for the next few weeks. > > https://svnweb.freebsd.org/base?view=revision&revision=364746 > > If you encounter problems please report them to me, Ryan Moeller, and -current. So I've compiled it twice so far but mixed success. FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT #11 r364771+8ffcd478e5d5-c270900(master): Tue Aug 25 11:35:50 PDT 2020 warlock@bsd13:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 This is pretty much stock (no extra openzfs ports added, in particular) since you said "transparent". The system boots up, but the zfs binary has some issues. Only my zroot/ROOT/default (root partition) gets mounted, not the rest of the zfs filesystems. /sbin/zfs --version ld-elf.so.1: /lib/libzfs.so.4: Undefined symbol "mutex_init" strings -a < /sbin/zfs | grep mutex_init pthread_mutex_init That looks like it should be in lib pthread or lib thr, which are below: ldd /sbin/zfs /sbin/zfs: libjail.so.1 => /lib/libjail.so.1 (0x801084000) libavl.so.2 => /lib/libavl.so.2 (0x80108c000) libnvpair.so.2 => /lib/libnvpair.so.2 (0x801091000) libgeom.so.5 => /lib/libgeom.so.5 (0x8010ab000) libuutil.so.2 => /lib/libuutil.so.2 (0x8010b3000) libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x8010bf000) libspl.so.2 => /lib/libspl.so.2 (0x8010c8000) libtpool.so.2 => /lib/libtpool.so.2 (0x8010d1000) libzutil.so.2 => /lib/libzutil.so.2 (0x8010d7000) libzfs.so.4 => /lib/libzfs.so.4 (0x8010e4000) libm.so.5 => /lib/libm.so.5 (0x8011e7000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80121a000) libthr.so.3 => /lib/libthr.so.3 (0x8014ea000) libc.so.7 => /lib/libc.so.7 (0x801517000) libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x801940000) libsbuf.so.6 => /lib/libsbuf.so.6 (0x80196d000) libmd.so.6 => /lib/libmd.so.6 (0x801973000) libumem.so.2 => /lib/libumem.so.2 (0x801996000) libutil.so.9 => /lib/libutil.so.9 (0x80199a000) libz.so.6 => /lib/libz.so.6 (0x8019b2000) /etc/src.conf WITHOUT_REPRODUCIBLE_BUILD=YES # WITH_KERNEL_RETPOLINE=YES # WITH_RETPOLINE=YES WITH_PIE=YES WITH_BIND_NOW=YES WITHOUT_SVNLITE=YES WITHOUT_PORTSNAP=YES WITHOUT_LLVM_TARGET_ALL=YES TMPDIR=/var/tmp POUDRIERE_TMPDIR=/var/tmp From owner-freebsd-fs@freebsd.org Tue Aug 25 19:49:34 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EBB9A3B8652; Tue, 25 Aug 2020 19:49:34 +0000 (UTC) (envelope-from driesm.michiels@gmail.com) Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BbfhZ1w3rz41MC; Tue, 25 Aug 2020 19:49:34 +0000 (UTC) (envelope-from driesm.michiels@gmail.com) Received: by mail-ej1-x634.google.com with SMTP id j25so8967543ejk.9; Tue, 25 Aug 2020 12:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-transfer-encoding:thread-index:content-language; bh=Va4XFuMk0X9ZdhVszV6F3bc/UaPdJtA4ZU4RULWU7hY=; b=flhPYHxaYd39ZA/0ff1o89hj/51CAfeaMicRiCc6MLsWuDA0OuVjxrS+gPqAUKuknr HJxE+ICvm23U+fmfu6M2qW+kL+mUIW5IkvgtFNry3/QSBR8zfSiBXfZav5lYQ5lUX/12 HKYUBqhoqWDgxmVQYaR4KzSJHSYTnaAktQpuVU0ht6CqmR+bg5QUFKPTbMCelNi14l1b SjLjVE5GCzCIk/s0NCp2Ou/vOHeS825y1lBo0N1J9zEvfVpuDHv7h2bkYnDuBoc3YPRJ qYzUlkHWVG9CtH744KKgbFV9+j5TcVFkl7YdEWhvPn73sfgC6GyR8NQ3H4xrRPZ+OHtR IvgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=Va4XFuMk0X9ZdhVszV6F3bc/UaPdJtA4ZU4RULWU7hY=; b=g0Lfr7D+xayzRI2sgAYt8/MKsCdsB83LDgto+c8LXmY02xHeRTRQfPLxGbAF1a0ZiK TTMNGIv7Giu9xFdeWNMDgTLRgX83kgx879kFuzPQEHkvUInBXH05eOM7pjVltqVTQX6c RMclXSsMObaibVEVbmxwOKt7H60MYcWRKv6yrdFkKuH1ZzVeA8J5SA6/4qRm2HA/B3WW 9K23t0piX0msD0dgbU49yEvOpVKuWGGNScpzcTXOWbHKv694bktrSFhWrGvdEHF821Gw xZQBVXXrScBd/vmZr+kcEoGkF7dshd4Lm3x90qHClx/lQD/daSYLafBoMUBFWbYbGiJX a93w== X-Gm-Message-State: AOAM532cqex2x/R1hav7Oh9wK43TRCl/SIHw3hKojH7Hh9cS3xzPbF9U iitBdwfKrZ8a41xlMnLaMxz3+rDnA55QHWS6 X-Google-Smtp-Source: ABdhPJxxuKa/Udn6xqLmm9RuU4yHaI57il9AStRo9W5kLsBXdWqoEJ2chhNrX8M7cHjW33XubgU2kQ== X-Received: by 2002:a17:906:2801:: with SMTP id r1mr11953518ejc.17.1598384970281; Tue, 25 Aug 2020 12:49:30 -0700 (PDT) Received: from DRIESPC (ptr-8sijbm761zvigy0vczy.18120a2.ip6.access.telenet.be. [2a02:1811:2505:1601:99a2:61dd:148c:9b8e]) by smtp.gmail.com with ESMTPSA id g18sm10305392ejx.55.2020.08.25.12.49.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Aug 2020 12:49:29 -0700 (PDT) From: To: "'Matthew Macy'" , "'freebsd-current'" , "'freebsd-fs'" , References: In-Reply-To: Subject: RE: OpenZFS support merged Date: Tue, 25 Aug 2020 21:49:30 +0200 Message-ID: <000101d67b18$d9b33400$8d199c00$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQG45GjBjVIGvWFunPLuQOIzqL+9WqmEXz2A Content-Language: en-be X-Rspamd-Queue-Id: 4BbfhZ1w3rz41MC X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=flhPYHxa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of driesmmichiels@gmail.com designates 2a00:1450:4864:20::634 as permitted sender) smtp.mailfrom=driesmmichiels@gmail.com X-Spamd-Result: default: False [-1.87 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.82)[-0.818]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::634:from]; NEURAL_HAM_SHORT(-0.05)[-0.050]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-fs,freebsd-hackers]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 19:49:35 -0000 > -----Original Message----- > From: owner-freebsd-fs@freebsd.org On > Behalf Of Matthew Macy > Sent: Tuesday, 25 August 2020 04:39 > To: freebsd-current ; freebsd-fs = fs@freebsd.org>; freebsd-hackers@freebsd.org > Subject: OpenZFS support merged >=20 > r364746 merged OpenZFS support in to HEAD. First of all a big thank you to the whole team that made this possible! Secondly, I would like to check in if the following statement is still = in effect (taken from https://zfsonfreebsd.github.io/ZoF): "Beware that the FreeBSD boot loader does not allow booting from root = pools with encryption active (even if it is not in use), so do not try = encryption on a pool you boot from" When we are talking about encryption active, does that mean the feature = flag active or from the moment one sets encryption=3Don on any dataset of the = root pool? As a follow-up, if it is not currently not possible, are there any long/short term plans to add the missing bits to our boot loader?=B5 >=20 > The change should be transparent unless you want to use new features. > I caution against 'zpool upgrade' for the next few weeks. >=20 > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D364746 >=20 > If you encounter problems please report them to me, Ryan Moeller, and = - > current. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Tue Aug 25 20:02:31 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 760E83B92EE; Tue, 25 Aug 2020 20:02:31 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BbfzV3WN8z42p2; Tue, 25 Aug 2020 20:02:30 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-ed1-x52d.google.com with SMTP id b2so12371098edw.5; Tue, 25 Aug 2020 13:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=n7+3YVjO1Khzh+V316JgdFE82RCQMoHlp3StPrR87VE=; b=V8ygSlWiLqlwfSaO4cSqVJbz4Vo/m6dy8iP+YGJYF7fgD/oNlY8U+tk8KlC8Gh0sdd KokTohhtXQX0Cyo+HA95oFzSrb0mBDGTDernEv0+1DRDPSUs7brECyB6Lzxe/BNUb8LE VWVlwifjRRAxEPnqcqkftGLrZnXiXnzw6eNoK0YzwcMT0r0jX39vP5dYfmwY2geCvTwG zGaGDv5w2tuz9Id3ZnZgBZIIkdwH2UdT699xAgX2JrkxqRoAYIonJP8sfPOeGQ4iHgrB u3bZm6W1xjZVI+45d+KpL/B6cDPurtUUubclNXyC48ew+3KZWtclG179hAbjQfhu6i4m MFeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=n7+3YVjO1Khzh+V316JgdFE82RCQMoHlp3StPrR87VE=; b=Ciscpfq0ekpoq/EC+tYLnmMmeQbYerP+sqJyRJ+HdqSXGnwF95tUtAaCogw6Yz4Uk9 +dWc82W2YKWfrJ+F/k8acgls1M/zl4soSHLdZAnkPzfxigW3bB7TlVyysnBMy4aRge8H 22XwR6eqqujsbnp/KfPnvZkWGg6Myuy8sJwdt/5wfc5sNTh5gvmUauh7m13LrjgGKiFw 3ASM/vbKH25SXb+x6Cz2Q90kYXUJKWJcxsO01M/g5DfgegPhxwlXAP3M2bocFIAnUjZ9 uqGtfegDNQJDfYl5fjSkXVo77iHEsSHZJcEOWiXHCuSr0eArI06k2o1UETsWM4Yf0Lqa nuMQ== X-Gm-Message-State: AOAM533SpNjUyqgFc2k5j9rHYagoKtPFuOysrAyBXVBfyBysa9vKB9SB hw9ew9cpBabZc0AAuMVuI28k9kLnWPzlLNbfpFVFqFU= X-Google-Smtp-Source: ABdhPJwCE73KA1u8kN+v1Ph4Ym4Wt+Y35NNUDJSo9OdgwxajlvsUMjOMcNjluFKHP8GuA7UsTSXjrtIixbJSKnJjn/8= X-Received: by 2002:a05:6402:306f:: with SMTP id bs15mr11595419edb.249.1598385748689; Tue, 25 Aug 2020 13:02:28 -0700 (PDT) MIME-Version: 1.0 From: Zaphod Beeblebrox Date: Tue, 25 Aug 2020 16:02:15 -0400 Message-ID: Subject: Boot time TRIM ? To: freebsd-fs , FreeBSD Hackers X-Rspamd-Queue-Id: 4BbfzV3WN8z42p2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=V8ygSlWi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2a00:1450:4864:20::52d as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-1.48 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.74)[-0.743]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52d:from]; NEURAL_HAM_SHORT(-0.74)[-0.736]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-fs,freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 20:02:31 -0000 So, as I was looking at the performance of an NVME that I use for swap and ZFS cache, I realized after reboot, that since ZFS cache doesn't survive reboot and OBVIOUSLY swap doesn't, might it be best practice to issue a TRIM on boot? Now... a trim on the whole device from userland before adding swap in RC... might work. AFAIK, we still don't have any structure to swap before it's added. I'm pretty sure this is not the right thing for ZFS cache and log partitions, tho. Also, as a point of information, does ZFS issue TRIMs to the LOG after data gets committed into a transaction? I'm curious about this because for NVME that are a hybrid of different storage --- commonly 10-20 percent faster MLC and 80 percent slower TLC (or what-have-you) ... the TRIM usage can dramatically affect the performance. From owner-freebsd-fs@freebsd.org Tue Aug 25 23:05:59 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 859643BF5E3; Tue, 25 Aug 2020 23:05:59 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-vk1-xa34.google.com (mail-vk1-xa34.google.com [IPv6:2607:f8b0:4864:20::a34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bbl3C1Zccz4Hth; Tue, 25 Aug 2020 23:05:59 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-vk1-xa34.google.com with SMTP id y4so926vkn.12; Tue, 25 Aug 2020 16:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0FwiPkkWpxEs53S34AHQamAF97KRAhtcuFYlQPvDm0w=; b=roV6r+lQwJAbbSdFjHuYyOuMITH5amBLmwISl0YfNeypwKAvDD2IjZN5OvI9ZfxPiC DQyNTk3LXsEWy5l9mHjdAyMpseU82EGEsQ2hkWLNDEb2Bpt01cd8ZoWWBQxp5S1WIcoP tSZRrdVHxvecbumo8TZMbJ4gHExjZXE455SvD+hfy/xgae6K7mvekt68fBTnbJgKvisC 6JbhCD3pZzH5irmgSi4nVWA7I9vQo4lM/9YqKlQhqySW12mBNySn/gft/dsV1Oi2V6hz hyeQqKXGjB62pUfjRjXvqC4pVxmPcs41J1kVjiTqGsuramXG10CTb2wB4ztv/zDK7yi8 1s7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0FwiPkkWpxEs53S34AHQamAF97KRAhtcuFYlQPvDm0w=; b=QRgzM/lEtSLyWRTtgC0Wg8AkpUIo1Or/hjFanJxRRUoWRJy6Uemg0q+SfIPlsW9oBw E50dFIw4fyMrFOas6J/H4n0ISSQFukDXiHzvTpNvfjAHXPaqX7JXVfmLHSEckebJkIQp wBJF0fW/nbwqj76bhUNomvuO5NlkIAZrW3dfYHXUwGNxLdZMXi2KJFJaRdsx33gLNGsx l2Uzi7IkIBVWBm2hDLFZ3Sa4HT24ECPzuYKZOL6BF+IvKtQUloM+NN5hp2ew9HhY2NtD B7KfRdX9/6fvAhR3f0l8riQhUo1q8cpEUxI8jEzniTBDJY5xT9OipjxNdi1YaTumvIs+ hwAw== X-Gm-Message-State: AOAM5326JMcy54Cn47ruqKzG6k6+sg1y0DcO2dUWEg3Vgvf1IUi1dc8U JiwnEYtI3QK2/3YpL7Q4CSZVlqfEzT2kzmwCUdDfXJH9waM= X-Google-Smtp-Source: ABdhPJyBiuRc22VWkqdNTr3p4XA0uWQdVUpQxLuACK+Ssn7wvuMzwtJ7670ouNul7XqRZhPNm6eRiKjxSh7dx6sye74= X-Received: by 2002:a1f:9f87:: with SMTP id i129mr2400816vke.46.1598396757806; Tue, 25 Aug 2020 16:05:57 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ben Woods Date: Wed, 26 Aug 2020 07:05:46 +0800 Message-ID: Subject: Re: OpenZFS support merged To: Matthew Macy Cc: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 4Bbl3C1Zccz4Hth X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 23:05:59 -0000 On Tue, 25 Aug 2020 at 10:39 am, Matthew Macy wrote: > r364746 merged OpenZFS support in to HEAD. > > > > The change should be transparent unless you want to use new features. > > I caution against 'zpool upgrade' for the next few weeks. > > > > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D364746 Congratulations and thank you to all involved. This is a great step forward for FreeBSD. Looking forward to trying ZFS native encryption, and send/recv of encrypted datasets! I=E2=80=99ll second that boot loader support for ZFS native encryption and = zstd compression would greatly complement this change once they are ready. Regards, Ben > --=20 -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-fs@freebsd.org Wed Aug 26 00:35:51 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 736723C2B2D for ; Wed, 26 Aug 2020 00:35:51 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bbn2s6vLpz4Pkg for ; Wed, 26 Aug 2020 00:35:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72a.google.com with SMTP id g26so435899qka.3 for ; Tue, 25 Aug 2020 17:35:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=P4u62pshsC6POJDUgAjwHHpvMUMY6DW+4S1Vy0E+l0k=; b=gHxKzvDNG8MjG116KNlLi+Q825vofgbAASDhfWidvxW5lMJxBEhBC40XB8RI6I6OEq b78w8BPU19iE4yf21qJ1lsQnlzWWG3sthORPDORl4fAc2dGYFPVmqXTrCEcv0Ww9JkXg cpujehy4gg2+ydKtkReCSN0GqXd1fhju/FGX/iRmPvPGVknbTMezecZIwm8lm5g5VP86 lpmkfYRR5BXU87gqv0Sp6b1aO1OFjCCl5+oM1PV1cSjegeOot3uCX8GPYtHEOjWQwsRU sLG6I4gQ7te7TNXhFUJOYHgFCqroQ4TAv4vPIuoLKXG6Tix5f/3IJRJyqBGDQzYGEbqe lRZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=P4u62pshsC6POJDUgAjwHHpvMUMY6DW+4S1Vy0E+l0k=; b=ZUEp2sQTpOl0+6zV/Z3PNW2H8oe2YWA0Eovkgu4j5DAwMDkzjxIVO2IAcGWt6aTps6 /GeGrs910dXlWC/URztz0THohsMD3yE95mrXNVNq+SWROMPw2Tm4MEktpWS/LF7e+MfZ umMQWVEdWHBJB1auJs7k9frFlXMnxUStjan1VHIKcP/b8M/qU1U4Y6jQm1PXjEnr5InC BHMVX1R8Idg1V+ChqGmoalmze8Y0D50WfJ5GZ+nhTGLH1EUx6U9UsWgCXBK/1b+jaeZE 51ur+0Dy35Ct882ejQDhk+e6dfE+0ePAF+nladRUJH9kcAOZQeUVX3x8CflQrFkAYDeD FTWQ== X-Gm-Message-State: AOAM533bJYnQTidiWTYJ03m4G1861a0OHJzDUJadZauaMRyP6OmMChIQ NL9ScRqhyue+C8CeghgBclKQerB34p9bmmx8V4bVafINNuk= X-Google-Smtp-Source: ABdhPJyz5wP+YFSlW2l8NyDd+BS6DUrsFCwNUByFMp6FiEgLVFJxMXXTO3OA+OgWKIafTC1A5Al/BZbOYb+uULNSd5o= X-Received: by 2002:a05:620a:686:: with SMTP id f6mr12056281qkh.60.1598402148752; Tue, 25 Aug 2020 17:35:48 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 25 Aug 2020 18:35:34 -0600 Message-ID: Subject: Re: Boot time TRIM ? To: Zaphod Beeblebrox Cc: freebsd-fs , FreeBSD Hackers X-Rspamd-Queue-Id: 4Bbn2s6vLpz4Pkg X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=gHxKzvDN; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72a) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [0.97 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.12)[-0.122]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.09)[0.091]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72a:from]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-fs]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Aug 2020 00:35:51 -0000 On Tue, Aug 25, 2020, 2:02 PM Zaphod Beeblebrox wrote: > So, as I was looking at the performance of an NVME that I use for swap and > ZFS cache, I realized after reboot, that since ZFS cache doesn't survive > reboot and OBVIOUSLY swap doesn't, might it be best practice to issue a > TRIM on boot? > No. Large TRIMs suck on too many devices. It might be helpful on a few devices. Some SSDs I've used would take minutes to trim... Now... a trim on the whole device from userland before adding swap in RC... > might work. AFAIK, we still don't have any structure to swap before it's > added. Not quite. We have to avoid trimming the first 8k to avoid stepping on boot blocks. The rest would be fair game. I'm pretty sure this is not the right thing for ZFS cache and log > partitions, tho. > Likely. ZFS should likely trim the useless areas. Also, as a point of information, does ZFS issue TRIMs to the LOG after data > gets committed into a transaction? > As it opens up holes, it will trim. FreeBSD ZFS and OpenZFS have different trim strategies, though, and I've not studied OpenZFS well enough. The FreeBSD implementation schedules TRIMs in batches... I'm curious about this because for NVME that are a hybrid of different > storage --- commonly 10-20 percent faster MLC and 80 percent slower TLC (or > what-have-you) ... the TRIM usage can dramatically affect the performance. > Yes, but the answer is it depends. The drive's firmware manages these things. Usually the mlc/slc bits of the drive are reserved for a pool to write quickly to. Then the drive rewrites this data to TLC when the free pool of blocks gets small enough. I did some testing of this at work wrt swap, but we didn't swap enough for this to matter... Warner _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Wed Aug 26 02:22:44 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC6913C6242 for ; Wed, 26 Aug 2020 02:22:44 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BbqQD2QMNz4WkY for ; Wed, 26 Aug 2020 02:22:44 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 6659B1E6CE; Wed, 26 Aug 2020 02:22:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 6659B1E6CE From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: ZSTD Project Weekly Status Update Message-ID: Date: Tue, 25 Aug 2020 22:22:33 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe" X-Rspamd-Queue-Id: 4BbqQD2QMNz4WkY X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Aug 2020 02:22:44 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe Content-Type: multipart/mixed; boundary="wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3"; protected-headers="v1" From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer Message-ID: Subject: Re: ZSTD Project Weekly Status Update References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> In-Reply-To: <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> --wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable This is the tenth weekly status report on my FreeBSD Foundation sponsored project to complete the integration of ZSTD compression into OpenZFS. Late last week the main pull request was merged, and ZSTD support is now part of OpenZFS's trunk branch. Last night, OpenZFS with ZSTD was imported into FreeBSD's -current branch= =2E I am continuing to work on a number of things related to ZSTD, including future-proofing support (so upgrading ZSTD won't cause problems with features like nopwrite), and improving the integration of ZSTD into FreeBSD, including enabling support for booting from ZSTD compressed datasets, and improving the performance of ZSTD on FreeBSD. I'll also be adding some additional tests to make sure we detect any issues when we do look at updating ZSTD. Additionally, I am working on a bunch of documentation around using ZSTD in ZFS. For my benchmarking of ZSTD, I have been using a zfs recv of a stream in a file on a tmpfs, and recording how long it takes to receive and sync the data. The test data is a copy of the FreeBSD 12.1 source code, since that is easily reproducible. Does anyone have experience or a better suggestion on how to get the most consistent and repeatable results when benchmarking like this? On 2020-08-18 18:51, Allan Jude wrote: > This is the ninth weekly status report on my FreeBSD Foundation > sponsored project to complete the integration of ZSTD compression into > OpenZFS. >=20 > https://github.com/openzfs/zfs/pull/10693 - The L2ARC fixes (for when > compressed ARC is disabled) have been merged. >=20 > https://github.com/openzfs/zfs/pull/10278/ - A number of other cleanups= > and fixes for the ZSTD have been integrated and squashed, and it looks > like the completed ZSTD feature will be merged very soon. >=20 > This included a bunch of fixes for makefiles and runfiles to hook the > tests I added up to the ZFS test suite so they are run properly. >=20 > It looks like this will mean that the ZSTD feature will be included in > OpenZFS 2.0. Thanks for everyone who has tested, reviewed, or > contributed to this effort, especially those who kept it alive while I > was working on other things. >=20 > Post-merge, the remaining work is to develop future-proofing around ZST= D > so that we will be able to more seamlessly upgrade to newer releases of= > ZSTD. Recompression of the same input resulting in the same on-disk > checksum is the main concern, as without this upgrading the compression= > algorithm will break features like nop-write. >=20 > This project is sponsored by the FreeBSD Foundation. >=20 --=20 Allan Jude --wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3-- --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfRcdsAAoJEBmVNT4SmAt+tpoQAN1PAjBf3b9hIyhRZ0foAiSP 6Yz7KeNNwvUHb7tlYrEpKIMiYftOzqVRM1hzkn1fZvjcbET0avAXFP7tlfKGADiK VVOZJuT4QcF3FQazG7eeOG1Qe6kQNXmqqMsV7kipJtHAdGGy3pjS701K6K2wg7ka J1vLcyDkMk+w++IVQlrQwI/WocActnaraeBLqk8aj8SX0tsj4NpU9BEaSslKeiZU 873MyiY/QVfITahuKjoKXq7Hj6cBQPW5KGVZ9LBlXeYXp9usIZZ+i8XR9xHFpXa6 dSVL79o4p4nTlNghZ9wLKfkGtN+Isd7atCQkdHZK9I9Xdb6enlYw8nkpVsvheFWP vOw+cE25fTAYAaXKa7X5UTV0+Ns9H6by5/fBlblqpkE4ZJK+dERfiQCRaYdti02U blH3MbfC8vFk485HMKK2M43/IJyjNrbca+NUIpI4ry3ipxTE3MsFJYf5iZNpSzcC RlF9QJpFJWkI6ewjYxijNtw/74441H+rUSFQ00GKTlLiJUKzy+sLg4rPjRjgku1W HJ1ffF/ZROiL+sH2KaPSUXukTiNJ8DMMmLvvb1nBoJ1cjPJUIV3NbedsmyQ8Z8VK gCz1mrjASM4GfFwaWG3gW4FWGVXxREwJpU2YqlSY0DZkDjWXhq5X77F0LG5d/Eli 8vbIHl4QCJafj4+FIvSX =MofC -----END PGP SIGNATURE----- --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe-- From owner-freebsd-fs@freebsd.org Thu Aug 27 17:37:26 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0DBEF3BA149; Thu, 27 Aug 2020 17:37:26 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bcqg910Kjz41F0; Thu, 27 Aug 2020 17:37:24 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.15.2/8.15.2) with ESMTPS id 07RHa2N7087496 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 27 Aug 2020 10:36:02 -0700 (PDT) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.15.2/8.15.2/Submit) id 07RHa2ka087495; Thu, 27 Aug 2020 10:36:02 -0700 (PDT) (envelope-from warlock) Date: Thu, 27 Aug 2020 10:36:02 -0700 From: John Kennedy To: Matthew Macy Cc: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org Subject: Re: OpenZFS support merged Message-ID: <20200827173602.GL10816@phouka1.phouka.net> References: <20200825192843.GA10816@phouka1.phouka.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200825192843.GA10816@phouka1.phouka.net> X-Rspamd-Queue-Id: 4Bcqg910Kjz41F0 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of warlock@phouka1.phouka.net has no SPF policy when checking 107.170.196.116) smtp.mailfrom=warlock@phouka1.phouka.net X-Spamd-Result: default: False [3.00 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[phouka.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.71)[0.713]; NEURAL_SPAM_SHORT(0.32)[0.323]; NEURAL_SPAM_LONG(0.77)[0.765]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[warlock@phouka.net,warlock@phouka1.phouka.net]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[warlock@phouka.net,warlock@phouka1.phouka.net]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-fs,freebsd-hackers] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Aug 2020 17:37:26 -0000 On Tue, Aug 25, 2020 at 12:28:43PM -0700, John Kennedy wrote: > On Mon, Aug 24, 2020 at 07:38:53PM -0700, Matthew Macy wrote: > > r364746 merged OpenZFS support in to HEAD. > > > > The change should be transparent unless you want to use new features. > > I caution against 'zpool upgrade' for the next few weeks. > > > > https://svnweb.freebsd.org/base?view=revision&revision=364746 > > > > If you encounter problems please report them to me, Ryan Moeller, and -current. > > So I've compiled it twice so far but mixed success. ... > /sbin/zfs --version > ld-elf.so.1: /lib/libzfs.so.4: Undefined symbol "mutex_init" Following up for mailing-list readers, I had my kernel compiled with WITH_BIND_NOW=YES which caught some otherwise hidden problems with the shared libraries. All the problems I've noticed are fixed via r364830, r364861 and probably r364863. Thanks Ryan Moeller, and thanks ZFS folks. From owner-freebsd-fs@freebsd.org Fri Aug 28 22:03:22 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2063B3C37E2 for ; Fri, 28 Aug 2020 22:03:22 +0000 (UTC) (envelope-from mason@blisses.org) Received: from phlegethon.blisses.org (phlegethon.blisses.org [50.56.97.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BdYWX52vzz4VqR for ; Fri, 28 Aug 2020 22:03:20 +0000 (UTC) (envelope-from mason@blisses.org) Received: from cocytus.blisses.org (service.blisses.org [64.223.129.151]) by phlegethon.blisses.org (Postfix) with ESMTP id 2143D194D03 for ; Fri, 28 Aug 2020 18:03:14 -0400 (EDT) Received: from blisses.org (acheron.int.blisses.org [10.0.1.10]) by cocytus.blisses.org (Postfix) with ESMTPSA id BDC5EA1 for ; Fri, 28 Aug 2020 18:03:13 -0400 (EDT) Date: Fri, 28 Aug 2020 18:03:12 -0400 From: Mason Loring Bliss To: freebsd-fs@freebsd.org Subject: Maildir on ZFS Message-ID: <20200828220312.GD6456@blisses.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="FFoLq8A0u+X9iRU8" Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 4BdYWX52vzz4VqR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mason@blisses.org designates 50.56.97.101 as permitted sender) smtp.mailfrom=mason@blisses.org X-Spamd-Result: default: False [-4.88 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.97)[-0.967]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[blisses.org]; NEURAL_HAM_SHORT(-0.53)[-0.526]; NEURAL_HAM_MEDIUM(-0.98)[-0.985]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:19994, ipnet:50.56.0.0/17, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Aug 2020 22:03:22 -0000 --FFoLq8A0u+X9iRU8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I sent this to BSD Now a month ago and haven't heard anything back, but I bet there are opinions to be had in here! Thanks in advance. I'm migrating a mail server to a FreeBSD jail, and I've seen conflicting notes about how best to tune ZFS for holding accounts with Maildirs. My current thinking is that I don't want to change recordsize as it's an upper limit, not a minimum, but that I might want to set primarycache=3Dmetadata = so I'm caching information about Maildir contents, but not the Maildir contents themselves. But are these valid thoughts, and what else is there to think about? The problem is that I don't know what I don't know, and I'm not sure about what I do know, so I'd love some advise about how best to tune a dataset for Maildirs. Thanks! --=20 Mason Loring Bliss (( If I have not seen as far as others, it is because mason@blisses.org )) giants were standing on my shoulders. - Hal Abels= on --FFoLq8A0u+X9iRU8 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEEXtBZz1axB5rEDCEnrJXcHbvJVUFAl9Jfx8ACgkQnrJXcHbv JVW+Jg//SRPWBIAwPzw86nEBdqECfqkp/bnErnkZyADlRNt/+A7MPliXo5hN5N9E IOjytjbrQrgzNfnMQawnUAbVtzv8ytrqfIB6DPAOUaegU9lhBZ1UE00lPNmEnLu3 7kHGMCgWZQyR2+91q8N/N1L+q9cEiOeQpt+1SlBHvYP0LEaRxhvS2pliXxjge3kh MLiU4J4Tn6Xc62t2tCiV7jIYDLQAmQgDi1zo2L6CPKw5nhQYUm0wBYG3l5Cafxws 8ya3lZszQ910X6VOVs+FUYDDAUA0dOPW6kDBGD/IxwZKhNagBKoD1sasfr87j/jU xG8JJTI/X0O/LJbVAeXj8zuRpawASEYSzkDRrr//CuOlvP1C9SGMhon7N1gRBb3I ralwvbU5O86huhZGXSvx9OcvkRnjTEWXUNbEOE8K1a0lWTMVHyUgcD+XjUmZCtau cihllASrV5I0MwzFvghE2XwvDCzZkEth4qTQtg9frGg3zNMPjeyybKwuqqnp7lDx cofTyplx6Cd8wg3xmXgMFwO1At2WORk0XWxBHOV/aYir+wQOlRRzvR+2Lg0dO0Zo 90ieJpMARYbYJ5uXGQPBZnOzt4a4sNhYud5qPUnjtj+867NmRtR9tLjqpqnXlUQh 9rwWFwmUiRZMosRaZ/fkZ9WnnqvZ7AsJj8strp1VF24ZvQZhSgY= =jrcF -----END PGP SIGNATURE----- --FFoLq8A0u+X9iRU8-- From owner-freebsd-fs@freebsd.org Fri Aug 28 22:45:02 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E69883C4D8C for ; Fri, 28 Aug 2020 22:45:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BdZRf1s8rz4YWW for ; Fri, 28 Aug 2020 22:45:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:d140:d4f:4615:ffde] ([IPv6:2607:f3e0:0:4:d140:d4f:4615:ffde]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id 07SMixtU015717 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 28 Aug 2020 18:44:59 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: Maildir on ZFS To: Mason Loring Bliss , freebsd-fs@freebsd.org References: <20200828220312.GD6456@blisses.org> From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWiQ== Message-ID: Date: Fri, 28 Aug 2020 18:45:00 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20200828220312.GD6456@blisses.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4BdZRf1s8rz4YWW X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.11 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; NEURAL_HAM_LONG(-0.97)[-0.970]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.21)[-1.212]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.93)[-0.927]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Aug 2020 22:45:03 -0000 Havent done benchmarking to see what sort of difference tweaking primarycache makes on our mail server. But I tried on a very busy sftp server where some directories had 500,000+ files in them.  It didnt seem to make a difference performance wise for me, at least when fiddling with min cache sizes etc.  Lots of RAM and fast disks.  Our mail spool is on the equiv of raid10 and thats "spinning rust" and it keeps up well.   I would approach it as your hardware should be able to meet its functional requirements in the default configuration.  Then, looking at zfs-stats and cache statistics, see what makes a difference for tuning.   I dont think its gonna be a game changer either way.  Not sure about record sizes. L2ARC doesnt seem to help for our situation (RELENG_11) ARC Efficiency:                                 35.40b         Cache Hit Ratio:                98.39%  34.83b         Cache Miss Ratio:               1.61%   571.23m         Actual Hit Ratio:               98.28%  34.79b         Data Demand Efficiency:         98.90%  3.45b         Data Prefetch Efficiency:       75.07%  23.39m         CACHE HITS BY CACHE LIST:           Most Recently Used:           4.94%   1.72b           Most Frequently Used:         94.94%  33.07b           Most Recently Used Ghost:     0.05%   17.29m           Most Frequently Used Ghost:   0.16%   54.30m         CACHE HITS BY DATA TYPE:           Demand Data:                  9.79%   3.41b           Prefetch Data:                0.05%   17.56m           Demand Metadata:              87.97%  30.64b           Prefetch Metadata:            2.19%   762.76m         CACHE MISSES BY DATA TYPE:           Demand Data:                  6.62%   37.83m           Prefetch Data:                1.02%   5.83m           Demand Metadata:              74.66%  426.48m           Prefetch Metadata:            17.70%  101.09m ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY)         Passed Headroom:                        29.46m         Tried Lock Failures:                    3.05m         IO In Progress:                         27.85k         Low Memory Aborts:                      4         Free on Write:                          2.41m         Writes While Full:                      31.85k         R/W Clashes:                            0         Bad Checksums:                          0         IO Errors:                              0         SPA Mismatch:                           17.47b L2 ARC Size: (Adaptive)                         136.03  GiB         Header Size:                    0.11%   156.63  MiB L2 ARC Evicts:         Lock Retries:                           440         Upon Reading:                           7 L2 ARC Breakdown:                               571.23m         Hit Ratio:                      4.53%   25.88m         Miss Ratio:                     95.47%  545.35m         Feeds:                                  10.88m L2 ARC Buffer:         Bytes Scanned:                          1022.25 TiB         Buffer Iterations:                      10.88m         List Iterations:                        43.51m         NULL List Iterations:                   362.24k L2 ARC Writes:         Writes Sent:                    100.00% 3.94m     ---Mike On 8/28/2020 6:03 PM, Mason Loring Bliss wrote: > I sent this to BSD Now a month ago and haven't heard anything back, but I > bet there are opinions to be had in here! Thanks in advance. > > > > I'm migrating a mail server to a FreeBSD jail, and I've seen conflicting > notes about how best to tune ZFS for holding accounts with Maildirs. My > current thinking is that I don't want to change recordsize as it's an upper > limit, not a minimum, but that I might want to set primarycache=metadata so > I'm caching information about Maildir contents, but not the Maildir > contents themselves. But are these valid thoughts, and what else is there > to think about? The problem is that I don't know what I don't know, and I'm > not sure about what I do know, so I'd love some advise about how best to > tune a dataset for Maildirs. > > Thanks! > From owner-freebsd-fs@freebsd.org Sat Aug 29 08:46:54 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 549FF3D2182 for ; Sat, 29 Aug 2020 08:46:54 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bdqp459WBz47dP for ; Sat, 29 Aug 2020 08:46:52 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.110.112]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1kBwVd-00016x-1m; Sat, 29 Aug 2020 10:46:50 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klop.ws; s=mail; h=In-Reply-To:Message-ID:From:Content-Transfer-Encoding:MIME-Version: Date:References:Subject:To:Content-Type:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=qQV+8WEMzHN3cFR+yMpkSbwO5c4xKYTW89Ox4l2C988=; b=MgkiKQO0EAWD1+Z5HlzKVYodH8 IMt3FiuxQBj3kSm4sjhaj+OXoj8M4nJ6GASYzHHiPiEGsaBpV6GxcT4CDNbn+x6BTHj/6v33CGbgT GcNcOSZ6nK5/ImOQBwQfNN2mnQfJZbI+6CbBli3lhHzbdVFeUtUmirVsm/es6JNljYts=; Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-fs@freebsd.org, "Mason Loring Bliss" Subject: Re: Maildir on ZFS References: <20200828220312.GD6456@blisses.org> Date: Sat, 29 Aug 2020 10:29:24 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: <20200828220312.GD6456@blisses.org> User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.4 X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED, BAYES_50, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF autolearn=disabled version=3.4.2 X-Scan-Signature: ba572e8a3bde05b4b19613c12a9e49fc X-Rspamd-Queue-Id: 4Bdqp459WBz47dP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=mail header.b=MgkiKQO0; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 195.190.28.88 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-2.79 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; R_DKIM_ALLOW(-0.20)[klop.ws:s=mail]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[195.190.28.88:from]; R_SPF_ALLOW(-0.20)[+ip4:195.190.28.64/27]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[klop.ws]; NEURAL_HAM_LONG(-0.98)[-0.980]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[195.190.28.88:from]; NEURAL_HAM_SHORT(-0.82)[-0.816]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:47172, ipnet:195.190.28.0/24, country:NL]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Aug 2020 08:46:54 -0000 On Sat, 29 Aug 2020 00:03:12 +0200, Mason Loring Bliss wrote: > I sent this to BSD Now a month ago and haven't heard anything back, but I > bet there are opinions to be had in here! Thanks in advance. > > > > I'm migrating a mail server to a FreeBSD jail, and I've seen conflicting > notes about how best to tune ZFS for holding accounts with Maildirs. My > current thinking is that I don't want to change recordsize as it's an > upper > limit, not a minimum, but that I might want to set primarycache=metadata > so > I'm caching information about Maildir contents, but not the Maildir > contents themselves. But are these valid thoughts, and what else is there > to think about? The problem is that I don't know what I don't know, and > I'm > not sure about what I do know, so I'd love some advise about how best to > tune a dataset for Maildirs. > > Thanks! > It depends on the clients also. Do they retrieve mail once, store it locally and only check the server for updates, than metadata might be enough, but if it is mostly webmail clients which retrieves the same mail multiple times, than caching data is probably beneficial. But it also depends on the amount of data you store on what medium (HDD/SSD/RAID) and how much memory your server has. As you don't specify a lot of information about the specifics it is hard to say. Is NFS involved or just ZFS? NB: Maildir is very (Z)FS friendly by having a lot of static data on disk. Dovecot creates some indices for fast retrieval of metadata. You can store these indices on tmpfs also or on another volume/dataset with other primarycache settings. The other advice is: test your own system. In the other reply I saw advice about how to look at cache statistics. ZFS is very fast without tuning already. Regards, Ronald. From owner-freebsd-fs@freebsd.org Sat Aug 29 19:21:11 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8E2D3B9E91 for ; Sat, 29 Aug 2020 19:21:11 +0000 (UTC) (envelope-from mason@blisses.org) Received: from phlegethon.blisses.org (phlegethon.blisses.org [50.56.97.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bf5sy5TXvz3fcZ for ; Sat, 29 Aug 2020 19:21:10 +0000 (UTC) (envelope-from mason@blisses.org) Received: from cocytus.blisses.org (service.blisses.org [64.223.129.151]) by phlegethon.blisses.org (Postfix) with ESMTP id 2703A194D1A; Sat, 29 Aug 2020 15:21:09 -0400 (EDT) Received: from blisses.org (acheron.int.blisses.org [10.0.1.10]) by cocytus.blisses.org (Postfix) with ESMTPSA id B9289A1; Sat, 29 Aug 2020 15:21:08 -0400 (EDT) Date: Sat, 29 Aug 2020 15:21:07 -0400 From: Mason Loring Bliss To: mike tancsa , Ronald Klop Cc: freebsd-fs@freebsd.org Subject: Re: Maildir on ZFS Message-ID: <20200829192107.GE6456@blisses.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="0QFb0wBpEddLcDHQ" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 4Bf5sy5TXvz3fcZ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mason@blisses.org designates 50.56.97.101 as permitted sender) smtp.mailfrom=mason@blisses.org X-Spamd-Result: default: False [-5.24 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[blisses.org]; NEURAL_HAM_LONG(-0.95)[-0.953]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.91)[-0.909]; NEURAL_HAM_MEDIUM(-0.98)[-0.981]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:19994, ipnet:50.56.0.0/17, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Aug 2020 19:21:11 -0000 --0QFb0wBpEddLcDHQ Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 28, 2020 at 06:45:00PM -0400, mike tancsa wrote: > But I tried on a very busy sftp server where some directories had > 500,000+ files in them.=A0 It didnt seem to make a difference performance > wise for me, at least when fiddling with min cache sizes etc. That's encouraging. I guess that's the anwer - try it and see if there are any observable issues. I've seen lots of complaints about IMAP on IRC and vague noises when searching online, but nothing definitive. Maybe there is nothing definitive. On Sat, Aug 29, 2020 at 10:29:24AM +0200, Ronald Klop wrote: > It depends on the clients also. Do they retrieve mail once, store it > locally and only check the server for updates, than metadata might be > enough, but if it is mostly webmail clients which retrieves the same mail > multiple times, than caching data is probably beneficial. It's mostly Mutt and Thunderbird. I'm not wholly familiar with what Thunderbird does, but Mutt just caches headers and looks for changes. > But it also depends on the amount of data you store on what medium > (HDD/SSD/RAID) and how much memory your server has. Hm? RAID? This'll be ZFS mirrors for this application. Everything I run is either a mirror or raidz2 lately. Lots of memory. Spinning rust. Just ZFS and imapd running in a jail, where the jail has one dataset to itself and doesn't important anything over the network. > NB: Maildir is very (Z)FS friendly by having a lot of static data on disk. > Dovecot creates some indices for fast retrieval of metadata. It'll almost certainly be Dovecot, and that sounds encouraging. (I used to run Courier stuff and that's often tempting, but Dovecot has been pain-free for years now.) > The other advice is: test your own system. In the other reply I saw advice > about how to look at cache statistics. ZFS is very fast without tuning > already. Yar. I was mostly wondering if there were any obvious pitfalls or things that I'm very likely to hit, but it looks like there probably aren't. Once I've got things migrated and seen it in action I'll gather some data and share my results. Thank you both for your input. --=20 Mason Loring Bliss mason@blisses.org They also surf, who only stand on waves. --0QFb0wBpEddLcDHQ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEEXtBZz1axB5rEDCEnrJXcHbvJVUFAl9Kqp4ACgkQnrJXcHbv JVWK2A//UbqmCHK2nCBpJIRGw3BdtLjWg2NnpNQcT44oZ87scMtetBfSOfwHJk5a xF7iTkYZhOrD7S/jlPHxyK6wo4Y+bfT+oxmMaRMAdgylQ1OXeeFWlBty449Ah2iA 4OKs3A6z7AnuAUzOBbxWOtCUjT3suvlDLTgHSWtDrvYgQdLT/5q7yN4N9dNkNSFb xcbR4zwVJMnOn2xNxP9DrIHAAiIHmMT56z0wBlgswuHkCRff0jB465DM8BnGDNhK U4/nVpfeB6aV5EA2xT/8ULSVFdy4r9Ge+sDnFEVIDK6G7oZfZticBlvZW0KvG4MX 0jWlqBbnTy1tyL7F+Yg/R6Ir4Tlj6ijfpJhet0XVZ1FkKVhlAHmW9tgF3ePFkKuV EpLNBZjtVcz4FnrtKtxhNon28RyoMBN0L9Gl+PWMmIr3EjXuhCVrJtthhNzgEdkd u6IkNsLkaGTdCh+ZQ/oDwrHTBvxtJT1pZWvBuGPgC4n2nnd3UBeUd2+liOIAt3lu DnXZPIOPQeVk6Hz25mn/HCHv8dnZiKbTKVshomWC5wg599Fd7TTCRBkUpnSb5n5Q jS3YDqTzs3AkAmK1aiaFpNOiH3++Vw4bFP6f4KBYk1iHZZOPHU1kLoIeOE5p6kPY kY9ZyJyFoIvUTsBfL8vYCB0TwW0Xj0iw4w4ROrj6qE5ejZ4iBmg= =8/XQ -----END PGP SIGNATURE----- --0QFb0wBpEddLcDHQ--