From owner-freebsd-questions@freebsd.org Tue Apr 6 18:53:10 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7A81A5B4987 for ; Tue, 6 Apr 2021 18:53:10 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "holgerdanske.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FFGr52kVLz4ZGQ for ; Tue, 6 Apr 2021 18:53:09 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Tue, 6 Apr 2021 11:53:00 -0700 Subject: Re: advise or best practices with ZFS snapshot To: freebsd-questions@freebsd.org References: From: David Christensen Message-ID: <52857d52-152e-335f-cb9a-97f342dc006b@holgerdanske.com> Date: Tue, 6 Apr 2021 11:53:00 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4FFGr52kVLz4ZGQ X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of dpchrist@holgerdanske.com has no SPF policy when checking 2001:470:0:19b::b869:801b) smtp.mailfrom=dpchrist@holgerdanske.com X-Spamd-Result: default: False [1.90 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:0:19b::b869:801b:from]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.13.84.17:email]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2001:470:0:19b::b869:801b:from:127.0.2.255]; DMARC_NA(0.00)[holgerdanske.com]; DBL_PROHIBIT(0.00)[0.13.84.17:email]; NEURAL_SPAM_SHORT(1.00)[0.999]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; R_DKIM_NA(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2021 18:53:10 -0000 On 4/5/21 8:41 PM, sesquivels--- via freebsd-questions wrote: > Geom name: ada1 > Providers: > 1. Name: ada1 >    Mediasize: 128035676160 (119G) >    Sectorsize: 512 >    Mode: r1w1e2 >    descr: MTFDDAK128MAM-1J1 >    lunid: 500a0751039d7440 >    ident: 1405039D7440 >    rotationrate: 0 >    fwsectors: 63 >    fwheads: 16 What is this drive used for? > zpool status >   pool: zroot > state: ONLINE >   scan: scrub repaired 0 in 0 days 01:43:28 with 0 errors on Thu Mar 18 01:57:21 2021 > config: > >         NAME        STATE     READ WRITE CKSUM >         zroot       ONLINE       0     0     0 >           raidz1-0  ONLINE       0     0     0 >             ada4p3  ONLINE       0     0     0 >             ada5p3  ONLINE       0     0     0 >             ada6p3  ONLINE       0     0     0 >             ada7p3  ONLINE       0     0     0 >           raidz1-2  ONLINE       0     0     0 >             ada0    ONLINE       0     0     0 >             ada2    ONLINE       0     0     0 >             ada3    ONLINE       0     0     0 >             ada8    ONLINE       0     0     0 >         logs >           nvd0p2    ONLINE       0     0     0 >         cache >           nvd0p1    ONLINE       0     0     0 > > errors: No known data errors So, your pool is comprised of two raidz1 vdevs of 4 @ 1 TB each, an NVMe partition for log, and an NVMe partition for cache. OS pools are named 'zpool' by the FreeBSD installer. Data pools are conventionally named 'tank' (I use the letter 'p' followed by a unique number). This article has some good ideas: https://b3n.org/zfs-hierarchy/ Is the above your OS pool? If your log device fails, you may lose data. The recommended practice is to use a vdev with redundancy for the log (e.g. 2 drive mirror, 3 drive raidz1, etc.). https://docs.oracle.com/cd/E19253-01/819-5461/gazgw/index.html Using a partition on your NVMe device for the cache should help performance. > About zfs administration, and snapshots, where can I found more information? I like the books by Lucas: https://mwl.io/nonfiction/os#af3e https://mwl.io/nonfiction/os#fmzfs https://mwl.io/nonfiction/os#fmaz > https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.html > zfs list -t snapshot > NAME USED AVAIL REFER MOUNTPOINT > zroot/Dom@03252021 32.6G - 353G - My pools were created with early versions of FreeBSD 11R and 12R, and do not have the 'listsnapshots' property. This is how I look at snapshots of a given filesystem: 2021-04-06 09:48:42 toor@f3 ~ # zfs list -t all -d 1 p3/ds2 | grep @ | tail p3/ds2@manual-20210217-1553 0 - 88K - p3/ds2@manual-20210226-2334 0 - 88K - p3/ds2@manual-20210228-1807 0 - 88K - p3/ds2@manual-20210301-1524 0 - 88K - p3/ds2@manual-20210306-0009 0 - 88K - p3/ds2@manual-20210314-1201 0 - 88K - p3/ds2@manual-20210321-1355 0 - 88K - p3/ds2@manual-20210328-1520 0 - 88K - p3/ds2@manual-20210328-2306 0 - 88K - p3/ds2@manual-20210404-1338 0 - 88K - > https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html > Doing a zfs send to file as this, may I delete [snapshot] from my zpool and then restore it using zfs receiver? > zfs send zroot/Dom@03252021 | gzip > /mnt/bkp/march2021.gz Do you understand that the above command will not backup any child filesystems of zroot/Dom? Nor any other snapshots? If and when you restore that filesystem, you may want to validate it. mtree(8) is one option: https://www.techrepublic.com/blog/it-security/use-mtree-for-filesystem-integrity-auditing/ Once you figure out how to properly backup, restore, and validate your filesystem(s), you will need to think about backup redundancy and aging policies. I would retain snapshots on the live system until it needed more space. I would retain backup files on the backup device or system until it needed more space. Are you aware of zfs-auto-snapshot? # pkg install zfstools zfs-auto-snapshot is configured via crontab(1): 2021-04-06 10:30:36 toor@f3 ~ # crontab -l | grep ds2 3 * * * * /usr/local/sbin/zfs-auto-snapshot -P p3/ds2 -k h 24 15 3 * * 0 /usr/local/sbin/zfs-auto-snapshot -P p3/ds2 -k w 4 21 3 1 * * /usr/local/sbin/zfs-auto-snapshot -P p3/ds2 -k m 24 27 3 1 1 * /usr/local/sbin/zfs-auto-snapshot -P p3/ds2 -k y 99 and via ZFS properties: 2021-04-06 10:33:05 toor@f3 ~ # zfs get all p3 p3/ds2 | grep auto-snapshot p3 com.sun:auto-snapshot true local p3/ds2 com.sun:auto-snapshot true inherited from p3 I write scripts for repetitive operations to get accuracy and consistency. My collection of scripts has grown and evolved as I have learned. ZFS may appear simple going in, but is non-trivial in practice. I had more than a few painful accidents early on. I suggest that you build a virtual machine, some pools, and some filesystems to learn and practice with. Go through the various use-cases and make sure that you understand what is happening before you attempt them on live data. David