From nobody Tue Sep 20 21:47:43 2022 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MXFXC0qpMz4cKtJ for ; Tue, 20 Sep 2022 21:47:55 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [184.105.128.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MXFXB0py7z3QGj for ; Tue, 20 Sep 2022 21:47:54 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1663710467; bh=yIGbo7d3o6Wo5/A/UHxrp3IGvI7bK4OI1Z41Og0EFh4=; h=Received:Message-ID:Date:MIME-Version:User-Agent:Subject: Content-Language:To:References:From:In-Reply-To:Content-Type: Content-Transfer-Encoding; b=Ce269jhT0HFex1lAajoncTklwZ225HDARVR/6/Wz/sOCTyNktpk/XEclR5LV5L/DZ yjOzToOIh+XmSXVUwssNGgSN4/hDVqq7cyJYuNpcSvTrPfvQ6DY1MqAHQPBrFql+3h YsRkSS/M9/PYmvFcuOj0p7E4yOPb7ot9XOhAhg2JfCMpeVt8WkDwpD3klyKInpsKlb Mx6tDgCNCzFDpL7R0QsFbJPTNSTKsJF84dXavgjNfdXX/PXQHMzL0cwEkcV3irj52F sMStHIkq4gpSdSEUou2MBRGtm3a6S//YjnK53Src9wnP4h4lJkW6J/ZRvyRSNRt15+ ateJXZZ+iM3J11eUDXg03tf19lcnRIAbKy+Ws1fGj+y7EeaxhPDONSWj1g+PYQfT1K N62rySS4xkZdX7z48qAIhhyto5u1NAaKkyPmg6rxlZIzwhDqgteTwoCQVaTem9bdn6 lfBwe95QSIsxTL4syDhDSFvz8sudXw5iqq+Pl95T5jlaC+iT0EEOG10sg4r4RVtZLP 3fx4xA63tNP5IuulUMlhRZB6JwxTK0rf+Bhcpbqx6F46rJ6f2SDNZO//ocUV/8yiWH OrM13adxv2scOn9cVoMVKwsfONGAhGxCpwoTWAn+i/W6NtpfCm7Y7yZsXoxZfFlAAz 3vrPPAzKoObxTkmMOlek47MA= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Tue, 20 Sep 2022 14:47:47 -0700 Message-ID: Date: Tue, 20 Sep 2022 14:47:43 -0700 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.12.0 Subject: Re: zfs replication tool Content-Language: en-US To: questions@freebsd.org References: <20220916133046.znfelln3fisrjnuz@x1> From: David Christensen In-Reply-To: <20220916133046.znfelln3fisrjnuz@x1> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4MXFXB0py7z3QGj X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=Ce269jhT; dmarc=pass (policy=none) header.from=holgerdanske.com; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 184.105.128.27 as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; NEURAL_HAM_LONG(-0.99)[-0.988]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; R_SPF_ALLOW(-0.20)[+a]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[holgerdanske.com:+]; ASN(0.00)[asn:6939, ipnet:184.104.0.0/15, country:US]; MLMMJ_DEST(0.00)[questions@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 9/16/22 06:30, Julien Cigar wrote: > Hello, > > I have the case here where I have to atomically snapshot recursively a > dataset with many children (100+) but exclude a dozen of them. > > There are dozen of ZFS replication tools and I was wondering which ones > supports such case and are recommended? > > It looks like "recursivity" in a lot of tools is handled through "zfs > snapshot -r somepool" whereas ZFS also supports "zfs snapshot > somepool/ds1 somepool/ds2 somepool/ds3/child1" which is also atomic I use the zfs(8) program from a terminal to recursively create snapshots via 'zfs snapshot -r'. RTFM I do not see an exclude option. Perhaps you should file a feature request. (I also use zfs-auto-snapshot from crontab(5). AIUI it is a Ruby script that automates the 'zfs snapshot' and 'zfs destroy' commands. It is configured via command-line options and dataset properties.) The KISS solution is to organize the datasets into two hierarchies -- one for recursive snapshots and another for without. This will also facilitate incremental replication via 'zfs send -R' with the '-i' or '-I' options. A brute force approach would be to do a recursive snapshot, followed by destroying the snapshots you do not want. But, beware -- incremental replication of dataset hierarchies with missing snapshots can be problematic. Another idea is to to halt operations on the hierarchy, create snapshots of the desired datasets, and then resume operations on the hierarchy. Of course, this means coordinating with every process that uses the datasets. The dataset 'readonly' property could be used as a fail-safe while creating snapshots. Again, beware replication issues. David