From owner-freebsd-questions@freebsd.org Thu Sep 30 23:48:47 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6E2A4672E72 for ; Thu, 30 Sep 2021 23:48:47 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HL91V4dXFz535H for ; Thu, 30 Sep 2021 23:48:46 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 239435C0070 for ; Thu, 30 Sep 2021 19:48:39 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 30 Sep 2021 19:48:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:mime-version:content-type; s= fm3; bh=9tvqFXd6lm5uppAgJT+S75IX8jVSgZ3Q5GiCSL21j3M=; b=tp2JiNyJ QwwwItiP6o+vZt/7PIjt+ad2XrB04gsaThuDrQj5FekSgRWyIed8OK7HhFVqJy5s 0ptRyT8R7nBoXy9tXBDs7fvmvzeVJ5F4s8qCDY1lBHziNaKUjF2rSTr8Oij0OCzz pwPobdfgMDup8pgOnyM+ceZ/7cf4FH/WyTtdZePuM0vEhDG/YvqDXV6jaFdKj8I1 sox11MPSkMF6ZqmtdqwCi8RGPB52YVLcwTYW30holeylOKGrsBqQf2yraXI5na5c ubRrF43xBLD3DHi82Mfs2eRj86cLAuFKRsSb781sIdVWydjIPpo4I883dbOrk2Z4 m0KzKGTUzo+Oqg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=9tvqFXd6lm5uppAgJT+S75IX8jVSg Z3Q5GiCSL21j3M=; b=ePhqIOHRjy1hJEgZCeq80p4iprHJkpSszcqFbo6Z6NUnX 6tYwIjCXzctb8uvaLrjvTPzQT+IuOaXfmfkGCdbRhB1ln+x4A43+m/icLlUqpHR5 UIBVJqtCFZTnw7iB5ZJqXcdl4rXqMsEBAAchcBQc2xc7Lg08kubPueyBOAPox95w XX7TBSNM2ddWtZ9UMiStCOPAQqw4xQbm6Fp/T0lGls7TXuX9wyQVEo30Y62XRxLl vU/9ers+ybKaQSNfAPoNxnYzI0/jUALlROyjZVVy3/utHjnA2x9s+finU9aO3ow9 mWbE7aHn6ngmwiHQs2FfaNtYCnKrMEa7hmcpw85eg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudekhedgvdegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesghdtreertd dtvdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseiihiig shhtrdhnvghtqeenucggtffrrghtthgvrhhnpeevgffhffdtfeekleelhedtjeelvdfhvd egieejveffgfduvdfhteegjeeujeeuieenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvth X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 30 Sep 2021 19:48:38 -0400 (EDT) Date: Fri, 1 Oct 2021 00:48:36 +0100 From: tech-lists To: freebsd-questions@freebsd.org Subject: zfs q regarding backup strategy Message-ID: Mail-Followup-To: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="boN+YW9GFVkJSx36" Content-Disposition: inline X-Rspamd-Queue-Id: 4HL91V4dXFz535H X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm3 header.b=tp2JiNyJ; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=ePhqIOHR; dmarc=none; spf=none (mx1.freebsd.org: domain of tech-lists@zyxst.net has no SPF policy when checking 66.111.4.28) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-6.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.28:from]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm3,messagingengine.com:s=fm3]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[zyxst.net]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; R_SPF_NA(0.00)[no SPF record]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2021 23:48:47 -0000 --boN+YW9GFVkJSx36 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, This is a bit chicken-and-egg for me, in that I might be formulating the wrong question, and so getting wrong answers, so I apologise in advance=20 if I'm being dense. Basically the end result I want is to be in a position at a backup site with full backups that can be restored to the live site if needed,=20 or even to get just one required file from that backup, given the filename= =20 and modification time. The backup site has very much more diskspace than the live site. I know how to make the full backup and to send it. What I'm looking for is incremental (or would it be differential) zfs backups (snapshots).=20 So let's say an incremental/differential backup to the remote site=20 happens every 6hrs. If 10 incremential backups happen after the full one, and on the live service someone loses a few files, later adding another few, maybe a couple of those have the same name as before just with a different modification time, all the while this is happening 24hrs pass. How are the accumulated differential/incremental snaps incorporated into the main backup? Would I need to run something else on the backup server to integrate the snaps with the full backup? Or would all=20 the snaps need to be kept?=20 How would one tell if the required files in this example are in one snap and not in the others? If the policy was to keep everything in backup regardless of whether it was deleted or not from the live service (so the backup is like a=20 write-only representation) I guess the approach would be different=20 to say if the backup just had to be the latest representation of the=20 data on the live server. But if it's "keep everything" then what happens if the live server makes foo.txt then deletes it then makes foo.txt 10 days later in the same place? All the while these (incremental/differential) snaps have been arriving every 6 hrs. Additionally the live site is the old zfs on releng/12.2-p10 and=20 the backup site is with the OpenZFS on stable/13 which might be an issue if they handle things significantly differently. thanks advance for any suggestions or any scripts/tools you can recommend. --=20 J. --boN+YW9GFVkJSx36 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAmFWTMwACgkQs8o7QhFz NAUdHg/+IDQBKwdvpa9M1MpQcswefXin+QwdFJBhrcklnu84pWa2hVZvi/gLNJUe KIGvoTpRHQe8cGOiKoqJs7seazUp8IP7UN90h117feUKAZfBTqtWMl7IjyN4F05Z 9RhudykJ6YUIyVTzmXC/ZZOpaBTENaeIJkthofKekVKyMZhJ/35MBSfWno0GOqbS MzwNxk/g+vMIRHWRnZEB00DceCPwd6qYypnBFBNWojf6Bvc9V4S9muUTvO2ubUFL nr+7/2GqiJvKYC2rk/huYzjHdSR5gF50pEyENDWyw0MDNk6a44T9/ByZokVup3V3 oEUXcBIYlT+AaO93qJbHGZg6aj62l4JXZxX8h/MhuU5cn0WSw/YEI8btNN3m/SI9 L2o54MT3Eev0E1YDcSVuTqVdVdPzKCH82rkbY8jF39XkpVpQG+A9lMYAgMb0oJCM KrrrZ5NOMwqGr+jIZhDllIqTTTxrDDiWrjxdBo9irw5YeIg7OPilLBn4qzX7/If9 EAi0O9BTl2nGt75SkvpIAMcbLwj93CCocCPUDq9G3A5Bc7ww9gNELALHl0eZDIUa F/L5m99KWT+/L9w8n4ZdvWb7p+8JRieYRALMUOB4ee92+P+ogPCSzcLQa7e6uNzi hg9D5FERLbUrKRYVHwKAtyY2hIOFQS5ocdPtBMS0tMNGMQFhuLA= =1jSr -----END PGP SIGNATURE----- --boN+YW9GFVkJSx36--