From owner-freebsd-questions@freebsd.org Sat Jun 22 20:28:46 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E78EB15B5C6E for ; Sat, 22 Jun 2019 20:28:45 +0000 (UTC) (envelope-from njt@ayvali.org) Received: from zaph.org (zaph.org [199.48.132.46]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "zaph.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E3F338B2DF for ; Sat, 22 Jun 2019 20:28:43 +0000 (UTC) (envelope-from njt@ayvali.org) Received: by zaph.org (Postfix, from userid 1001) id 7755991A7F; Sat, 22 Jun 2019 13:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ayvali.org; s=zaph; t=1561235315; bh=HkNPmN6LFQNcLxTTqZhw9KThizArnAeKYrIWpXTNrSk=; h=Date:From:To:Subject:References:In-Reply-To; b=EPamPRH4CdABm6uV1pXRDjJ84/p6liNUg/l99LLeNAWm9vViTXbvRVVxCpPPuJX9N 18pAUk2IPin7hDrz8nRixhDSHIXEK/pbikPHPxFt3YuzKDLMmcmiPaTz8iXcjSD2xQ DyadlKi9KvhY5hUpM7jHFsYUU3yzDMQSDYPTK2Ww= Date: Sat, 22 Jun 2019 13:28:35 -0700 From: "N.J. Thomas" To: freebsd-questions@freebsd.org Subject: Re: ZFS replication suggestions Message-ID: <20190622202835.GN43212@ayvali.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: E3F338B2DF X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ayvali.org header.s=zaph header.b=EPamPRH4; dmarc=pass (policy=none) header.from=ayvali.org; spf=pass (mx1.freebsd.org: domain of njt@ayvali.org designates 199.48.132.46 as permitted sender) smtp.mailfrom=njt@ayvali.org X-Spamd-Result: default: False [-6.03 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ayvali.org:s=zaph]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_MED(0.00)[ayvali.org.dwl.dnswl.org : 127.0.6.2]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RBL_COMPOSITE_RCVD_IN_DNSWL_MED_DWL_DNSWL_MED(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-1.54)[ipnet: 199.48.132.0/22(-3.76), asn: 36236(-3.88), country: US(-0.06)]; DKIM_TRACE(0.00)[ayvali.org:+]; RCVD_IN_DNSWL_MED(-0.20)[46.132.48.199.list.dnswl.org : 127.0.6.2]; DMARC_POLICY_ALLOW(-0.50)[ayvali.org,none]; MX_GOOD(-0.01)[khwab.org,zaph.org]; NEURAL_HAM_SHORT(-0.78)[-0.778,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.132.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jun 2019 20:28:46 -0000 * mike tancsa [2019-06-21 15:25:20-0400]: > was wondering what people are using these days to sync their ZFS files > systems between servers? I use a combination of syncoid (the replication subsystem of Sanoid) and zxfer. Syncoid doesn't delete snapshots it has not created itself, so because I use zfstools to create my snapshots, syncoid won't delete these --- I don't consider this a bad thing necessarily, the author is very cautious. So what I do is use syncoid to replicate my snapshots to a backup server (called A), and use zxfer to prune older snapshots. The zxfer runs right after syncoid, so it really only just deleting old snaps. (NB: This is technically not a backup on server A, it's just a sync.). I use syncoid to replicate everything on backup server A to two other backup servers (B and C, with server C being offsite), ie. backups of the backup server. There is no automatic deletion of older snaps on these servers. So this is my real backup. Every couple of months, I'll go in and manually prune older stuff on B and C. I'm told that once ZFS gets above 10,000 snapshots, bad things happen, so I try not to go above 10k snaps on any one partition. Manually pruning by hand about every 2 months seems to work fine. The reason I don't just use zxfer for everything: syncoid uses mbuffer and lzop, so it's very fast, it can also resume partially replicated transfers and is aware of zfs bookmarks, etc. One last thing to note, zfstools and zxfer is in ports. Sanoid is not. I clone sanoid from Github, and use a small patch to fix the shebang line and some minor things in the syncoid script to make it work on BSD (it's written in Perl and assumes Linux). Thomas