From owner-freebsd-questions@freebsd.org Sat Jun 22 03:21:54 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4F4215C9A1B for ; Sat, 22 Jun 2019 03:21:54 +0000 (UTC) (envelope-from hartzell@alerce.com) Received: from corvid.alerce.com (corvid.alerce.com [206.125.171.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D8A88DD39 for ; Sat, 22 Jun 2019 03:21:53 +0000 (UTC) (envelope-from hartzell@alerce.com) Received: from postfix.alerce.com (76-226-160-236.lightspeed.sntcca.sbcglobal.net [76.226.160.236]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by corvid.alerce.com (Postfix) with ESMTPSA id B5F8A284FE; Fri, 21 Jun 2019 20:21:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alerce.com; s=dkim; t=1561173704; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nWqJu4Ye+BAmXcEAo3at2VieyOGZSVBv9/X6kDfDm6o=; b=e0z9vyt11/68pW50oCIK/0wxm4nyX7fgu6RUkq1FkIsfEsZtN8mEFvWCEeVd/2se0EdxzB QhE39B8HEM92ZPDhOg9xfWbtnAK4gPkZkA+EAiyvNmNHSIOWig9O4xl/4szhUCVds8us6J vNvLcC9RdJW6FMDKh4xPDcfQITyHmXoqJBMgLIpCemL5IC3CYUIMWfm6DQgJRPoWaG5pOu J3uSau3T0FohZD1njLhi2vre2bdSGTuf7/xnMIxQVT4sKa6lYqypgPse/2otx9vavMxXac Ihoi9UYRrDfMoG6gO2uG2Sfbx7Pi0VylGaipND0GqYy1sOv4rvncpjZuQspclg== Received: by postfix.alerce.com (Postfix, from userid 501) id 0217F200F24315; Fri, 21 Jun 2019 20:21:43 -0700 (PDT) From: George Hartzell MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Message-ID: <23821.40647.901447.793730@alice.local> Date: Fri, 21 Jun 2019 20:21:43 -0700 To: mike tancsa Cc: freebsd-questions@freebsd.org Subject: Re: ZFS replication suggestions In-Reply-To: References: X-Mailer: VM undefined under 26.1 (x86_64-apple-darwin14.5.0) Reply-To: hartzell@alerce.com X-Rspamd-Queue-Id: 5D8A88DD39 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=alerce.com header.s=dkim header.b=e0z9vyt1; dmarc=pass (policy=none) header.from=alerce.com; spf=pass (mx1.freebsd.org: domain of hartzell@alerce.com designates 206.125.171.163 as permitted sender) smtp.mailfrom=hartzell@alerce.com X-Spamd-Result: default: False [-4.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[alerce.com:s=dkim]; HAS_REPLYTO(0.00)[hartzell@alerce.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_ADDR_EQ_FROM(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[alerce.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[alerce.com,none]; MX_GOOD(-0.01)[corvid.alerce.com]; NEURAL_HAM_SHORT(-0.82)[-0.820,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:25795, ipnet:206.125.168.0/21, country:US]; IP_SCORE(-0.99)[ipnet: 206.125.168.0/21(-4.69), asn: 25795(-0.22), country: US(-0.06)] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jun 2019 03:21:54 -0000 mike tancsa writes: > I have a new project coming up with replication.=A0 In the past, I h= ave > used various scripts out of ports, but was wondering what people are= > using these days to sync their ZFS files systems between servers ?=A0= It > will be about 10-40G in daily diffs.=A0 Would like to send every 5-1= 0 min > or so. Files being slightly out of sync for a short period of time i= s > not a big deal eg. in case of total failure on one box (e.g both pow= er > supplies die) missing some data for a period of short time is tolera= ble I've been playing with [sanoid] for generating/pruning snapshots on a remote machine, `syncoid` from the same package for "pull" syncing from the remote machine to a FreeNAS system, and [sanoid] on the local backup server solely to prune the snapshots. It's working reasonably well (e.g. I just got a weird email (cron) about it failing to remove something, but it's gone on the next iteration, so....). I've submitted [a bug report][new-ports] that contains a release port and a devel port. Any help getting it committed would be most welcome. I also have both ports in my ["personal ports tree"][my-ports], it's intended to be layered over the full ports tree with e.g. portshaker. g. [sanoid]: https://github.com/jimsalterjrs/sanoid [new-ports]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238584= [my-ports]: https://github.com/hartzell/freebsd-ports/tree/master/sysut= ils