From owner-freebsd-questions@freebsd.org Thu Feb 21 13:36:05 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A1EF14E09F2 for ; Thu, 21 Feb 2019 13:36:05 +0000 (UTC) (envelope-from a199e59c87e914e7b5fdb9459865d66e@zxas.fi) Received: from box.zxas.fi (box.zxas.fi [IPv6:2a05:b9c0::1:0:a4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B38C172B46 for ; Thu, 21 Feb 2019 13:35:54 +0000 (UTC) (envelope-from a199e59c87e914e7b5fdb9459865d66e@zxas.fi) Received: from authenticated-user (box.zxas.fi [185.87.111.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by box.zxas.fi (Postfix) with ESMTPSA id D68837FE9F; Thu, 21 Feb 2019 15:35:23 +0200 (EET) Received: from authenticated-user (box.zxas.fi [185.87.111.174]) by zero.my.domain (Postfix) with ESMTP id 607FC33C39; Thu, 21 Feb 2019 15:35:21 +0200 (EET) Received: from authenticated-user (box.zxas.fi [185.87.111.174]) by thunderbolt.my.domain (8.15.2/8.15.2) with ESMTPS id x1LDZJ6n032130 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 21 Feb 2019 15:35:19 +0200 (EET) (envelope-from ejk@thunderbolt.my.domain) Received: from authenticated-user (box.zxas.fi [185.87.111.174]) by thunderbolt.my.domain (8.15.2/8.15.2/Submit) id x1LDZJGF032129; Thu, 21 Feb 2019 15:35:19 +0200 (EET) (envelope-from ejk) Date: Thu, 21 Feb 2019 15:35:19 +0200 From: Esa Karkkainen To: "@lbutlr" , freebsd-questions@freebsd.org Subject: Re: Duplicating file system Message-ID: <20190221133519.GA1554@pp.htv.fi> Mail-Followup-To: Esa Karkkainen , "@lbutlr" , freebsd-questions@freebsd.org References: <0A33E3BE-96C9-4D83-B9F7-D4D2792B5161@kreme.com> <32153EA7-4BC5-4EE2-98FA-5BDEE1903BA3@cretaforce.gr> <4CAB4BC4-0473-41CF-AF03-D1CE796F5545@cretaforce.gr> <0F4BBC09-3F9D-4740-A8E4-BB3B87F2A657@kreme.com> <20190221032121.c0006dfe.freebsd@edvax.de> <20190221075515.c815b269.freebsd@edvax.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190221075515.c815b269.freebsd@edvax.de> X-Rspamd-Queue-Id: B38C172B46 X-Spamd-Bar: - X-Spamd-Result: default: False [-1.83 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-0.995,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.02)[country: FI(-0.09)]; NEURAL_HAM_MEDIUM(-0.99)[-0.985,0]; NEURAL_SPAM_SHORT(0.17)[0.174,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[zxas.fi:~]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[zxas.fi,quarantine]; R_DKIM_PERMFAIL(0.00)[zxas.fi:s=mail]; FORGED_SENDER(0.30)[freebsd.lists@zxas.fi,a199e59c87e914e7b5fdb9459865d66e@zxas.fi]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:201057, ipnet:2a05:b9c0::/29, country:FI]; FROM_NEQ_ENVFROM(0.00)[freebsd.lists@zxas.fi, a199e59c87e914e7b5fdb9459865d66e@zxas.fi] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2019 13:36:05 -0000 On Thu, Feb 21, 2019 at 07:55:15AM +0100, Polytropon wrote: > On Wed, 20 Feb 2019 20:59:21 -0700, @lbutlr wrote: > > I will try cpdup as I have tried rsync already and it is too > > resource intensive to run it constantly. Even with a 5 minute > > delay sometimes a task will not finish before the next one > > starts and then a cascade will bring the system down as more > > and more processes get stuck. FWIW, kludgey way of doing this is to run the sync in a "while true" loop, with a sleep at the end, so then you'll have only one sync active at the time. Lock files give also the same result. But then you have to clean up lock files after boot and start duplication process etc. > This makes the scenario even more look like a situation where > a mirrored approach (like software RAID, gmirror) is a good > solution; writes will happen in quasi-parallel and transparent > to the writing process, so those race conditions won't appear. The only negative aspect, that I can think of, having a mirror between filesystem and mass storage device does not protect from filesystem corruption. Most likely the mirrored approach will be less of an hassle. Best regards, Esa -- "In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move." -- Douglas Adams 1952 - 2001