From owner-freebsd-fs@FreeBSD.ORG Sun Jul 27 16:09:36 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E687A4D; Sun, 27 Jul 2014 16:09:36 +0000 (UTC) Received: from denrei.darkbsd.org (denrei.darkbsd.org [IPv6:2001:41d0:1:f442::1]) by mx1.freebsd.org (Postfix) with ESMTP id E16F12918; Sun, 27 Jul 2014 16:09:35 +0000 (UTC) Received: from denrei.darkbsd.org (localhost [127.0.0.1]) by denrei.darkbsd.org (Postfix) with ESMTP id 5351116CB; Sun, 27 Jul 2014 18:09:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=darkbsd.org; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=selector1; bh=9ziGo4FI2dwOOfTKymTjxF0m1YA=; b=A w0QwQEQN3103MLQL3ihknaG6teSBe6ANpkbT8EK5jULB5ly1llqyPesPzIsr4ZWV e4ym4MGJp+0/JOGOKslA5N1BltvNg5E4LiShoXHzmMucBk1RE6r4e9BnFTS3S+vF VJ3qGONh2qPQXHDga67iXJfN+n8lzpVnahhRx2mqWQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=darkbsd.org; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; q=dns; s=selector1; b=giAu4V+82A77xxK9K4RfHNgPb0K eP0rbKkS96bfPxE5JFZ0Y+9JDp1m+jTByffk1296zdg4k+DUxgniyt5RvzPDpDMo M0YFLxQDU37NYqEkjb6KAJFoA63ZJZg0dgq6yTZ129f/62HuMZT24wPE8qEv6bEw 7qmJg3+sloMs/PgM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=darkbsd.org; h= content-type:content-type:in-reply-to:references:subject:subject :mime-version:user-agent:from:from:date:date:message-id:received :received; s=selector1; t=1406477370; bh=bEB9sTfAWicvBsqpyA0Y7wl bRFvKr+brzF+iesPqNrM=; b=igDWQGHdIrm5ZlHpN8jiNqYKzEi44E3WP7wLAgt igJIHG7k100xz2zaU+rX5YXahZ/FW4gLzmntCwx77RZ4ghgzYDBss/fRQYeKAI2E iwyqBj5XZwz2jXYGiVsPXVFFZX5HT9OrLmsmht8UhQ8aMvC0crKihvf/qEMboJOM TgQQ= X-Virus-Scanned: amavisd-new at darkbsd.org Received: from denrei.darkbsd.org ([127.0.0.1]) by denrei.darkbsd.org (denrei.darkbsd.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id nrSHC5BN46Nf; Sun, 27 Jul 2014 18:09:30 +0200 (CEST) Received: from [IPv6:2001:470:24:42d::42] (archer.yomi.darkbsd.org [IPv6:2001:470:24:42d::42]) (Authenticated sender: darksoul@darkbsd.org) by denrei.darkbsd.org (Postfix) with ESMTPSA id 2BED416CA; Sun, 27 Jul 2014 18:09:26 +0200 (CEST) Message-ID: <53D52434.7030308@darkbsd.org> Date: Mon, 28 Jul 2014 01:09:24 +0900 From: DarkSoul User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Vladislav Prodan , questions@freebsd.org Subject: Re: Say me, please, how I can transfer between servers ZFS-partitions larger than 20Gb. References: <1406471619.238727352.o7o2ccyx@frv35.fwdcdn.com> In-Reply-To: <1406471619.238727352.o7o2ccyx@frv35.fwdcdn.com> X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="5jXL8L7B7ukTrP2NGVQMs1IExhQWAeuPa" Cc: fs@freebsd.org, "net@freebsd.org net@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jul 2014 16:09:36 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --5jXL8L7B7ukTrP2NGVQMs1IExhQWAeuPa Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 07/27/2014 11:45 PM, Vladislav Prodan wrote: > Say me, please, how I can transfer between servers ZFS-partitions large= r than 20Gb. > I have 4 partitions with backup data. There are a lot of small pictures= =2E > > NAME USED AVAIL REFER MOUNTPOINT > tank2 1,42T 1,32T 118M legacy > ... > tank2/ay-files 31,9G 1,32T 31,8G /backup/beas= tie/upload_ftp/ay/files > tank2/XXXXXmarket-images 135G 1,32T 135G /backup/beas= tie/upload_ftp/XXXXXmarket/images > tank2/XXXXXmarket-uploads 17,6G 1,32T 17,5G /backup/beas= tie/upload_ftp/XXXXXmarket/uploads > tank2/YYYYmarket 24,4G 1,32T 152K /backup/beas= tie/upload_ftp/YYYYmarket > tank2/YYYYmarket/uploads 24,4G 1,32T 24,2G /backup/beas= tie/upload_ftp/YYYYmarket/uploads > ... > > Say me, please, how I can transfer it at another server with ZFS. > Servers are connected through unmanaged 100M switch. > Sending via (zfs send) and (ssh host2 zfs recv) ends with error after 2= 0GB of data. > Sending via rsync is very lasting and requires often restart of rsync. > > What an alternative? For the initial 20G upload, you could zfs send to a file, and transfer that file via rsync or other resumable file transfer system that ensures integrity : host1# zfs send tank2/XXXXXmarket-images@snapshot > tank2-XXXXXmarket-images@snapshot.zfs host1# rsync tank2-XXXXXmarket-images@snapshot.zfs host2: host2# cat tank2-XXXXXmarket-images@snapshot.zfs | zfs recv tank3 Then the rest can be done via small snapshots and plain zfs send | ssh host2 zfs recv. You could even add bzip2 compression and decompression at each endpoint. This is what I am doing to replicate data of often updated filesystems (mailbox filesystems, with LOTS of small files) between ZFS servers. Cheers, > --=20 > Stephane LAPIE, EPITA SRS, Promo 2005 > "Even when they have digital readouts, I can't understand them." > --MegaTokyo > > --=20 > Stephane LAPIE, EPITA SRS, Promo 2005 > "Even when they have digital readouts, I can't understand them." > --MegaTokyo --5jXL8L7B7ukTrP2NGVQMs1IExhQWAeuPa Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlPVJDQACgkQDJ4OK7D3FWR20wEAlEwxKf4EqYiLuYSTLua3Pkdo vGLQncD91yq63gc2CSoA/3skYXgw/ADY6Fq77LHEHxBnSeWhJYfID6Kd/pPtWy9B =1/W9 -----END PGP SIGNATURE----- --5jXL8L7B7ukTrP2NGVQMs1IExhQWAeuPa--