From owner-freebsd-fs@FreeBSD.ORG Wed Oct 1 13:57:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A6AFE42 for ; Wed, 1 Oct 2014 13:57:26 +0000 (UTC) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18DCBA13 for ; Wed, 1 Oct 2014 13:57:26 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id bj1so324841pad.15 for ; Wed, 01 Oct 2014 06:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:message-id:mime-version:subject:date:references :to:in-reply-to; bh=FG32i/v81sqjEf6yNO9uRXRDP7awwnN7GHREPlvN14o=; b=KQtsK1ytMAtZESYRHxw4U0E4pzNAfDfyfaA97An+S6WYqlBDXI0wGQiCBPwRxekOKM kctEAZ1fZ227qidDzPu7DSwndatqTaU3MAI8/PRcRjTC+AknIKCgFa3D3lnyYyyGeEeP rZPAi2KTkGEAoMrNGhNENRBYsU3Utt1HKOW+tNpwf+fKrL+geDY23I9haSICIjEfpYWJ A8WlrZU13ACjAXI7EPS3hAFt+bVIwLfhjIMDSdifG2ZfoVPXwPu9AmiaS3Ftp/cn5nEs g47BQ6fVcU/TP/cz80m9FFYC6UbqjH9F9VsX53KtiGe2KqNpvGfPmjulnOEOI+sN1jja bbuQ== X-Received: by 10.66.220.230 with SMTP id pz6mr77419409pac.145.1412171845599; Wed, 01 Oct 2014 06:57:25 -0700 (PDT) Received: from [10.0.180.25] (50-46-123-160.evrt.wa.frontiernet.net. [50.46.123.160]) by mx.google.com with ESMTPSA id i10sm1153489pdn.26.2014.10.01.06.57.23 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 01 Oct 2014 06:57:24 -0700 (PDT) From: John Terrell Message-Id: <2CAB03AD-B755-4F85-A6CA-E30B76517269@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: zfs-send/receive between FreeBSD10 and SmartOS (Illumos) fails. Date: Wed, 1 Oct 2014 06:57:22 -0700 References: <37BD986A-9B05-4CE8-9D55-CAB4B2713D9D@gmail.com> <99C93627-2B9C-423F-9C3D-D8B6ACB9C74A@gmail.com> <5C3F3898-9C40-48A6-9B8D-C2118448BE42@gmail.com> To: "freebsd-fs@freebsd.org" In-Reply-To: X-Mailer: Apple Mail (2.1878.6) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2014 13:57:26 -0000 Eureka! I think I=92ve found the issue and it=92s nothing to do with = ZFS - it=92s an infrastructure problem. I was running another test = last night and just happened to be watching about two hours into the = test when the Mac I use to connect to the network suddenly (and = unexpectedly) lost connectivity for about five seconds. When it came = back, the SSH connection was intact but the ZFS transfer in progress = immediately failed. I=92m not sure if this is a DHCP lease = re-aquisition or what but to see if this is indeed the problem, I gave = the Mac a manual IP address and started the transfer over night. As of = now, it=92s been running for about 7 hours without issue. I=92m not = quite ready to call it solved until this transfer completes successfully = but this is the furthest it=92s gotten. On Sep 30, 2014, at 8:02 PM, John Terrell = wrote: > Sure. On the SmartOS (receiving) side: >=20 > [root@smartos /]# zpool get all zones | grep feature@ > zones feature@async_destroy enabled = local > zones feature@empty_bpobj active = local > zones feature@lz4_compress active = local > zones feature@multi_vdev_crash_dump enabled = local > zones feature@spacemap_histogram active = local > zones feature@enabled_txg active = local > zones feature@hole_birth active = local > zones feature@extensible_dataset enabled = local > zones feature@embedded_data active = local > zones feature@bookmarks enabled = local > zones feature@filesystem_limits enabled = local >=20 > =85on the FreeBSD 10 (sending) side: >=20 > [root@freebsd /]# zpool get all tank | grep feature@ > tank feature@async_destroy enabled = local > tank feature@empty_bpobj active = local > tank feature@lz4_compress active = local > tank feature@multi_vdev_crash_dump enabled = local >=20 >=20 > So, the FreeBSD side is a subset of the SmartOS side. >=20 >=20 > On Sep 30, 2014, at 7:51 PM, Rich wrote: >=20 >> It's possible that your feature flags and the receiving end's aren't >> compatible - can you show us a zfs get all tank on both sides? >>=20 >> - Rich >>=20 >> On Tue, Sep 30, 2014 at 10:39 PM, John Terrell = wrote: >>> By the way, here's another report that sounds similar to mine = (though on Linux): >>>=20 >>> = http://stackoverflow.com/questions/19754375/why-does-zsh-send-fail-midway >>>=20 >>> His repro doesn't even use ssh (appears to be completely local). >>>=20 >>>=20 >>> On Sep 30, 2014, at 6:41 PM, John Terrell = wrote: >>>=20 >>>> Sorry for the tardy response (busy at work. :)). Here's the = command and output (run from the SmartOS box): >>>>=20 >>>> ssh -c arcfour256 root@freebsdnas zfs send -R tank@daily20140920 | = zfs receive -v -F zones/tank >>>>=20 >>>> The output (after running for a long time xferring data): >>>>=20 >>>> receiving full stream of tank@daily20140920 into = zones/tank@daily20140920 >>>> received 1.35GB stream in 15 seconds (91.9MB/sec) >>>> receiving full stream of tank/photography@daily20140920 into = zones/tank/photography@daily20140920 >>>> Read from remote host nas00: Connection reset by peer >>>> cannot receive new filesystem stream: invalid backup stream >>>>=20 >>>>=20 >>>> Is it possible the command is simply timing out for some reason and = closing the connection? >>>>=20 >>>>=20 >>>> On Sep 26, 2014, at 3:04 PM, Will Andrews = wrote: >>>>=20 >>>>> What do the zfs commands print? The -v option prints some metadata = that might help diagnose the problem. >>>>>=20 >>>>> --Will. >>>>>=20 >>>>> On Monday, September 22, 2014, John Terrell = wrote: >>>>> Hello all, I'm trying to replicate one of the ZFS filesystems that = lives on my FreeBSD 10 NAS to a SmartOS (Illumos based) server. The = transfer dies at about the 1.7TB (of almost 4TB) mark indicating: >>>>>=20 >>>>> "cannot receive incremental stream: invalid backup stream" >>>>>=20 >>>>> The stream being sent is not incremental so I'm not sure what the = receiver is trying to do. Here's the command I'm using to transfer = (executed on the SmartOS machine): >>>>>=20 >>>>> ssh root@fbsd10 zfs send -v -R tank@daily20140920 | zfs receive -v = tank >>>>>=20 >>>>> I've also tried to use mbuffer as the transport interface = (removing SSH from the equation) and it has the same result. >>>>>=20 >>>>> Is the possibly an incompatibility between FreeBSD10 and SmartOS = ZFS implementations? >>>>> _______________________________________________ >>>>> freebsd-fs@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>>>=20 >>>=20 >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >=20