From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 18:46:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41D1848A; Thu, 24 Jul 2014 18:46:26 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 107C92A37; Thu, 24 Jul 2014 18:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=CKplyrXSnxOQ8eYWV5hIc4WxW1JzBSOm3KqymhXNLow=; b=t2yZCQ3kXYCdNw19kCMuqsQ4mCV4P+DjSl3QcCmb4DAhJE9oKDz0g8UXGU7uZXHEZoKXopXXJpjY56OH6xkvxYl3w9NqhRaaSDYN2AS/PuUftpenluNmrjLdnj/qIEGTR+owYfYb+VhgTltOmQouOouA6Dw2lKH1lfU9DZguFbI=; Received: from localhost.lerctr.org ([127.0.0.1]:45581 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAO23-000HlW-Eh; Thu, 24 Jul 2014 13:46:25 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 13:46:23 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 13:46:23 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D15438.6040105@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> Message-ID: <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 18:46:26 -0000 On 2014-07-24 13:45, Allan Jude wrote: > On 2014-07-24 14:25, Larry Rosenman wrote: >> On 2014-07-24 12:43, Larry Rosenman wrote: >>> On 2014-07-24 12:38, Allan Jude wrote: >>>> On 2014-07-24 13:33, Larry Rosenman wrote: >>>>> TRYING to use zfs send/recv between a 10-STABLE and an 11-CURRENT >>>>> system, and receive the non-descript >>>>> "invalid backup stream". >>>>> >>>>> borg.lerctr.org /home/ler $ sudo bin/backup-TBH-ZFS-initial.sh >>>>> Password: >>>>> receiving full stream of zroot@2014-07-24 into >>>>> zroot/backups/TBH@2014-07-24 >>>>> received 41.7KB stream in 300 seconds (142B/sec) >>>>> receiving full stream of zroot/usr@2014-07-24 into >>>>> zroot/backups/TBH/usr@2014-07-24 >>>>> received 41.7KB stream in 1 seconds (41.7KB/sec) >>>>> receiving full stream of zroot/usr/local@2014-07-24 into >>>>> zroot/backups/TBH/usr/local@2014-07-24 >>>>> received 2.81GB stream in 1116 seconds (2.58MB/sec) >>>>> receiving full stream of zroot/usr/src@2014-07-24 into >>>>> zroot/backups/TBH/usr/src@2014-07-24 >>>>> cannot receive new filesystem stream: invalid backup stream >>>>> borg.lerctr.org /home/ler $ cat bin/backup-TBH-ZFS-initial.sh >>>>> #!/bin/sh >>>>> DATE=`date "+%Y-%m-%d"` >>>>> #DATE2=2013-03-24 >>>>> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >>>>> # snap the source >>>>> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >>>>> # zfs copy the source to here. >>>>> ssh root@tbh.lerctr.org "zfs send -R -D zroot@${DATE} | \ >>>>> ssh home.lerctr.org \"zfs recv -F -u -v -d >>>>> zroot/backups/TBH\"" >>>>> # make sure we NEVER allow the backup stuff to automount. >>>>> /sbin/zfs list -H -t filesystem -r zroot/backups/TBH| \ >>>>> awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh >>>>> borg.lerctr.org /home/ler $ >>>>> >>>>> This has been happening for YEARS and I can't seem to interest >>>>> anyone in >>>>> fixing it. >>>>> >>>>> How can we get to the bottom of this? >>>>> >>>>> borg.lerctr.org /home/ler $ uname -a >>>>> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #56 >>>>> r268982M: >>>>> Tue Jul 22 10:14:59 CDT 2014 >>>>> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >>>>> borg.lerctr.org /home/ler $ ssh tbh uname -a >>>>> FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #39 >>>>> r269019M: Wed Jul 23 11:44:35 CDT 2014 >>>>> root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 >>>>> borg.lerctr.org /home/ler $ >>>>> >>>> >>>> Try adding -v to the 'zfs send' and see if it gives you more detail. >>>> >>>> Can you also try this script for the replication: >>>> >>>> http://github.com/allanjude/zxfer >>> I've done that in the past and nothing, but I will try again. >>> >>> I will also look at zxfer.... :) >> >> >> with the -v, no more info, just what looks like normal messages.... >> >> 13:23:55 3.68G zroot/usr/src@2014-07-24 >> 13:23:56 3.68G zroot/usr/src@2014-07-24 >> 13:23:57 3.68G zroot/usr/src@2014-07-24 >> 13:23:58 3.68G zroot/usr/src@2014-07-24 >> 13:23:59 3.68G zroot/usr/src@2014-07-24 >> 13:24:00 3.69G zroot/usr/src@2014-07-24 >> 13:24:01 3.69G zroot/usr/src@2014-07-24 >> 13:24:02 3.69G zroot/usr/src@2014-07-24 >> 13:24:03 3.69G zroot/usr/src@2014-07-24 >> 13:24:04 3.69G zroot/usr/src@2014-07-24 >> 13:24:05 3.70G zroot/usr/src@2014-07-24 >> 13:24:06 3.70G zroot/usr/src@2014-07-24 >> 13:24:07 3.70G zroot/usr/src@2014-07-24 >> 13:24:08 3.70G zroot/usr/src@2014-07-24 >> 13:24:09 3.70G zroot/usr/src@2014-07-24 >> 13:24:10 3.71G zroot/usr/src@2014-07-24 >> 13:24:11 3.71G zroot/usr/src@2014-07-24 >> cannot receive new filesystem stream: invalid backup stream >> borg.lerctr.org /home/ler/bin $ > > I notice you are doing a deduplicated stream. Does it work without > deduplication (zfs send -D)? I will try that after this zxfer test I'm running finishes, but IIRC it doesn't matter.. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688