From owner-freebsd-stable@freebsd.org Wed May 27 15:11:37 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C36283350EA for ; Wed, 27 May 2020 15:11:37 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [128.127.146.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 49XDnM6X1Kz4X0W for ; Wed, 27 May 2020 15:11:34 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([94.50.161.126]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id 04RFDKGZ031062 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 27 May 2020 20:13:20 +0500 (+05) (envelope-from emz@norma.perm.ru) To: freebsd-stable@freebsd.org From: "Eugene M. Zheganin" Subject: zfs receive -s: transfer got interrupted, but no token on the receiving side. Message-ID: Date: Wed, 27 May 2020 20:11:19 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Server: localhost X-Rspamd-Scan-Time: 1.47 X-Rspamd-Queue-Id: 49XDnM6X1Kz4X0W X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=perm.ru; spf=pass (mx1.freebsd.org: domain of emz@norma.perm.ru designates 128.127.146.8 as permitted sender) smtp.mailfrom=emz@norma.perm.ru X-Spamd-Result: default: False [6.22 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_SPAM_SHORT(0.91)[0.911]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(0.00)[+a]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[0.996]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; HFILTER_HELO_IP_A(1.00)[elf.hq.norma.perm.ru]; RBL_VIRUSFREE_BOTNET(2.00)[128.127.146.8:from]; DMARC_POLICY_ALLOW(0.00)[perm.ru,none]; HFILTER_HELO_NORES_A_OR_MX(0.30)[elf.hq.norma.perm.ru]; NEURAL_SPAM_LONG(1.01)[1.009]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:57401, ipnet:128.127.146.0/24, country:RU]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[94.50.161.126:received] X-Spam: Yes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 May 2020 15:11:37 -0000 Hello, I have a ZFS dataset about 10T of actual size (may be more) that I need to send over a very laggy connection. So I'm sending it from the shell-script that reattempts to send it after a short timeout, retrieving the send token first. Like that: ===Cut=== #!/bin/sh exitstatus=1 token=`ssh user@server zfs get receive_resume_token data/reference | grep -v SOURCE | awk '{print $3}'` while ([ $exitstatus -ne 0 ]) do ššš zfs send -t $token | ssh -C user@server sudo zfs receive -Fus data/reference exitstatus=$? ššš echo "Send interrupted/ended, sleeping for 5 secs." ššš sleep 5 done ===Cut=== Usually this goes just flawlessly. But this time, due to a low transfer speed and a very laggy connectivity, thus resulting in a send time of several weeks, I got a problem: about one reattempt out of 200 fails with a situation when there's no snapshot on the receiving side (only an incomplete dataset which is definitely smaller than original one), thus meaning that the dataset is incomplete, but there's no token on this dataset. This happened already twice, each time on a different size. How can this even be possible ? Recieving side side: [root@playkey-nas:~]# zfs list -t all NAMEšššššššššššš USEDš AVAILš REFERš MOUNTPOINT dataššššššššššš 4,67Tš 20,8Tšš 128Kš /data data/referenceš 4,67Tš 20,8Tšš 128Kš /data/reference (as you can see there's no snapshot) Sending side and the snapshot: [root@san1:/usr/src]# zfs list -t all | grep data/reference@ver2_5917 data/reference@ver2_5917 44,3Gššššš -š 7,73Tš - Eugene.