From owner-freebsd-current@FreeBSD.ORG Mon Nov 3 17:59:24 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCB5CA80; Mon, 3 Nov 2014 17:59:24 +0000 (UTC) Received: from mail-oi0-x235.google.com (mail-oi0-x235.google.com [IPv6:2607:f8b0:4003:c06::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 87DF025D; Mon, 3 Nov 2014 17:59:24 +0000 (UTC) Received: by mail-oi0-f53.google.com with SMTP id a141so7250037oig.26 for ; Mon, 03 Nov 2014 09:59:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0wkmiQBafCnOwYjzlPlp3LcwCNCaPHNW5LmT5pYOsSs=; b=ppWrPEPtMMrMNpNWA2A/a4yTjhE0RSWkrj9gdI6lr7ri2O36p3ukOPQ0pMQOFOTz7B cHJWWoU+ZtBMJaeSPmC/r0rTWVAwq0pbU/afviv38c6RXu+xb17Q0jatHUlySzAPaK4m W4hY3BSMR0rcfgRgtE5z0RAH9Lg0KVFd5dEQarKuPiza2bloypzzLGhp9vN4jIHgAmA+ S/5tvWE1svpeGIraR2BgBvrvY9OGhogKu1XpJArszpPfKZJlZR1KP7VczuaxdY5vA+SU 08wkOa0RuBZkh/GyxiaEflTnM2EOcme/DBkbMm0pAd6BoRdYSHyHApSWXch73EoybnLM 0mgA== MIME-Version: 1.0 X-Received: by 10.202.172.12 with SMTP id v12mr5137590oie.48.1415037563771; Mon, 03 Nov 2014 09:59:23 -0800 (PST) Received: by 10.202.6.21 with HTTP; Mon, 3 Nov 2014 09:59:23 -0800 (PST) In-Reply-To: <5442C040.4090200@freebsd.org> References: <5441E834.2000906@freebsd.org> <544246E8.1090001@ijs.si> <5442C040.4090200@freebsd.org> Date: Mon, 3 Nov 2014 09:59:23 -0800 Message-ID: Subject: Re: ssh None cipher From: Freddie Cash To: Allan Jude Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD-Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2014 17:59:25 -0000 On Sat, Oct 18, 2014 at 12:32 PM, Allan Jude wrote: > On 2014-10-18 13:21, Freddie Cash wrote: > > On Oct 18, 2014 3:54 AM, "Mark Martinec" > > wrote: > >> > >> If the purpose of having a none cipher is to have a fast > >> file transfer, then one should be using sysutils/bbcp > >> for that purposes. Uses ssd for authentication, and > >> opens unencrypted channel(s) for the actual data transfer. > >> It's also very fast, can use multiple TCP streams. > > > > That's an interesting alternative to rsync, scp, and ftp, but doesn't > help > > with zfs send/recv which is where the none cipher really shines. > > > > Without the none cipher, SSH becomes the bottleneck limiting transfers = to > > around 400 Mbps on a gigabit LAN. With the none cipher, the network > becomes > > the bottleneck limiting transfers to around 920 Mbps on the same gigabi= t > > LAN. > > > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > > Actually, looking into it, the bbcp command can support a pipe at each > end instead of files, so you can actually do a zfs send | zfs receive > via bbcp, and use multiple concurrent connections, to get around TCP > window stuff when going transatlantic > > I am going to be trying it out shortly. > > Note: the other big improvement in newer ssh is the HPN stuff, that is > switched on since 9.2 I think. =E2=80=8BAfter much finagling and testing, I have managed to incorporate bb= cp into my ZFS send/recv script. And it works much better than regular, encrypted SSH in my setup. Regular SSH transfers tended to top out around 400 Mbps, using 100% of 1 CPU. Was not able to get the multi-threadded AES cipher working. SSH connections using the NONE cipher saturated the gigabit link with minimal CPU usage. And a bbcp connection is currently running between 500-800 Mbps (depending on the size of the snpashot), also with minimal CPU usage. NOTE: I expect this be running much better next week, as the receiving pool is currently resilvering a drive, slowing everything down. Got things working using the following bbcp command format: bbcp -N io "zfs send -I pool/fs@snap1 pool/fs@snap2" username@remotesys:"zf= s recv -d pool" Have not played with any of the myriad tuning options for bbcp. Just wanted to see if I could get it to work, and how an untuned connection compared to an untuned SSH connection (with and without NONE cipher). So far, I'm impressed. =E2=80=8BThanks for the suggestion. It's another tool in the box. :)=E2= =80=8B --=20 Freddie Cash fjwcash@gmail.com