From owner-freebsd-net@FreeBSD.ORG Sat Nov 1 20:49:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BF7516D for ; Sat, 1 Nov 2014 20:49:10 +0000 (UTC) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9BF634A for ; Sat, 1 Nov 2014 20:49:09 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 1358 invoked from network); 1 Nov 2014 21:48:59 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1414874939; bh=kE8hbdYiTIlNi0a7IwSUMnzbP22LSh56jgK8Sp5ZptQ=; h=From:To:Subject; b=vVVCTdsgAmVGnHEm+yP4OSlhxaKAny+e5qPBF3PwD4f5yE5XVHXSYrSFLQxG6BP0i etl9BLS4TDpcKRfogUDHh0tdKukj/h4Z8eJzQgWyBlBwulHigXZi8xq/wwUeKHF11T kFVCNXMJ5pMFU0SnqPmqEmVlFF+SPAO8Rya2zDT0= Received: from 250-210-250-178.ftth.cust.kwaoo.net (HELO [10.0.5.10]) (marek_sal@[178.250.210.250]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with ECDHE-RSA-AES256-SHA encrypted SMTP for ; 1 Nov 2014 21:48:59 +0100 Message-ID: <5455472F.7030905@wp.pl> Date: Sat, 01 Nov 2014 21:48:47 +0100 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: Extending TCP window size in FreeBSD 10-RELEASE? Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [MbN0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Nov 2014 20:49:10 -0000 Hi all, There are 2 boxes with FreeBSD 10-RELEASE: FreeBSD storage3 10.0-RELEASE-p10 FreeBSD 10.0-RELEASE-p10 #0: Mon Oct 20 12:42:25 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 They are connected using L2 IPsec VPN iperf with extended window size (iperf -s -w 1400K) shows that the connection bandwidth between hosts is around 4Mbit/s I would like to transmit data from box1 ZFS pool to box2 ZFS pool. I am using the mbuffer to do this: on box2 (dst): # mbuffer -v 5 -4 -I 9090 | zfs receive -Fduv tank1/ze-storage2 on box1(src): # zfs send -R tank1/PROD@20141030 | mbuffer -v 5 -4 -O 172.25.25.85:9090 The transfer speed is around 180KB/s (~2Mbit/s) After Googling, I realised I might need to extend the TCP Window (as mbuffer does not extend TCP Window size). On both boxes, I have tuned the following sysctls: # sysctl kern.ipc.maxsockbuf=16777216 # sysctl net.inet.tcp.sendspace=1048576 # sysctl net.inet.tcp.recvspace=1048576 # sysctl net.inet.tcp.rfc1323=1 However, it doesn't make the speed transmission faster. Changing the mbuffer options to: -s 128k -m 1G Neither helps. Do you have any idea how to saturate the L2 IPSec VPN connection between hosts? regards, Marek -- Marek Salwerowicz