From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:58:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DB5730F for ; Fri, 25 Apr 2014 11:58:00 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00DD41870 for ; Fri, 25 Apr 2014 11:57:59 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 19820 invoked from network); 25 Apr 2014 13:57:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398427073; bh=KGhtPINQ/GZjGenWso+t5UUmPhypV59vqGjvQLZihZ8=; h=From:To:CC:Subject; b=JB53817nk9g2gHvU635zLw7srt63np+zTl/IoYg9WUkE/b7R1Du8HRGAr7CcxiPX6 jt2WzRpmuPJPxTqC3f5t0qEG4OBV8PKn4OypZ8uaOLTOoFRCJ+AVpnD1s7lOP2xhnz 3RjJTuDYPReOJUuYXbEQySp3Wr+0nLeGsj/nXUGw= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 13:57:53 +0200 Message-ID: <535A4DC2.605@wp.pl> Date: Fri, 25 Apr 2014 13:57:54 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rick Macklem Subject: Re: NFS over LAGG / lacp poor performance References: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> In-Reply-To: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [AdOE] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:58:00 -0000 W dniu 2014-04-25 13:48, Rick Macklem pisze: > Well, you don't mention what command(s) you are using to transfer the > data, but I would guess you have one serial data transfer for each command. > (Put another way, if you are only running one command to transfer the data, > there will only be one RPC happening at a time and that will only use one > network interface.) I don't know anything about lagg, so I can't comment > related to it, but if there is only one NFS RPC at a time, you'll only > be transferring one message at a time on the wire.) I need to transfer 15 files, each is about 1TB sized. >From 9.1-RELEASE[storage1] to 10-RELEASE[storage2] I have tried to run concurrent 'cp' and transfer 4 files at the same time: (executed on storage1) # cp -a file1 /net/storage2/ & # cp -a file2 /net/storage2/ & # cp -a file3 /net/storage2/ & # cp -a file4 /net/storage2/ & But in fact I did not observe bigger throughput Both servers have filesystem exported using NFS, so I can execute copy on source, or destination. Would you recommend running this on source-side, or rather destination-side ? > > Adding the mount option "readahead=8" to the machine receiving the data > might help, if the data transfer command is being done there. (ie. The machine > the data is being copied to has the other one NFS mounted and it is where > you are running the data transfer command(s).) Regarding what I wrote above - how should I mount the NFS volumes? Cheers, Marek